5.3. How to tune your system for best Caudium performances

5.3.1. Linux

Until a Linux guru has time to make some nice documentation, here is something that came from PureFtpd software :


 o Increase your system max descriptors numbers :
 
 # echo 60000 > /proc/sys/fs/file-max
 # echo 180000 > /proc/sys/fs/inode-max
 # ulimit -n 60000
 o mount your filesystems with the "noatime" option
 o make sure your disks holding the logs are "fast enough"

 o You can tweak a bit your TCP/IP stack :
 # echo 0 > /proc/sys/net/ipv4/tcp_syncookies
 # echo 0 > /proc/sys/net/ipv4/tcp_ecn
 # echo 0 > /proc/sys/net/ipv4/tcp_timestamps
 # echo 0 > /proc/sys/net/ipv4/tcp_window_scaling
	

Finally don't forget to compile Pike with --with-max-fd=60000 (already done in Debian packages).

WarningWarning from Caudium people
 

  • Personally deactivate window scaling seems to be a bad idea.

  • This needs some touches. 2.2 and 2.4 options are mixed above very badly. Also, turning off ECN might do you more harm than gain.

Also see LinuxPerf, and LinuxPerf kernel tuning section.

5.3.2. FreeBSD

Here are the optimizations you can try on your servers. They are provided without any warranty.

Note

All this is for FreeBSD 4.2 or more recent.

First, check to see if your filesystems use Soft Updates:

# tunefs -p /dev/da0s1a
tunefs: soft updates:  (-n)                                disabled
tunefs: maximum contiguous block count: (-a)               15
tunefs: rotational delay between contiguous blocks: (-d)   0 ms
tunefs: maximum blocks per file in a cylinder group: (-e)  2048
tunefs: average file size: (-f)                            16384
tunefs: average number of files in a directory: (-s)       64
tunefs: minimum percentage of free space: (-m)             8%
tunefs: optimization preference: (-o)                      time
	
If soft updates are set to disabled it may be a good idea to enable them. We do not recommend you enable them on "/" filesystem on servers machines, there are issues and in general this is not recommended by the FreeBSD team. If you can unmount the others filesystems do this and remount them later:

# tunefs -n enable /dev/"whatever"
	
If you cannot unmount the others filesystem, drop into single mode (do a shutdown now or a boot -s) and then type:

# tunefs -n enable "filesystem"
	
I suggest /usr or /var. In the fstab you can add ,async to the options of all filesystems. As for soft updates we do not recommend that on server machines for the root (/) filesystem.

In /boot/loader.conf add the following:

kern.ipc.maxsockets="5000"
kern.ipc.nmbclusters="65536"
	
If you have ATA (IDE / UltraDMA) disks you can add also in /boot/loader.conf:

hw.ata.wc="1"
	
Then in /etc/sysctl.conf you can add:

vfs.vmiodirenable=1
kern.ipc.maxsockbuf=2097152
kern.ipc.somaxconn=8192
kern.ipc.maxsockets=16424
kern.maxfiles=65536
kern.maxfilesperproc=32768
net.inet.tcp.rfc1323=1
net.inet.tcp.delayed_ack=0
net.inet.tcp.sendspace=65535
net.inet.tcp.recvspace=65535
net.inet.udp.recvspace=65535
net.inet.udp.maxdgram=57344
net.local.stream.recvspace=65535
net.local.stream.sendspace=65535
	
And then reboot (or for /etc/sysctl.conf use the sysctl(8) tool to setup this by hand.

Another way to have good performance is to make a custom kernel, with the minimum of drivers and processor support. A maxuser size = size of memory (for example you have 512M of RAM, then set the maxusers value in your kernel configuration variable to 512). Add Posix 1003.1b real time extensions to the kernel with:

options         P1003_1B                #Posix P1003_1B real-time extensions
options         _KPOSIX_PRIORITY_SCHEDULING
        
Enable the SMP options (only for FreeBSD 4.x, the FreeBSD 5.0 is now automatic).

For French users, check these URLs :

5.3.3. Solaris 2.x

Most of the optimization values are in /etc/system file. This file is read by the kernel when it is loading. Please notice that the following optimization is focused on Solaris 8 machines with at least 256M of RAM.

* A comment is started by a star '*'
* 
* 1 maxusers per mega of ram. This machine has 512M then maxusers = 512
set maxusers=512
set hires_tick=1
set rlim_fd_max=10000
set rlim_fd_cur=4096
*set tcp:tcp_conn_hash_size=8192
* Don't setup this if you don't have more than 256M of RAM
set bufhwm=48000

* Folling are used to delay page reaping with databases.
set fastscan = 32000
set slowscan = 200
set maxpgio =  280
set lotsfree = 1024
set desfree = 512
set minfree = 128
set autoup = 280

*  Hash buffer sizes during Specweb testing
set tcp:tcp_conn_hash_size = 2097152

* Nic Interface
set hme:hme_adv_100fdx_cap=1
set hme:hme_adv_100hdx_cap=0
set hme:hme_adv_10fdx_cap=0
set hme:hme_adv_10hdx_cap=0
set hme:hme_adv_autoneg_cap=0

* To prevent buffer overflow
set noexec_user_stack=1
set noexec_user_stack_log=1
	
Please notice that Solaris needs fast disks. If you have IDE / UDMA disks, double check they are recent. For example the disks in Ultra 5 / 10 machines are snail slow, and slow down the whole machine when you make read write operation. Please consider getting faster disks.

Also, if you use software raid (Solaris Disk Suite) you will have optimal performance if you have more than one SCSI controller in the machine.

Another good read is the well known document from Adrian Cockroft about tuning Solaris. Sunhelp.org has also a good section about tuning. Finally, you can read a network guide at http://www.sean.de/Solaris/tune.html.