This article will explain how to configure Squid and tweak a FreeBSD box for web caching and conserving bandwidth. Usually, Squid will reduce the traffic 30% or more from normal usage (without squid), and enhance response time.
This configuration and setting has been tested and the Squid FreeBSD boxes are running at several colleges in Sydney, Australia. Each Squid FreeBSD box serves about 100+ nodes.
Squid needs a lot of memory. More is better but 128 MB of memory is good to start with. Squid also needs fast disk storage. Use SCSI drives if you can, though IDE drives will do the job. Also have a FreeBSD 4.5 box setup and running! You will need to recompile the kernel with additional options and components.
You can install Squid using Ports but you can't play with configure options, so I'll cover the steps to install from a tarball.
You can download the Squid source from http://www.squid-cache.org. The lastest version is squid-2.4.STABLE6-src.tar.gz. Run the following commands as root.
# cd /path/to/tarball # tar zxvf squid-2.4.STABLE6-src.tar.gz # cd squid-2.4.STABLE6 # ./configure --enable-delay-pools --enable-ipf-transparent \ --enable-storeio=diskd,ufs --enable-storeio=diskd,ufs \ --disable-ident-lookups --enable-snmp --enable-removal-policies # make all # make installThe explaination of configure script options are below:
# Need for transparent proxy # You need to --enable-ipf-transparent http_port 3128 httpd_accel_host virtual httpd_accel_port 80 httpd_accel_with_proxy on httpd_accel_uses_host_header on # Physical memory / 3 cache_mem 128 MB # Max out Squid I/O perfomance, 15 GB cache and use Squid special diskd but you need to recompile the kernel # To use disk you need to --enable-storeio=diskd,ufs # Reasonable values for Q1 and Q2 are 72 and 64, respectively. # Q1 value must bigger Q2 cache_dir diskd /usr/local/squid/cache 15360 16 256 Q1=72 Q2=64 # You can use normal ufs instead #cache_dir ufs /usr/local/squid/cache 15360 16 256 # I dont want to log anything # The reason is to save some expensive I/O operation. cache_access_log /dev/null cache_store_log none cache_log /dev/null # Cache replacement policy # The heap GDSF policy optimizes object-hit rate by keeping smaller popular # objects in cache, so it has a better chance of getting a hit. It achieves a # lower byte hit rate than LFUDA, though, since it evicts larger (possibly popular) # objects. # The heap LFUDA ( Least Frequently Used with Dynamic Aging ) policy keeps # popular objects in cache regardless of their size and thus optimizes byte hit # rate at the expense of hit rate since one large, popular object will prevent # many smaller, slightly less popular objects from being cached. # You need to --enable-removal-policies cache_replacement_policy GDSF # Standard Access List # I have two subnets, one for student and another one for admin # Modify this according to your network acl all src 0.0.0.0/0.0.0.0 acl manager proto cache_object acl localhost src 127.0.0.1/255.255.255.255 acl outgoing src 192.168.10.2/255.255.255.255 acl student src 192.168.0.0/255.255.255.0 acl admin src 192.168.1.0/255.255.255.0 acl SSL_ports port 443 563 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 563 # https, snews acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT http_access allow manager http_access allow localhost http_access allow outgoing http_access allow student http_access allow admin http_access deny !Safe_ports http_access deny CONNECT !SSL_ports http_access deny all icp_access allow localhost icp_access allow student icp_access allow admin icp_access deny all # Avoid caching cgi scripts acl QUERY urlpath_regex cgi-bin no_cache deny QUERY acl magic_words1 url_regex -i 192.168 acl magic_words2 url_regex -i ftp .exe .mp3 .vqf .tar.gz .gz .rpm .zip .rar .avi .mpeg .mpe .mpg .qt .ram .rm .iso .raw .wav .mov # Delay Pool # For delay pool, you need to --enable-delay-pools delay_pools 2 # I have ADSL 2Mbits line # 2 mbits == 256 kbytes per second # 256 KB/s, 5 KB/s # It means 256 KB/s bandwith for the whole network, but 5 KB/s for each node, which is fair for everybody delay_class 1 2 delay_parameters 1 256000/256000 5000/256000 delay_access 1 allow magic_words2 delay_access 1 allow student delay_access 1 allow admin # -1/-1 means that there are no limits for local traffic. delay_class 2 2 delay_parameters 2 -1/-1 -1/-1 delay_access 2 allow magic_words1 # Cancel download if file is bigger than 1MB reply_body_max_size 1024 KB # snmp stuff acl snmppublic snmp_community public snmp_access allow snmppublic localhost snmp_access deny all # Change to your domain # visible_hostname yourdomain.domain.com # cache_mgr yourname@youremail.com
# mkdir /usr/local/squid/cache # chown nobody:nogroup cache # /usr/local/squid/bin/squid -k parse # /usr/local/squid/bin/squid -z
# add these lines to enable ipfilter ipfilter_enable="YES" ipnat_enable="YES" ipmon_enable="YES" ipfs_enable="YES"
# add this line # I assume rl0 is your internal nic # Redirect everything else to squid on port 3128 rdr rl0 0/0 port 80 -> 127.0.0.1 port 3128 tcp
Consult the Freebsd Handbook for recompiling the kernel and add the following lines. Your kernel must have:
options SYSVMSGYou can set the parameters in the kernel as follows. This is just an example. Make sure the values are appropriate for your system:
options MSGMNB=8192 # max # of bytes in a queue options MSGMNI=40 # number of message queue identifiers options MSGSEG=512 # number of message segments per queue options MSGSSZ=64 # size of a message segment options MSGTQL=2048 # max messages in systemThe following is the explanation of the kernel options from the Squid FAQ:
The messages between Squid and diskd are 32 bytes for 32-bit CPUs and 40 bytes for 64-bit CPUs. Thus, MSGSSZ should be 32 or greater. You may want to set it to a larger value, just to be safe.
We'll have two queues for each cache_dir, one in each direction. So, MSGMNI needs to be at least two times the number of cache_dir's.
I've found that 75 messages per queue is about the limit of decent performance. If each diskd message consists of just one segment (depending on your value of MSGSSZ), then MSGSEG should be greater than 75.
MSGMNB and MSGTQL affect how many messages can be in the queues at one time. Diskd messages shouldn't be more than 40 bytes, but let's use 64 bytes to be safe. MSGMNB should be at least 64*75. I recommend rounding up to the nearest power of two, or 8192.
MSGTQL should be at least 75 times the number of cache_dir's that you'll have.
Also you can tweak the kernel by commenting out unnecessary lines in the kernel config to gain extra perfomance. Then recompile the kernel.
#!/bin/sh
echo -n ' Squid '
case "$1" in
start)
/usr/local/squid/bin/squid -D
;;
stop)
/usr/local/squid/bin/squid -k shutdown
;;
restart)
/usr/local/squid/bin/squid -k reconfigure
;;
*)
echo "Usage: `basename $0` {start|stop|restart}"
;;
esac