Squid - the cross-platform open source proxy

Per­form­ance is one of the most important aspects of a good website. It’s been proven that longer loading times lead to visitors turning their back on sites before these portals have the chance to convince users to stay with the quality of their content. As vital com­pon­ents of modern user ex­per­i­ences, page speed has been an important Google ranking factor since 2010. Website operators should, therefore, make sure they’re well versed in this topic and optimise the speed of their website. Users have the option of com­press­ing images (Compress images with the help of free tools), ag­greg­at­ing and op­tim­ising code files, or reducing the number of inquiries. A further approach that will provide your web server with more relief involves using reverse proxy servers. This software component acts as an interface between browsers and web servers by pro­cessing browser requests by proxy and in­de­pend­ently de­liv­er­ing cached static content without con­tact­ing the web server. This is es­pe­cially effective when servers have to dy­nam­ic­ally create the pages in the server upon each request but don’t have to change them con­stantly. One of the most popular solutions for setting up such a cache proxy is the free programme, squid.

Published in 1998 the proxy server software, Squid, was ori­gin­ally released by developer Duane Wessel as a free spinoff of ‘Harvest object cache’. At the same time, the com­mer­cial version was also released under the name ‘NetCache’ whose de­vel­op­ment has since been dis­con­tin­ued. Squid is available under the GNU General Public Licence and supports the protocols HTTP, HTTP/2, HTTPS, and FTP among other variants. Squid proxy servers run on most con­ven­tion­al operating systems, such as the various Linux dis­tri­bu­tions, Mac OS X or Windows. Operating the proxy server can either be done via the cor­res­pond­ing command line tool or a graphical user interface, like GAdmin SQUID or SquidMan.

Thousands of website operators take advantage of the caching pos­sib­il­it­ies of the open source proxy practice. For example, Wikipedia has been using Squid proxies for years to deliver its content and relieve its database and web­serv­ers. Fur­ther­more, thanks to the support of HTTPS, Squid is able to take on the con­struc­tion of secured SSL con­nec­tions. Various internet providers worldwide use Squid as a trans­par­ent proxy in order to ensure optimised internet access. Of course, the open source software can also be used to operate a general forward proxy for an in­di­vidu­al client; this would hide a user’s IP address and provide ad­di­tion­al pro­tec­tion to the firewall’s packet filter.  Al­tern­at­ively, Squid is also able to filter in­de­pend­ent packets with the expansion Squid­Guard.

Since its very first version, Squid has been an open source product, which is why there’s no licence required and source text is also freely available. The software can be down­loaded for free and also can be adjusted to meet users’ various diverse needs. However, this only proves to be rarely needed: decades of ex­per­i­ence from the Squid project employee that have been main­tain­ing and de­vel­op­ing the programme on a volunteer basis for years shows the ver­sat­il­ity and speed op­tim­isa­tion that Squid offers. The software also proves con­vin­cing for private use due to its definable access control lists. On the one hand, access to certain content can be blocked or the usable bandwidth can be reduced while on the other hand users are able to seam­lessly analyse the used proxies in order to control the dataflow. An important char­ac­ter­ist­ic of Squid is its high flex­ib­il­ity, which really pays off for larger, more complex networks. Following this, it’s possible to create a cache-proxy set-up by utilising multiple Squid proxies and dis­trib­ut­ing requests to these. This type of or­gan­isa­tion relieves the in­di­vidu­al com­pon­ents and enorm­ously increases the system’s re­li­ab­il­ity . Just as is the case in a Content Delivery Network the in­di­vidu­al reverse proxy servers are able to be located in multiple locations.

The listed security and control functions that can be realised with this set-up make it clear how versatile the software can be. Firstly, Squid is an at­tract­ive option due to its core function as a proxy server for caching data. In order to guarantee that it’s up to date and available, Squid cal­cu­lates its statuses regularly, and there are two potential results for this: the inspected object can either still be up to date (fresh) or it can be no longer up to date (stale). In order avoid the task of having to inspect the entire data set, an algorithm cal­cu­lates how often each in­di­vidu­al object requires veri­fic­a­tion. Here, the following in­form­a­tion is con­sidered into the eval­u­ation:

Until the date X, the object in the cache remains valid:

The following link depicts Squid’s caching algorithm. Simply put, the algorithm is designed to do the following: the Squid proxy server controls the status of an object more often when the object itself is un­der­go­ing more frequent changes. Here, the earliest in­spec­tion period is MIN, i.e. the assigned minimum duration in the cache. When the maximum duration date MAX is reached then Squid has to contact the web server. To this end, the proxy software a GET request with if-Modified-Since entries, including the OBJ_Date. The web server verifies the status of the object and then forwards it.

• The status code 304 (not modified) when the object is unchanged

• Or the status 200 (OK) as well as the unchanged object

This means that data will only be trans­ferred when something has actually changed.

If you want to use a Squid reverse proxy for your web server, then you should first make sure that you have the necessary hardware struc­tures. A caching proxy doesn’t feature any special re­quire­ments in terms of pro­cessing power. It requires instead the proper amount of working and hard drive memory. Nowadays, both com­pon­ents are easily obtained, which is why acquiring these is less a question of price and more one of the right cal­cu­la­tion. Calculate your demand in relation to your web project and allow the potential for growth to be accounted for as well. When pur­chas­ing your hardware make sure to choose modern com­pon­ents like SDD storage, which are cat­egor­ised by quick access time and so allow the best possible speed op­tim­isa­tion of their website.

Generally, you have two options for in­stalling the squid software on your system. The first variety requires Squid to be located in the packet man­age­ment of your used dis­tri­bu­tion. If this is the case, then in­stall­a­tion of the proxy programme is carried out according to the known pattern via the command line. Under Ubuntu, the cor­res­pond­ing command is:

The second path for in­stall­a­tion is carried out by down­load­ing the in­stall­a­tion file. This can be unpacked and compiled using con­ven­tion­al methods (as seen in versions 3.5.20):

With the command:

After this step is finished, start the in­stall­a­tion

An un­of­fi­cial prefab MSI in­stall­a­tion packet has been available for Windows systems since version 3.5; just double click after download to get the process started.

For every published stable version, there’s a de­vel­op­ment version as well as a beta version that contain new features. Both versions are used first and foremost to test this function, which is why users should only resort to using these if they’re able to fully un­der­stand Squid software.

The con­fig­ur­a­tion file, squid.conf allows users to define the type of proxy Squid should act as. These are generally found under /etc or /usr/local/squid/etc or also in the directory that you de­term­ined during in­stall­a­tion. There are already various pre­defined settings that are labeled via comment lines, which begin with hash marks (#). The following para­graphs we’ve compiled some important options that you’ll need in order to setup Squid.

This area allows users to configure IP addresses and ports that are relevant for operation Squid servers. The following entries are intended for the cache proxy. http_port Syntax: http_port [hostname or IP address:]Port number De­scrip­tion: defines the port on which the Squid listens in on HTTP requests. Generally, port 3128 is cited here. If neither hosts names nor IP addresses are available, then the settings are valid for all the in­teg­rated IP addresses. Entering multiple ports is also possible. Example: http_port 192.168.0.1:3128 https_port

Syntax: https_port [IP addresses:]Port number cert=path to SSL cer­ti­fic­ate [key=path to private SSL key] [options] De­scrip­tion: the in­form­a­tion on the HTTPS port is required if the Squid proxy is to receive the SSL or TLS con­nec­tions. The path’s in­form­a­tion for the applied cer­ti­fic­ate (in PEM format) is required. If not private SSL key is given, Squid auto­mat­ic­ally assumes that the PEM file already contains a key. The parameter options allow you to enter ad­di­tion­al options according to OpenSSL doc­u­ment­a­tion. Icp_port

Syntax: icp_port Port number De­scrip­tion: here you can enter the port via which the Squid ICP receives requests (Internet Cache protocol) or UDP packets. One statement is only necessary if you use multiple proxies that are to com­mu­nic­ate with one another. The standard port is 3130; in order to turn off the function enter the parameter 0. Example: icp_port 3130

The caching options allows users to determine, among other things, whether and/or how much working memory your Squid proxy should use for caching purposes. It can also be used to define the minimum and maximum object sizes and the general caching behaviour.

cache_mem

Syntax: cache_mem working memory in MB

De­scrip­tion: with cache_mem you’re allowed to determine the size of the reserved main memory for in-transit objects, hot objects, and negative-cached objects. Given that the data is arranged in blocks of 4KB, the indicated value must also be a multiple of 4KB. Make sure that you do mistake this option with the absolute storage needs of Squid, which isn’t regulated in this way.

Example: cache_mem 256 MB

maximum_object_size

Syntax: maximum object_size object size in KB/MB

De­scrip­tion: this entry gives Squid in­form­a­tion on the size of the object that’s to be cached. The minimum lim­it­a­tion for size can be de­term­ined with minimum_object_size.

Example: maximum_object size 4 MB

In addition to the entries on ports and caching behaviour, the squid server needs in­form­a­tion regarding the directory in which the content and in­cid­ent­al log data should be cached.

cache_dir

Syntax: cache_dir directory type directory path storage space directory amount

De­scrip­tion: define the caching directory as well as its maximum storage capacity in megabytes and the number of dir­ect­or­ies and sub­dir­ect­or­ies with cache_dir. By default, the installed directory type is ufs. Generally, this option is turned off and has to be re­act­iv­ated.

Example: cache_dir ufs/usr/local/squid/var/cache/squid 100 16 256

Cache_log

Syntax: cache_log file path

De­scrip­tion: determine the storage location of your squid proxy server’s log file, which records in­form­a­tion on the software’s behaviour.

Example: cache_log /usr/local/squid/var/logs/cache.log

Finally, you’ll need clearly defined access lists for the ports used by Squid. Here, the following two para­met­ers are key: acl syntax: acl listname list type argument De­scrip­tion: here you have the pos­sib­il­ity to create a details access list for all HTTP, ICP, andTCP con­nec­tions. For a more precise overview of types and options, take a look at the official online guide. Example: acl all src 0.0.0.0 http_access Syntax: http_access allow|deny [!] listname
De­scrip­tion: allow or deny the access to the HTTP port with the pre­vi­ously defined access lists. The ex­clam­a­tion mark at the beginning means that al­loc­a­tions for all the con­nec­tions apply that do not belong to the named list. Example: http_access deny!SSL_portsThe simple solution: a cache proxy