Computing Service

Web cache

The University of Cambridge web cache server

What is a web cache?

A web proxy web server (the one we are using is Squid) "sits between" your web browser and the servers from which you are retrieving documents, and acts as an intermediary ("proxy") by fetching documents on your behalf and then passing them to your browser. It can also save copies of the documents (with exceptions e.g. for documents requiring username/password for access) to form a collection ("cache") of documents that are available immediately they are requested, in which case it is referred to as a web cache server. The web cache can handle requests for http, https, ftp, and gopher URLs.

The web cache's current "transitional" status

Most of the historically-important functions of the University's web cache - such as improving web response times and reducing the volume of external network access - are no longer important due to greatly improved national and international network speeds, and changes to the charging arrangements for the University's connection to JANET (the UK academic network) and beyond.

There was, however, one rôle for which the web cache or a suitable alternative was essential. IP addresses from the "CUDN-wide private address" block (172.16.0.0 to 172.30.255.255) are being used widely around the University, but they inherently have no direct access to computers (such as web servers) outside the University. Those addresses comprise one of several address blocks reserved in RFC 1918 for private use within an organisation. Within the CUDN, the other private address blocks are limited to use inside individual departments or colleges, and are not relevant here.

Until January 2010, computers with CUDN-wide IP addresses were dependent on the web cache for access to external web sites - but the web cache could only handle the traditional "core" web protocols, and not the increasingly wide range of protocols which are needed by current web content, many of which are unsuitable for proxying.

From 11 Jan 2010, the centrally-provided web browser proxy configuration scripts have been amended so that all clients (previously only those with global IP addresses) will not use the web cache and instead will attempt to connect direct to web sites. For clients with CUDN-wide private addresses, that is now made possible by the recently-launched NAT (Network Address Translation) service. Clients with global addresses were switched to "direct fetch" in early 2008.

These changes relate to both the explicit configuration of http://www.cam.ac.uk/proxyconfig.pac as the URL from which to load the proxy configuration, and use of the WPAD (Web Proxy Auto-Discovery) protocol via browser options with descriptions like "automatically detect settings".

For more details about the history and transition plans for the web cache, see the slides from some talks given in June 2008: web cache (PDF) and NAT (PowerPoint)

See also the cache shutdown timetable.

When should a computer use the web cache

As of January 2010, no explicit web client (browser and other) configuration is needed with regard to proxy/cache servers, and none should be specified when configuring new client systems. Existing settings to use the centrally-provided automatic configuration options described above may be left in place or removed, but any "manual" configuration of the web cache hostname and port (wwwcache.cam.ac.uk port 80) must be removed before the web cache is finally turned off. The Computing Service will monitor residual use of the web cache resulting from such "manual" configuration, and attempt to get it corrected before eventually turning off the web cache.

It is intended that the central configuration facilities (explicit URL and WPAD) will remain available indefinitely, for the benefit of browsers configured to use them - returning configurations telling the clients to "go direct" for all requests). However, removing such browser configuration may be required eventually, and doing it sooner would have the benefit of avoiding an interaction with www.cam.ac.uk or the WPAD server during browser startup.

The comments above make the assumption that an institution's local network configuration and policies do not affect the situation. If, for example, an institution provided its own local WPAD server or if its staff/students had been instructed to use a different (local) auto-configuration URL, the requirements would very likely be different.

Configuring web browsers to use the cache

See Leaflet G63: How to use the Web proxy server for details of how to configure your web browser to use the web cache (or de-configure it so as not to use the cache).

Reporting problems with the cache server

If you encounter problems with the web cache server, how to report them depends on the type of problem.

If the web cache appears not to be working at all, please inform the Computing Service operators by phoning extension 34666 during normal working hours. Outside working hours, sending details by mail to webmaster@ucs.cam.ac.uk may enable the problem to be resolved sooner, but there is no guarantee of action outside working hours. In either case, please provide whatever information you can about the symptoms of the problem and also which computer (or in which department or college) you saw the problem - as it may not be affecting everyone.

Since it is no longer necessary to use the web cache, the quickest way to avoid any problems reported by it is likely to be to reconfigure the web browser so that it doesn't use the cache (if that's not prevented be the system's configuration).

Web cache performance

Information about web cache load (level of use), performance, etc., are available in the form of (almost) real-time monitoring graphs.

The title of this document is: The University of Cambridge web cache server
URL: http://www.cam.ac.uk/cs/webcache/