1. NAME intelligentmirror - Squid url rewriter plugin for caching rpm/deb packages. 2. SYNOPSIS To be used to with squid as a url rewriter plugin. 3. DESCRIPTION IntelligentMirror can be used to create a mirror of static HTTP content on your local network. When you download something (say a software package) from Internet, it is stored/cached on a local machine on your network and subsequent downloads of that particular software package are supplied from the storage/cache of the local machine. This facilitate the efficient usage of bandwidth and also reduces the average download time. 4. DEPENDENCIES 1. squid >= 2.6 2. python >= 24 3. python-urlgrabber 4. python-iniparse 5. httpd 5. CONFIGURATION To configure intelligentmirror, you need to configure Proxy Server (squid) and the global intelligentmirror configuration file. Below are the steps to configure intelligentmirror. 5.1 Squid Configuration To configure squid to use intelligentmirror as a url rewriter, you need to add the following lines to your sqiud configuration file (generally found at /etc/squid/squid.conf ) # --BEGIN-- intelligentmirror config for squid # url_rewrite_program /usr/bin/python /etc/squid/intelligentmirror/intelligentmirror.py url_rewrite_children 20 acl intelligentmirror urlpath_regex -i \.rpm$ acl intelligentmirror urlpath_regex -i \.deb$ url_rewrite_access allow intelligentmirror url_rewrite_access deny all cache deny intelligentmirror redirector_bypass on # --END-- intelligentmirror config for squid # 5.2 intelligentmirror Global Configuration Config file : /etc/intelligentmirror.conf base_dir Base directory for caching the packages. All other directory paths should be relative to base_dir. Default : /var/spool/squid/intelligentmirror/ temp_dir Directory to store partially downloaded packages. Directory name relative to base_dir. e.g. if you specify temp_dir = tmp, actual path for storing partially downloaded packages would be /var/spool/squid/intelligentmirror/tmp/ . Default : tmp proxy Proxy for http, https, ftp content. Example : proxy = http://:/ or http://proxy.example.com:3128/ Default : http://localhost.localdomain:3128/ proxy_username If the above proxy requires authentication, please specify the username. Default : proxy_password If the above proxy requires authentication, please specify the password. Default : cache_host The hostname or IP address of the system on which caching is being done. This is used for serving the packages from the cache. The cached packages will be available at http://cache_host/intelligentmirror/ . Please don’t use http:// or slashes (/). Just specify the domain name or IP address. Example : localhost.localdomain or proxy.example.com or 192.168.36.204 or Default : localhost.localdomain rpc_host XMLRPCServer is used for memory sharing across different instances of intelligentmirror. Leave these settings as it is if you don’t have a fair idea of XMLRPCServer. This will be same as cache_host in almost all the cases. Default : localhost.localdomain rpc_port Please make sure this port is not currently in use. If it is in use by some other program, change this to some port above 1024 which is not in use by any other program. Default : 9000 max_parallel_downloads The maximum number of parallel downloads allowed. If all connections are consumed, packages will be queued for download. Default : 30 logfile File where intelligentmirror log will be stored. Default : /var/log/squid/intelligentmirror.log max_logfile_size Maximum size of logfile specified above. The size is in mega bytes. Note : Please don’t use max_logfile_size = 10MB. Don’t append MB. Default : 10 max_logfile_backups The logfiles are automatically rotated once they have exceeded the max_logfile_size. max_logfile_backups is the number of backup files you want to keep. e.g. max_logfile_backups = 2 will keep intelligentmirror.log and intelligentmirror.log.1 and intelligentmirror.log.2 as logfiles. Default : 10 enable_rpm_cache This option enables the caching of rpm packages. It is assumed that all rpm packages will have a .rpm extension. This option’s value can be either 0 or 1. Default : 1 rpm_cache_dir Directory to cach rpm packages. Directory name relative to base_dir. e.g. if you specify rpm_cache_dir = rpm, actual path for caching rpm packages would be /var/spool/squid/intelligentmirror/rpm/ . Default : rpm rpm_cache_size Maximum total size of caching directory. 0 for unlimited. This size is in Mega Bytes. e.g. rpm_cache_size = 200 will limit your cache size to 200MB. NOTE : Don’t use rpm_cache_size = 200MB. Don’t append MB. Default : 0 max_rpm_size RPMs with size more than max_rpm_size will not be cached. This size is in kilo bytes. Use 0 for unlimited caching. e.g. max_rpm_size = 10000. Only rpms with size less than 10MB (10000KB) will be cached. NOTE : Don’t use max_rpm_size = 10000KB. Don’t append KB. Default : 0 min_rpm_size RPMs with size less than min_rpm_size will not be cached. This size is in kilo bytes. Use 0 for no lower limit. e.g. min_rpm_size = 10. Only rpms with size more than 10KB will be cached. NOTE : Don’t use min_rpm_size = 10KB. Don’t append KB. Default : 0 enable_deb_cache This option enables the caching of debian packages. It is assumed that all debian packages will have an .deb extension.This option’s value can be either 0 or 1. Default : 1 deb_cache_dir Directory to cach debian packages. Directory name relative to base_dir. e.g. if you specify deb_cache_dir = deb, actual path for caching deb packages would be /var/spool/squid/intelligentmirror/deb/ . Default : deb deb_cache_size Maximum total size of caching directory. 0 for unlimited. This size is in Mega Bytes. e.g. deb_cache_size = 200 will limit your cache size to 200MB. NOTE : Don’t use deb_cache_size = 200MB. Don’t append MB. Default : 0 max_deb_size DEBs with size more than max_deb_size will not be cached. This size is in kilo bytes. Use 0 for unlimited caching. e.g. max_deb_size = 10000. Only debs with size less than 10MB (10000KB) will be cached. NOTE : Don’t use max_deb_size = 10000KB. Don’t append KB. Default : 0 min_deb_size DEBs with size less than min_rpm_size will not be cached. This size is in kilo bytes. Use 0 for no lower limit. e.g. min_deb_size = 10. Only debs with size more than 10KB will be cached. NOTE : Don’t use min_deb_size = 10KB. Don’t append KB. Default : 0 6. FILES /etc/intelligentmirror.conf /etc/httpd/conf.d/intelligentmirror.conf /etc/squid/intelligentmirror/ /var/spool/squid/intelligentmirror/ /var/log/squid/intelligentmirror.log 7. SEE ALSO squid (8) Project Website : https://fedorahosted.org/intelligentmirror How to configure squid : http://fedora.co.in/content/how-configure-squid-proxy-server How to write custorm squid redirector in python : http://fedora.co.in/content/writing-custom-redirector-plugin-squid-python 8. AUTHOR Kulbir Saini 9. BUGS Mail bugs/suggestions to Kulbir Saini