deleted file mode 100644

Binary files a/docs/clamav-mirror-howto.pdf and /dev/null differ

deleted file mode 100644

@@ -1,438 +0,0 @@

-%% LyX 1.3 created this file.  For more info, see http://www.lyx.org/.

-%% Do not edit unless you really know what you are doing.

-\documentclass[english]{article}

-\usepackage{times}

-\usepackage[T1]{fontenc}

-\usepackage[latin1]{inputenc}

-\setcounter{secnumdepth}{4}

-\setcounter{tocdepth}{4}

-\usepackage{setspace}

-\onehalfspacing

-

-\makeatletter

-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% User specified LaTeX commands.

-\usepackage[official]{eurosym}

-\usepackage{listings}

-\usepackage{color}

-\lstset{ 

- commentstyle=\color{blue}, 

- keywordstyle=\color{red}, 

- frame=single, 

- breaklines, 

- showstringspaces=false,

- tabsize=2, 

- numbers=left, 

- numberstyle=\tiny, 

- stepnumber=5, 

- numbersep=5pt,

- basicstyle=\normalfont\footnotesize\tt,

- language={[]Perl} 

- } 

-

-\usepackage{babel}

-\makeatother

-\begin{document}

-

-\title{Mirroring the Virus Database}

-

-

-\author{Luca Gibelli}

-

-\maketitle

-Some guidelines for people interested in contributing to the distribution

-of ClamAV virus database.

-

-

-\section{Introduction}

-

-

-\subsection{This doc}

-

-The latest version of this document is always available at http://www.clamav.net/doc/mirrors/.

-Before going any further, please check that you are reading the latest

-version.

-

-Japanese sysadmins can find a translated version of this doc at http://www.orange.co.jp/\textasciitilde{}masaki/clamav/mirror-howto-jp.html

-(not necessarily up to date).

-

-

-\subsection{Who is responsible for the virus database}

-

-The virusdb team take care of reviewing virus signatures, checking

-for new viruses in the wild and committing changes to the virus database

-file.

-

-The updates are released quite often (usually no less than three times

-a week). If you want to be notified whenever the virus database is

-updated subscribe to clamav-virusdb \emph{at} lists.clamav.net . 

-

-Every time the virusdb team updates the database, the ChangeLog will

-be posted to the mailing-list. 

-

-Visit for the list description and archives. 

-

-If you need to contact the virusdb team please write to: virus-team

-\emph{at} clamav.net

-

-

-\subsection{Virus submission}

-

-Whenever you find a new virus which is not detected by ClamAV you

-should send it to the virusdb team by filling the form at http://www.clamav.net/sendvirus.html.

-They will review your submission and update the database so that the

-whole ClamAV user community can take benefit from it.

-

-\textbf{Never} send virus samples to ClamAV mailing-lists or developers

-addresses.

-

-

-\subsection{Getting a copy of the latest virus database}

-

-The most important factor for an antivirus's efficiency is to be up

-to date. ClamAV comes with a tool to update the virus database automatically:

-its name is \emph{freshclam}.

-

-freshclam looks up the TXT record associated with \textit{current.cvd.clamav.net}

-and extracts the latest database version available from the string

-returned. If the local database is outdated, freshclam tries to connect

-to the hostnames listed in freshclam.conf (DatabaseMirror directive).

-If the first server in the list fails or the latest database is not

-available on that mirror (e.g. in case there has been a problem sync'ing

-the mirror), freshclam will sleep for 10 secs and then try again with

-the next one, and so on. 

-

-After freshclam downloads the new database, it sends a notify to clamd

-(if active) to reload the database.

-

-It is important for the machine running ClamAV to be able to make

-DNS lookups and to connect to port 80 of external hosts on Internet

-either directly or through a proxy. There are known problems with

-some transparent proxies caching what they shouldn't cache. If you

-should run into this kind of problem, please check your proxy configuration

-before reporting a bug.

-

-

-\section{Mirroring the database}

-

-

-\subsection{The need for mirrors}

-

-To prevent the spread of worms it is essential to check for updates

-frequently. ClamAV users often configure freshclam with a check interval

-of 30 minutes. 

-

-With an exponentially growing number of ClamAV users, the servers

-hosting the virus database files get easily overloaded. 

-

-Without mirrors, the traffic on our main site was 100GB/month (May

-2003). 

-

-On Feb 2004 the traffic on each mirror (11 in total) reached 120GB/month.

-\\

-Thanks to some improvements in freshclam and the increasing number

-of mirrors (currently 60), the traffic on each mirror was lowered

-to 40GB/month (Aug 2004). That makes about 2.5TByte/month of global

-traffic.

-

-Our users are encouraged to add the following directives to their

-freshclam.conf :

-

-DatabaseMirror db.XY.clamav.net

-

-DatabaseMirror db.local.clamav.net

-

-where XY stands for the country the server lives in %

-\footnote{a full list is available at http://www.iana.org/cctld/cctld-whois.htm%

-}

-

-Each db.XY.clamav.net record points to the mirrors available in that

-country%

-\footnote{For a complete list of the mirrors available in each country visit

-http://www.clamav.net/mirrors.html%

-} or, in case there are none, the continent.

-

-If freshclam can't connect to db.XY.clamav.net, it will fail back

-on db.local.clamav.net, which \textbf{attempts} to redirect the user

-to the closest pool of mirrors by looking up its ip source address

-in GeoIP database (http://www.maxmind.com/app/geoip\_country).%

-\footnote{See:

-

-http://www.iana.org/assignments/ipv4-address-space

-

-http://ip-to-country.webhosting.info/

-

-http://ftp.apnic.net/stats/apnic/

-

-http://www.ripe.net/db/erx/erx-ip/

-

-Some of the information were contributed by Walter Hop (from transip.nl).%

-} We are aware that looking up the ip source address is not an accurate

-method to find the user location from a network topology point of

-view. We accept the risk.

-

-

-\subsection{Requirements to become a mirror}

-

-We need fast reliable mirrors. Servers eligible for becoming mirrors

-have to provide:

-

-\begin{itemize}

-\item At least a 10Mbit/s link to the Internet%

-\footnote{Traffic is bursty, that's why we request such a large pipe%

-}

-\item Unlimited traffic

-\item At least 50MB of web space

-\item Support for our \emph{push-mirroring} system

-\item The mirror has to be available to all ClamAV users. We DO NOT support

-private mirrors.

-\item ssh 2 (read on)

-\end{itemize}

-We also appreciate (but do not require) having shell access to the

-server hosting the mirror. FTP access is no longer accepted. \\

-The virusdb team will use the account \emph{only} to update the virus

-database.

-

-

-\subsection{How to become a mirror}

-

-Before setting up a mirror contact \emph{luca -at- clamav.net}! 

-

-You have to follow these steps:

-

-\begin{enumerate}

-\item Set up a virtual host for \\

-http://database.clamav.net, http://db.{*}.clamav.net and http://clamav.your-domain.tld\\

-Note there is an asterisk in the second hostname. A literal asterisk.\\

-Do not replace it with your country code. \\

-If you are using name based virtual hosts%

-\footnote{You can check whether the mirror setup is correct or not, simply by

-adding a line like this:

-

-your-server-ip database.clamav.net

-

-to the /etc/hosts on your client machine. Then visit http://database.clamav.net

-and see if you can download files from your mirror's directory.%

-} see \\

-http://httpd.apache.org/docs/mod/core.html\#serveralias for more information.

-\\

-Here is an example for a typical setup:\\

-\\

-\emph{<VirtualHost 10.1.2.3> }\\

-\emph{ServerAdmin john@clamav.foo.com }\\

-\emph{DocumentRoot /home/users/clamavdb/public\_html }\\

-\emph{ServerName database.clamav.net}\\

-\emph{ServerAlias db.{*}.clamav.net}\\

-\emph{ServerAlias clamav.foo.com}\\

-\emph{</VirtualHost>}\\

-If you are not using Apache and you cannot create wildcard vhosts,

-you must use IP based virtual hosts!\\

-Please note that an http redirect (e.g. RedirectPermanent) is not

-enough! freshclam can't handle redirects yet. \emph{}\\

-If you are running Apache 2.x, please use the following directive

-for proper logging:\\

-\emph{LogFormat \char`\"{}\%h \%l \%u \%t \textbackslash{}\char`\"{}\%r\textbackslash{}\char`\"{}

-\%>s \%O \textbackslash{}\char`\"{}\%\{Referer\}i\textbackslash{}\char`\"{}

-\textbackslash{}\char`\"{}\%\{User-Agent\}i\textbackslash{}\char`\"{}\char`\"{}

-combinedrealsize }\\

-\emph{CustomLog /path/to/clamav-access.log combinedrealsize }\\

-See the {}``Statistics'' paragraph for more info.

-\item Create an account with login {}``clamavdb'' and give it write access

-to the virtual host's DocumentRoot. \\

-You may want to disable password authentication for this account and

-change the password to something obscure.\\

-The {}``clamavdb'' user's shell must be /bin/sh or /bin/bash . Otherwise

-the user won't be able to run the command associated with the ssh

-public key%

-\footnote{Take a look at the content of \emph{{}``authorized\_keys\_noshell''}:

-the only command which can be executed by the owner of the corresponding

-ssh private key is \textasciitilde{}/bin/clam-clientsync. We will

-only be able to trigger the execution of that script and nothing else!

-

-However, shell access is really appreciated. If you are willingly

-to give us shell access, use \emph{authorized\_key}s\emph{\_shell}

-instead which contains Luca Gibelli and Tomasz Papszun ssh public

-keys too.%

-}.

-\item Download the following files:\\

-clam-clientsync.conf\\

-clam-clientsync\\

-authorized\_keys\_shell\\

-authorized\_keys\_noshell\\

-authorized\_keys\_shell.sig\\

-authorized\_keys\_noshell.sig\\

-from http://www.clamav.net/doc/mirrors/

-\item Verify the signature using:\\

-\$ gpg --verify authorized\_keys\_noshell.sig authorized\_keys\_noshell\\

-\$ gpg --verify authorized\_keys\_shell.sig authorized\_keys\_shell\\

-My PGP public key is available on most keyservers and on ClamAV web

-site. It can eventually be verified by telephone. Contact me by email

-first.

-\item If you don't want to give us shell access, copy \emph{authorized\_keys\_noshell}

-to \emph{\textasciitilde{}clamavdb/.ssh/authorized\_keys}:\\

-\$ cp authorized\_keys\_noshell \textasciitilde{}/.ssh/authorized\_keys\\

-If you want to give us shell access, use \emph{authorized\_keys\_shell}

-instead:\\

-\$ cp authorized\_keys\_shell \textasciitilde{}clamavdb/.ssh/authorized\_keys\\

-\$ chmod go-w \textasciitilde{}clamavdb\\

-\$ chmod 700 \textasciitilde{}clamavdb/.ssh\\

-\$ chmod 600 \textasciitilde{}clamavdb/.ssh/authorized\_keys

-\item Copy clam-clientsync to \textasciitilde{}clamavdb/bin/\\

-Copy clam-clientsync.conf to \textasciitilde{}clamavdb/etc/\\

-chmod 600 \textasciitilde{}clamavdb/etc/clam-clientsync.conf\\

-chmod 755 \textasciitilde{}clamavdb/bin/clam-clientsync\\

-Everything must be owned by user clamavdb.\\

-The clam-clientsync requires the {}``lockfile'' program, which is

-part of the \emph{procmail} package. Before going any further, please

-check that {}``lockfile'' is available. 

-\item Send the server's details (ip address, country, virtual host aliases,

-available bandwidth and sysadmin's full name and email address) to

-\emph{luca} \emph{at} \emph{clamav.net} .

-\item Edit \textasciitilde{}clamavdb/etc/clam-clientsync.conf . If your

-DocumentRoot (see paragraph 1) is \emph{/home/users/clamavdb/public\_html}

-, your login is \emph{foo} and your password \emph{guessme}, then

-your clam-clientsync.conf will look like this: \\

-TO=/home/users/clamavdb/public\_html\\

-RSYNC\_USER=foo\\

-RSYNC\_PASSWORD=guessme\\

-EXCLUDE=\char`\"{}--exclude local\_{*}\char`\"{} 

-\item Reconfigure your packet filter to allow incoming connections on port

-22/tcp and outgoing connections to ports 873/tcp and 873/udp.\\

-You can furtherly restrict access to these ports by only allowing

-connections from/to the following IP addresses: \\

-194.109.142.194, 64.18.103.6, 194.242.226.43 .\\

-rsync.clamav.net is a round robin record which points to our master

-mirror servers. Any changes to this record will be announced on the

-clamav-mirrors mailing-list.

-\item You are welcome to put your company logo on the mirror home page.

-Just copy it to the DocumentRoot and rename it to {}``local\_logo.png''.

-The index.html is unique for every mirror. Please note that any file

-in the DocumentRoot whose name doesn't match {}``local\_{*}'' will

-be deleted at every mirror sync.

-\item Subscribe to clamav-mirrors \emph{at} lists.clamav.net: see \\

-http://lists.clamav.net/mailman/listinfo/clamav-mirrors for more info.

-\\

-Subscribe requests have to be approved. We will approve your subscription

-request only \emph{after} reviewing your server's info.

-\end{enumerate}

-When everything is done, your server's IP address will be added either

-to your country's dns record (db.XY.clamav.net) or one of the round

-robin record (db.<continent>.clamav.net) and your company will be

-listed on our mirrors list page.

-

-

-\subsection{Statistics}

-

-Although it's not required, we really appreciate if you can make access

-statistics of your mirror available to us. They should be available

-at http://your-mirror-host-name/local\_stats/ and they \textbf{must}

-be protected with login and password. You should use the same login

-and password you are using in your \textasciitilde{}clamavdb/etc/clam-clientsync.conf

-file.

-

-If possible, please tell your statistics generator to ignore requests

-made by the {}``ClamAV-MirrorCheck'' agent. 

-

-If you are using Webalizer, you can add the following directive to

-your conf. file:

-

-HideAgent ClamAV-MirrorCheck

-

-If you are using AWStats, you can add this one instead:

-

-SkipUserAgents=\char`\"{}ClamAV-MirrorCheck''

-

-Refer to your stats generator's manual for more info.

-

-\textbf{Important note} for Apache2 users: 

-

-As stated in the Apache documentation from http://httpd.apache.org/docs/2.0/mod/mod\_log\_config.html:

-

-\textit{Note that in httpd 2.0, unlike 1.3, the \%b and \%B format

-strings do not represent the number of bytes sent to the client, but

-simply the size in bytes of the HTTP response (which will differ,

-for instance, if the connection is aborted, or if SSL is used). The

-\%O format provided by mod\_logio will log the actual number of bytes

-sent over the network.}

-

-

-\subsection{Admin's duty}

-

-\begin{itemize}

-\item Scheduled downtimes should be announced on the clamav-mirrors mailing-list

-in advance.

-\item IP address changes should be notified in advance too. 

-\item Changes in the ssh host public key of the mirror host should be announced

-on the clamav-mirrors mailing-list.

-\item It is essential to be able to contact the sysadmin responsible for

-the mirror server and get a quick response. Whenever a problem with

-a mirror occurs we need to immediately find out its cause and act

-consequently.

-\end{itemize}

-

-\section{Notes for sigmakers}

-

-New sigmakers should send their ssh2 public key to \emph{luca at clamav.net}

-. Their public key will be added to rsyncX.clamav.net authorized\_keys

-(after being verified).

-

-Sigmakers can upload a new database to either rsync1.clamav.net or

-rsync2.clamav.net using a (scp|sftp|rsync)-only account. 

-

-The new database won't be available to other people immediately. First,

-sigmakers have to notify the rsyncX.clamav.net server that a new database

-is available. 

-

-Here is the step-by-step procedure to release a new database version

-and propagate it around the world:

-

-\begin{enumerate}

-\item Assume your ssh private key is \textasciitilde{}/.ssh/id\_rsa and

-you've just built a new daily.cvd. Assume you want to use rsync1.clamav.net

-\item In order to upload the new database, you have to run:\\

-\$ rsync -tcz --stats --progress -e 'ssh -i \textasciitilde{}/.ssh/id\_rsa'

-daily.cvd clamupload@rsync1.clamav.net:public\_html/

-\item Next, you need to notify rsync1.clamav.net that a new database is

-available:\\

-\$ ssh rsync1.clamav.net -i \textasciitilde{}/.ssh/id\_rsa -l clamavdb

-sleep 1

-\item rsync1.clamav.net will verify the digital signature of the newly uploaded

-database using \emph{sigtool -i}. If it finds an error, it will refuse

-to distribute the database to other mirrors.

-\item rsync1.clamav.net will copy the previously uploaded database to its

-rsync shared directory.

-\item rsync1.clamav.net will notify every mirror that a new database is

-available

-\item Every mirror will rsync its copy of the database from \emph{rsync1.clamav.net::clamavdb}

-(only mirrors can access the rsync server at rsync1.clamav.net, it's

-password protected)

-\end{enumerate}

-As a fallback, every three hours, either rsync1.clamav.net or rsync2.clamav.net

-force an update on every mirror. 

-

-If rsync1 can't reach rsync2 or viceversa, the automatic update doesn't

-take place. This is done to avoid propagating an old database.

-

-To avoid conflicts, sigmakers should use rsync1 by default and if

-it fails, switch to rsync2. Whenever a sigmaker uses rsync2, he should

-announce it on the clamav-team mailing-list so that every other sigmaker

-uses rsync2 too, until the issues with rsync1 are over. 

-

-

-\section{Mirror status}

-

-Every mirror is continously monitored to ensure that every ClamAV

-user gets the latest virus database.

-

-Every three hours we upload a file called \emph{timestamp} on every

-mirror. Every hour we choose a random mirror and check that \emph{timestamp}

-is fresh. If the file is one day old or unavailable, the mirror if

-marked as {}``old'' and the ClamAV team receive a warning. If the

-situation persists for two days, the mirror is temporarily removed

-from the list. 

-

-You can view the current status of every ClamAV database mirror at

-http://www.clamav.net/mirrors.html . 

-

-Please note that this page doesn't reflect how \emph{often} the database

-is propagated to mirrors. It just shows the trend of mirrors availability.

-\end{document}

@@ -204,7 +204,7 @@

     \item \textbf{clamav-virusdb*lists.clamav.net} - database update announcements, moderated

     \end{itemize}

     \noindent You can subscribe and search the mailing list archives at: 

-    \url{http://www.clamav.net/support/ml/}\\

+    \url{http://www.clamav.net/contact.html#ml}\\

     Alternatively you can try asking on the \verb+#clamav+ IRC channel - launch

     your favourite irc client and type:

     \begin{verbatim}

@@ -216,7 +216,7 @@

     If you have got a virus which is not detected by your ClamAV with the latest

     databases, please submit the sample at our website:

     \begin{center}

-	\url{http://www.clamav.net/sendvirus}

+	\url{http://www.clamav.net/malware-sample}

     \end{center}

     \section{Base package}

@@ -238,7 +238,7 @@

     \subsection{Binary packages}

     You can find the up-to-date list of binary packages at our website:

-    \url{http://www.clamav.net/download/packages/}

+    \url{http://www.clamav.net/download.html#otherversions}

     \section{Installation}

@@ -386,7 +386,7 @@ All 4 tests passed

     \subsection{Reporting a unit test failure bug}

 	If \verb+make check+ says that some tests failed we encourage you to report a bug on our bugzilla: \url{http://bugs.clamav.net}.

-	The information we need is (see also \url{http://clamav.net/bugs}):

+	The information we need is (see also \url{http://www.clamav.net/documentation.html#ins-bugs}):

 	\begin{itemize}

 	 \item The exact output from \verb+make check+	 

 	 \item Output of \verb+uname -mrsp+ 

@@ -332,7 +332,7 @@ How do I look in hex?

         (ClamAV$\ge$0.98.2)

 	\item \verb+HEXSIG[x-y]aa+ or \verb+aa[x-y]HEXSIG+\\

 	Match aa anchored to a hex-signature, see

-	\url{https://wwws.clamav.net/bugzilla/show_bug.cgi?id=776} for

+	\url{https://bugzilla.clamav.net/show_bug.cgi?id=776} for

 	discussion and examples.

 	\item \verb+(B)+\\

 	Match word boundary (including file boundaries).

@@ -500,7 +500,7 @@ cf43987e4f519d629b103375;SL+550:6300680065005c0046006900

     subsignature. The macro subsignature makes its preceding subsignature

     considered a match only if both of them get matched. For more

     information and examples please see

-    \url{https://wwws.clamav.net/bugzilla/show_bug.cgi?id=164}.

+    \url{https://bugzilla.clamav.net/show_bug.cgi?id=164}.

     \subsection{Icon signatures for PE files}

     ClamAV 0.96 includes an approximate/fuzzy icon matcher to help