Hosting a Private Database Mirror - ClamAV Documentation
Description: An open source malware detection toolkit and antivirus engine.

## Keyboard shortcuts

Press `←` or `→` to navigate between chapters

Press `S` or `/` to search in the book

Press `?` to show this help

Press `Esc` to hide this help

# Private Local Mirrors

There are some situations in which it may be desirable to set up a private mirror for distributing ClamAV databases.

If you run ClamAV on many clients on your network, each new installation will download a copy of the database files. This is a waste of bandwidth and resources for your network and for our mirrors network.

Sometimes the servers which perform the scan are not directly connected to Internet and can only download updates from a server in the same network segment.

For people who face these problems, we recommend using one of the following solutions:

- Use `cvdupdate` to serve whole databases and database patch files from a private mirror
- Use `freshclam` to serve only whole database files from a private mirror
- Use an HTTP proxy

## Use `cvdupdate` to serve whole databases and database patch files from a private mirror

You may use a tool named `cvdupdate` on a private mirror to maintain the latest CVD databases and CDIFF patch files.

This solution will allow you to host a mirror that functions in the same way as the official database CDN, serving CVD and CDIFF files. This means that your downstream `freshclam` clients will be able to update using the CDIFF patch files, which should save some bandwidth between your private mirror and your clients.

These instructions use a tool named cvdupdate. `cvdupdate` requires:

- Python 3.6 or newer.
- An internet connection with DNS enabled.
- It should work fine on Linux/Unix and on Windows.

IMPORTANT: Please do NOT use `cvdupdate` if you don’t need to host a private database mirror. `freshclam` is far more efficient, even for a small cluster of installs, because it will update with CDIFF patches after the initial database downloads. `cvdupdate`, on the otherhand, will download both the new daily CDIFF and the daily CVD every day.

You can easily install `cvdupdate` using Python 3’s Pip package manager:

    pip3 install cvdupdate

(optional) Once installed, you may wish to configure where the databases are stored:

    cvd config set --dbdir <your www path>

Now run this as often as you need, or at least once a day to download/update the databases:

    cvd update

You may wish to set up a `cron` job to check for updates.

If you didn’t set a custom database path, the databases will be stored in `~/.cvdupdate/database`

> Tip: You can use `--help` with any `cvd` command to learn more. For ore detailed instructions, or to report issues, please visit: https://github.com/Cisco-Talos/cvdupdate](https://github.com/Cisco-Talos/cvdupdate)

Once you have the database files, host them with your favorite webserver, or use the `cvd serve` test-webserver (not intended for production).

Next, you’ll need to configure the `freshclam` clients so they’ll update from your private mirror.

For `freshclam.conf` on your downstream `freshclam` clients, set:

    # Replace `mirror.mylan` and `8000` with your domain and port number.
    DatabaseMirror http://mirror.mylan:8000

You may wish to set up a proxy to enable HTTPS. If you do, you can specify `https` instead of `http`:

    DatabaseMirror https://mirror.mylan

You could also host the files in a subdirectory. E.g.:

    DatabaseMirror http://mirror.mylan:8000/clamav

When you run `freshclam` on your client machines, they will still use a DNS query to clamav.net to find out if there should be an update before attempting to update from your private server. If your `freshclam` clients attempt to update before your private mirror updates, that’s okay. The `freshclam` clients will tolerate being 1 version behind what was advertised on clamav.net.

> Tip: If the `freshclam` clients will not have access to the internet to perform that DNS lookup, you may wish to set `DNSDatabaseInfo no` in your `freshclam.conf` file. `freshclam` may complain that the DNS lookup to “no” failed, which is fine. It will fall-back to checking the database version using an HTTP Range-request to your server.

> CAUTION: If your `freshclam` clients cannot use DNS to check if there is an update, be certain that your private mirror’s webserver supports HTTP Range-requests, or else it may serve the ENTIRE database CVD file when a `freshclam` client means to check if a newer version exists, and not just a small portion containing the database version.

> The Python `simple.http` server does NOT support HTTP Range requests.

## Use `freshclam` to serve only whole database files from a private mirror

You may use `freshclam` on a private mirror to maintain the latest CVD or CLD databases.

The `freshclam` program running on your private mirror will update using the CDIFF patch files. When you update a CVD database with ClamAV’s CDIFF patching process, it produces a CLD “local” database. With this solution for hosting a private mirror, you will serve those CVD or CLD databases to downstream `freshclam` clients. Unlike when using `cvdupdate`, this option will not allow you to serve CDIFF patch files.

> Tip: This method may be best if your public IP address is shared with other clients. At present we rate-limit CVD downloads by IP address. So if your public IP address is used by others, `cvdupdate` may be rate-limited when it attempts to download `daily.cvd`. But `freshclam` should never be rate limited for attempting to download the lateset CDIFF patch file.

This solution is simple to implement. But because you will not be serving CDIFF patch files, it is only effective if your clients are all on the same local network or if bandwidth between your private mirror and your clients is not an issue for you.

To get started, configure a local webserver on one of your machines (say `mirror.mylan`). Set up `freshclam` on that server so it downloads the database files from http://database.clamav.net and stores them in your webserver’s DocumentRoot directory.

For `freshclam.conf` on your private mirror, set:

    # The private mirror will update from database.clamav.net.
    DatabaseMirror database.clamav.net

    # Customize the DatabaseDirectory so that FreshClam will update the DocumentRoot.
    DatabaseDirectory /your/server/www

    # Enable CLD compression to save bandwidth between your mirror and your clients.
    CompressLocalDatabase yes

Set up `freshclam` to run as a service or in a cron job so that your private mirror always serves the latest databases.

Next, you’ll need to configure the `freshclam` clients so they’ll update from your private mirror.

For `freshclam.conf` on your downstream `freshclam` clients, set:

    # PrivateMirror is used instead of DatabaseMirror so that FreshClam will:
    # 1. Accept CVD or CLD files, not just CVD files.
    # 2. Use an HTTP Range-request to check if there is an update, rather than DNS.
    PrivateMirror http://mirror.mylan:8000

    # ScriptedUpdates is not needed because you won't be serving CDIFF files.
    ScriptedUpdates no

When you run `freshclam` on your client machines, they should check for updates from your private server over HTTP by downloading just the database header`*`. If there is a new version, the client will download the whole CVD or CLD file from your private server to update.

> `*`Important: Make sure your HTTP server will accept and handle HTTP Range requests. If yours does not, then each time a client checks for an update it will download the whole database!

> The Python `simple.http` server does NOT support HTTP Range requests.

## Use an HTTP proxy

This solution is really easy to implement and is bandwidth efficient.

Install a proxy server that supports caching files (e.g. squid) and then tell your `freshclam` clients to use it. This can be done by setting the HTTPProxyServer parameter in freshclam.conf (see `man 5 freshclam.conf` for the details).