spamdyke

A drop-in connection-time spam filter for qmail

spamdyke Documentation

This file is updated with each version of spamdyke to reflect the latest features and behavior. If you need documentation for an older version, each version's README file is included in the download package for that version.

This document applies to spamdyke version 4.0.3.

• About spamdyke
• Support
• How spamdyke works: When a message is not blocked
• How spamdyke works: When a message is blocked
• Usage
• Configuration Files
• Configuration Directories
• Configuration Tests
• Log Messages
• SMTP Error Codes
• Logging All Data
• Permissions
• DNS Queries
• Filter Levels
• TLS
• SMTP AUTH
• Relaying
• Reverse DNS
• Blacklists
• DNS RBLs
• DNS RHSBLs
• Whitelists
• Rejecting Senders and Recipients
• DNS Whitelists
• Whitelisting Senders and Recipients
• Graylisting / Greylisting
• Earlytalkers
• Limiting Numbers of Recipients
• Timeouts
• Extra Utilities

Other Documentation

The following additional documents are available:

Installation instructions: INSTALL.txt

Upgrading instructions: UPGRADING.txt

Frequently Asked Questions: FAQ.html

Change log: Changelog.txt

To-do list: TODO.txt

About spamdyke

help
version

spamdyke is a filter for monitoring and intercepting incoming SMTP connections to a qmail server. It acts as a transparent middleman, observing the conversation without interference unless it sees something it should block. Because it can silently monitor, it can also log mail traffic in several different ways.

spamdyke is ©2008 Sam Clippinger, samc (at) silence (dot) org. It is distributed under the GNU General Public License (version 2 only) from http://www.spamdyke.org/

The --help command line option will give a brief summary of the available command line options. The --version command line option will give just the version and copyright statement.

Support

spamdyke support is available from the spamdyke-users mailing list: www.spamdyke.org/mailman/listinfo/spamdyke-users.

The mailing list archives are searchable thanks to mail-archive.com: www.mail-archive.com/spamdyke-users@spamdyke.org.

If all else fails, email the author directly at samc (at) silence (dot) org.

How spamdyke works: When a message is not blocked

spamdyke works by acting as a middleman between qmail and the network (in Unix terms, it's a pipe). When no spamdyke filters are triggered and a message is delivered normally, spamdyke silently passes data in both directions. As the SMTP conversation takes place, spamdyke collects a few pieces of data (e.g. the sender and recipient addresses) so they can be logged.

spamdyke does modify the incoming message in one way. The SMTP protocol requires the remote sender to end every line with a two character terminator -- a carriage return and a line feed. Unlike most other mail servers, qmail chooses to strictly enforce this requirement. If a remote sender uses only a line feed to end a line (a typical and easy mistake to make), qmail will reject the message:

451 See http://pobox.com/~djb/docs/smtplf.html.

How spamdyke works: When a message is blocked

spamdyke's filters are described in detail below. When one of them is triggered, spamdyke moves in to block the incoming message.

First, it considers the enabled filters and waits until there is no way the client can avoid a rejection. For example, if authentication could take place but has not done so, spamdyke will wait to see if the remote sender authenticates. Authenticated or whitelisted connections are never filtered.

Next, once spamdyke is certain the message should be filtered, it cuts the connection between the remote sender and qmail. In the background, spamdyke closes the connection to qmail, so qmail will exit normally, believing the remote sender disconnected.

spamdyke continues sending responses to the remote server, just as qmail would have. Once the remote sender has identified the sender and recipient, spamdyke sends an error code and refuses to accept the message. The remote server disconnects, never knowing that spamdyke hijacked the conversation. spamdyke, meanwhile, uses the sender and recipient information it gathered to construct its log messages.

Usage

spamdyke's behavior is controlled through options given on the command line or in configuration files (or both).

On the command line, long options should be prefixed with two hyphens (--). Some options have short versions, which should be prefixed with one hyphen (-).

In a configuration file, only the long versions are valid and an equals sign must separate the value from the option. See Configuration Files for details.

For example, consider the max-recipients option, which restricts the maximum number of recipients per message. On the command line, limiting the number of recipients to 5 might look like this:

spamdyke --max-recipients 5 ...

spamdyke -a 5 ...

max-recipients=5

If the option's value contains spaces, it should be surrounded by quotes on the command line. For example, consider the rejection-text-ip-blacklist option, which changes the error message spamdyke sends if the remote server's IP address is blacklisted. On the command line, changing the message might look like this:

spamdyke --rejection-text-ip-blacklist "Go away spammer" ...

rejection-text-ip-blacklist=Go away spammer

After all options are given, spamdyke expects the rest of its command line to contain the qmail command. For example:

spamdyke -a 5 /var/qmail/bin/qmail-smtpd

ERROR: Missing qmail-smtpd command

spamdyke -a 5 -- /var/qmail/bin/qmail-smtpd

The following options are only valid on the command line:

The following options are valid on the command line and in configuration files. Some options are not valid in files within configuration directories; those options are noted below. See Configuration Directories for details.

Configuration Files

config-file

The configuration file format is very simple. Each line should use the following format:

OPTION=VALUE

VALUE is the parameter for the option. Note: While multi-word values must be quoted on the command line, quotes are not allowed in configuration files. spamdyke reads the entire VALUE after the equals sign, even if it contains spaces, so no quoting is needed.

Blank lines and lines beginning with # are ignored.

For example:

smtp-auth-command=/home/vpopmail/bin/vchkpw /bin/true

rdns-blacklist-dir=/home/vpopmail/blacklist_rdns.d

graylist-dir=/home/vpopmail/graylist.d

check-dnsrbl=dul.dnsbl.sorbs.net

check-dnsrbl=zombie.dnsbl.sorbs.net

max-recipients=5

reject-empty-rdns

reject-empty-rdns=yes

reject-empty-rdns=true

reject-empty-rdns=1

spamdyke --config-file /etc/spamdyke.conf ...

spamdyke --reject-empty-rdns --config-file /etc/spamdyke.conf ...

reject-empty-rdns=false

Some options can be given multiple times and spamdyke will use all of the values it finds. For example, if the following lines are given, spamdyke will search each of the files for a match to the sender's email address:

sender-blacklist-file=/home/vpopmail/sender_blacklist.txt

sender-blacklist-file=/home/vpopmail/more_sender_blacklist.txt

sender-blacklist-file=/home/vpopmail/additional_sender_blacklist.txt

OPTION=!VALUE

sender-blacklist-file=!/home/vpopmail/sender_blacklist.txt

To clear all values from a list, three exclamation points should be given instead of a value:

OPTION=!!!

sender-blacklist-file=!!!

Note: spamdyke processes configuration directives in the order they are read. If an option is cleared and later set again, the option will retain the last value. For example, if the following five lines appear in this order:

sender-blacklist-file=/home/vpopmail/sender_blacklist.txt

sender-blacklist-file=/home/vpopmail/more_sender_blacklist.txt

sender-blacklist-file=/home/vpopmail/additional_sender_blacklist.txt

sender-blacklist-file=!!!

sender-blacklist-file=/home/vpopmail/last_blacklist.txt

NOTE: It may seem that scanning a configuration file instead of the command line would impose a performance penalty each time spamdyke is started. However, the reverse seems to be true. Some rudimentary testing has indicated the configuration files are actually faster. This is likely due to inefficiencies in GNU's getopt_long() function.

Configuration Directories

config-dir
config-dir-search

Configuration directories allow spamdyke's behavior to be altered based on the remote server's IP address, the remote server's rDNS name, the sender's email address, the recipient's email address or any combination of those things. This can be very useful when, for example, graylisting should be deactivated for a specific sender. IP addresses can be whitelisted for specific recipients without whitelisting them for everyone. The possibilities are nearly endless.

NOTE: Configuration directories are confusing and complicated. Unless you really need the advanced configuration scenarios they offer, don't use them.

Configuration directories are given with the config-dir option. The option's value should be the path to the directory that contains the subdirectories explained below. If config-dir is given multiple times, spamdyke will search each given directory structure and load all of the matching files before it continues processing the SMTP connection.

In essence, a configuration directory is a special directory structure that contains configuration files. spamdyke determines which files to load based on the names of the directories and the details of the SMTP connection. Not all options are valid within configuration directories, but in all other respects the files follow the same rules as global configuration files. See Usage for details of which options are valid within configuration directories. See Configuration Files for details of the configuration file format.

When spamdyke loads a file from a configuration directory, it will do so because the names of the directories and the name of the file match all or part of the information from the SMTP connection. The last piece of information should always be used as the name of the file, not the name of a directory.

To create a file using the IP address of the remote server, first create a directory structure that begins with _ip_ and uses the first three octets of the address as directory names. For example, if the IP address is 11.22.33.44, the directory structure should look like this:

.../_ip_/11/22/33

.../_ip_/11/22/33/44

To create a file using the rDNS name of the remote server, first create a directory structure that begins with _rdns_ and contains directories named using the rDNS name with its words reversed. For example, if the rDNS name is mail.internal.headquarters.example.com, the directory structure should look like this:

.../_rdns_/com/example/headquarters/internal

.../_rdns_/com/example/headquarters/internal/mail

To create a file using the sender's email address, first create a directory structure that begins with _sender_ and contains directories using the domain portion of the sender's email address with its words reversed and ending in _at_. For example, if the sender's email address is mom@home.example.com, the directory structure should look like this:

.../_sender_/com/example/home/_at_

.../_sender_/com/example/home/_at_/mom

To create a file using the recipient's email address, first create a directory structure that begins with _recipient_ and contains directories using the domain portion of the recipient's email address with its words reversed and ending in _at_. For example, if the recipient's email address is kid@school.example.com, the directory structure should look like this:

.../_recipient_/com/example/school/_at_

.../_recipient_/com/example/school/_at_/kid

.../_ip_/11/22/33/44

.../_ip_/11/22/33

.../_ip_/11/22

.../_ip_/11

.../_rdns_/com/example/headquarters/internal/mail

.../_rdns_/com/example/headquarters/internal

.../_rdns_/com/example/headquarters

.../_rdns_/com/example

.../_rdns_/com

.../_recipient_/com/example/school/_at_/kid

.../_recipient_/com/example/school

.../_recipient_/com/example

.../_recipient_/com

.../_sender_/com/example/home/_at_/mom

.../_sender_/com/example/home

.../_sender_/com/example

.../_sender_/com

Configuration directories can be nested to create more specific targets. For example, if the IP address of the remote server is 11.22.33.44 and the sender's email address is mom@home.example.com, spamdyke will read a configuration file if its path is either of the following:

.../_ip_/11/22/33/44/_sender_/com/example/home/_at_/mom

.../_sender_/com/example/home/_at_/mom/_ip_/11/22/33/44

.../_ip_/11/22/33/44/_sender_/com/example/home

.../_ip_/11/22/33/44/_sender_/com/example

.../_ip_/11/22/33/44/_sender_/com

.../_ip_/11/22/33/_sender_/com/example/home/_at_/mom

.../_ip_/11/22/_sender_/com/example/home/_at_/mom

.../_ip_/11/_sender_/com/example/home/_at_/mom

.../_sender_/com/example/home/_ip_/11/22/33

.../_sender_/com/example/_ip_/11/22

.../_sender_/com/_ip_/11

If all of that isn't confusing enough, spamdyke will only read one file from a _ip_, _rdns_, _sender_ or _recipient_ directory, even if more matches are possible. For example, if the remote IP address is 11.22.33.44, the sender's email address is mom@home.example.com and the recipient's email address is kid@school.example.com and two files exist with the following paths:

.../_ip_/11/22/33/44/_sender_/com/example/home/_at_/mom

.../_ip_/11/22/33/44/_recipient_/com/example/school/_at_/kid

config-dir-search can be given multiple times; the values will be added together to create a composite value. The possible values are:

• first: Match each _ip_, _rdns_, _sender_ or _recipient directory only once. NOTE: The first value erases the composite value created from the other possible values, essentially "resetting" the config-dir-search option.
• all-ip: Match each _ip_ directory as many times as possible.
• all-rdns: Match each _rdns_ directory as many times as possible.
• all-sender: Match each _sender_ directory as many times as possible.
• all-recipient: Match each _recipient_ directory as many times as possible.

To aid with troubleshooting, spamdyke will log the paths it searches if the log-level option is debug or higher.

Configuration Tests

config-test
config-test-smtpauth-username
config-test-smtpauth-password

spamdyke has the ability to scan its configuration and look for common configuration mistakes. It checks file paths, permissions, graylist folders, directory structures, SMTP AUTH commands, TLS certificates and more. This feature was inspired by Apache's ability to check its configuration file for syntax errors.

To use the testing feature:

1. Find and copy the entire spamdyke command line from your "supervise" script or xinetd configuration file, including the qmail command(s).
2. At a command prompt, login as root and paste the spamdyke command without running it.
3. Add the option --config-test among the spamdyke options (before the qmail command). If appropriate, add the options --config-test-smtpauth-username and --config-test-smtpauth-password.
4. Run the command and carefully read the results. More output can be obtained by increasing the logging level (no test output goes to syslog).

If spamdyke is configured to provide SMTP AUTH (using the smtp-auth-level and smtp-auth-command options), the --config-test-smtpauth-username and --config-test-smtpauth-password options should be used to provide a valid username and password for authentication. spamdyke will run the SMTP AUTH command to test its capabilities and make recommendations.

IMPORTANT! DANGER! WARNING! DO NOT EVER PUT THE --config-test OPTION IN THE SPAMDYKE COMMAND LINE THAT IS RUN FOR INCOMING CONNECTIONS! YOUR MAIL SERVER WILL IMMEDIATELY STOP RECEIVING MAIL AND REMOTE USERS WILL SEE ONLY THE DIAGNOSTIC OUTPUT! If you make this mistake and ask for help, expect to be publicly mocked. You have been warned.

Log Messages

log-level
log-target

The log-target option controls where spamdyke logs its messages. By default, log-target is set to syslog, which sends log messages to the system syslog facility. When log-target is set to stderr, messages are sent to standard error (stderr) instead. For most qmail installations, this will cause spamdyke's messages to be logged by the "multilog" program, along with qmail-smtpd's output. If log-target is given multiple times with different values, spamdyke will sends its output to each given target.

When spamdyke logs to syslog, it uses the LOG_MAIL facility, which typically puts the messages in /var/log/maillog. (Note: Plesk reconfigures syslog to put the messages in /usr/local/psa/var/log/maillog.)

Regardless of how the messages are logged, errors are always be preceded by the text ERROR: and are fairly self-explanatory. Whenever possible, spamdyke will recover from an error and continue processing mail. Philosophically, it's better to continue receiving spam than to block all mail.

The log-level option controls how much logging takes place. The following values are supported:

• none: No logging at all, even if errors occur. This is not recommended.
• error: Critical errors only, including authentication failures. This is the default when log-level is not given.
• info: Everything from error plus logging of messages (sender, recipient, IP address, rDNS name and authenticated username). This is the value used when log-level is given with no value.
• verbose: Everything from info plus non-critical errors such as network errors caused by the remote host, protocol errors, config-test status messages and child process error messages. At this level, spamdyke will also print messages to show which filter blocked the connection (if applicable) and some details about the filter's settings. These messages will be prefixed with FILTER:.
• debug: Everything from verbose plus high-level debugging messages, intended to show the processing path within spamdyke. This level is handy for troubleshooting but it can be rather noisy. Extra messages generated by this level will be prefixed with DEBUG(): and will show the file and line number within the spamdyke source code where the message was produced. NOTE: If the configure script is run with the --without-debug-output option, spamdyke will accept the debug value but it will not print any more output than if verbose were used.
• excessive: Everything from debug plus lots of internal status messages. This value should only be used for development. Extra messages generated by this level will be prefixed with EXCESSIVE(): and will show the file and line number within the spamdyke source code where the message was produced. NOTE: Unless the configure script is run with the --with-excessive-output option, spamdyke will not produce any more output for excessive than if debug were used.

/usr/local/bin/spamdyke --log-level=verbose ...

/usr/local/bin/spamdyke -lverbose ...

/usr/local/bin/spamdyke --log-level ...

/usr/local/bin/spamdyke -l ...

Each message log entry (produced when the value of log-level is info or higher) takes the following form (error messages and debugging statements are text preceeded by ERROR:, FILTER:, DEBUG: or EXCESSIVE:):

CODE from: SENDER to: RECIPIENT origin_ip: IPADDRESS origin_rdns: RDNSNAME auth: USERNAME [ reason: REALCODE ]

The possible values of CODE are listed below:

SENDER is the sender email address, if known, or (unknown) otherwise. NOTE: According to RFC 821, it is legal to deliver messages with no sender address. Most bounce messages are delivered this way.

RECIPIENT is the recipient email address, if known, or (unknown) otherwise. If CODE is ALLOWED, the recipient email address will be known.

IPADDRESS is the IP address of the remote server. This value is always known.

RDNSNAME is the rDNS name of the remote server, if known, or (unknown) otherwise.

USERNAME is the username given during authentication, if authentication was successful, or (unknown) otherwise.

REALCODE is only present if CODE is TIMEOUT and the connection was going to be blocked anyway. For example, if a remote server has no rDNS entry and the connection is going to be blocked but the connection times out instead, CODE will be TIMEOUT and REALCODE will be DENIED_RDNS_MISSING.

SMTP Error Codes

policy-url
rejection-text-access-denied
rejection-text-auth-failure
rejection-text-auth-unknown
rejection-text-dns-blacklist
rejection-text-earlytalker
rejection-text-empty-rdns
rejection-text-graylist
rejection-text-ip-blacklist
rejection-text-ip-in-cc-rdns
rejection-text-ip-in-rdns-keyword-blacklist
rejection-text-local-recipient
rejection-text-max-recipients
rejection-text-missing-sender-mx
rejection-text-rdns-blacklist
rejection-text-recipient-blacklist
rejection-text-reject-all
rejection-text-relaying-denied
rejection-text-rhs-blacklist
rejection-text-sender-blacklist
rejection-text-smtp-auth-required
rejection-text-timeout
rejection-text-tls-failure
rejection-text-unresolvable-rdns
rejection-text-zero-recipients

When spamdyke blocks a connection and returns an error code to a remote server, the text it sends is different from what appears in the logs (above). It is more user-friendly, just in case a human ever reads it (some, but not all, mail servers display the rejection message in bounce messages).

The messages can be changed using the options that are listed in the third column of the table below.

The messages that correspond to the syslog codes are:

If a policy location URL is given with the policy-url option, it will be appended to the end of the message, just in case a human ever reads it. This option should always be used. When a legitimate remote user is incorrectly blocked, the URL should provide your contact information so the error can be corrected.

spamdyke will always append the syslog code to the policy URL so a web browser will jump to an anchor within the HTML document. Most of the time, the code is prefixed with a # character. For example, if the policy URL is:

http://www.example.com/policy.html

http://www.example.com/policy.html#DENIED_RDNS_MISSING

http://www.example.com/policy?code=

http://www.example.com/policy?code=DENIED_RDNS_MISSING

Logging All Data

full-log-dir

spamdyke has the ability to log all SMTP data to files. This is very helpful when debugging but (depending on the mail server traffic levels) it can generate a huge number of files.

This