Difference between revisions of "Spamassassin"
(Learning from dovecot) |
(`pyzor discover` command was removed https://github.com/SpamExperts/pyzor/commit/50f2bf5aa47ed863de78c413ff7114f5e54f5a9b) |
||
(18 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
− | + | SpamAssassin is a spam detection software intended to be run on your mail server. It rank mail using several criteria criteria that can be put in the following families | |
+ | * DNS Whitelist/Blacklist: does the server that sent you the email sent spam before? | ||
+ | * URI Blacklist: does the body of the message contain links to some bad sites? | ||
+ | * Distributed Spam Hashes: does someone reported the same message as spam already? | ||
+ | * Bayesian Filter: compare email to your past spam and ham | ||
+ | * SPF/DKIM: check is the ''from'' email address that you see is legitimate | ||
+ | * Static Rules: a lot of manually crafted rules by SpamAssassin contributors | ||
+ | |||
+ | == Prerequisites == | ||
+ | This article is part of the [[Emails/Complete|emails]] series. It is assumed that you already covered [[Dovecot]] and [[Exim]]. | ||
+ | |||
+ | Optional prerequisites: | ||
+ | * [[nftables]] is used as a firewall here. You can however replace it by any firewall you use. | ||
+ | * [[Munin]] allows you to monitor the spam/ham ratio of your installation. | ||
== Install == | == Install == | ||
<syntaxhighlight lang="console"> | <syntaxhighlight lang="console"> | ||
− | + | $ sudo apt install spamassassin | |
</syntaxhighlight> | </syntaxhighlight> | ||
Line 11: | Line 24: | ||
After changing config in <code>/etc/spamassassin/</code>, don't forget tell SpamAssassin to reload config | After changing config in <code>/etc/spamassassin/</code>, don't forget tell SpamAssassin to reload config | ||
<syntaxhighlight lang="console"> | <syntaxhighlight lang="console"> | ||
− | + | $ sudo service spamassassin reload | |
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
</syntaxhighlight> | </syntaxhighlight> | ||
Line 33: | Line 35: | ||
<syntaxhighlight lang="console"> | <syntaxhighlight lang="console"> | ||
− | + | $ sudo -u vmail sa-learn --spam --progress --dir /var/maildir/<username>/Maildir/.Spam/cur/ | |
− | + | $ sudo -u vmail sa-learn --ham --progress --dir /var/maildir/<username>/Maildir/cur/ | |
</syntaxhighlight> | </syntaxhighlight> | ||
Line 40: | Line 42: | ||
<syntaxhighlight lang="console"> | <syntaxhighlight lang="console"> | ||
− | + | $ sudo -u vmail sa-learn --dump magic | |
</syntaxhighlight> | </syntaxhighlight> | ||
+ | === Pyzor === | ||
+ | |||
+ | ==== Install ==== | ||
+ | <syntaxhighlight lang="console"> | ||
+ | $ sudo apt install pyzor | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | ==== Firewall ==== | ||
+ | {{nftables/config|category=Pyzor (Spamassassin)|tcp_port_in=|udp_port_in=|tcp_port_out=24441|udp_port_out=24441|user_out=}} | ||
+ | |||
+ | === Razor === | ||
+ | |||
+ | ==== Install ==== | ||
+ | <syntaxhighlight lang="console"> | ||
+ | $ sudo apt install razor | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | ==== Firewall ==== | ||
+ | {{nftables/config|category=Razor (Spamassassin)|tcp_port_in=|udp_port_in=|tcp_port_out=2703|udp_port_out=|user_out=}} | ||
+ | |||
+ | ==== Configure ==== | ||
+ | <syntaxhighlight lang="console"> | ||
+ | $ sudo -u vmail razor-admin -create | ||
+ | $ sudo -u vmail razor-admin -register | ||
+ | Register successful. Identity stored in /var/maildir/.razor/identity-xo4OkrHieL | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | === Report Headers === | ||
+ | |||
+ | SpamAssassin can had headers in the messages it scan. It will help you investigate things in case you get false-positive are false-negative. | ||
+ | |||
+ | Add the following lines to <code>/etc/spamassassin/local.cf</code> | ||
+ | <syntaxhighlight lang="shell"> | ||
+ | # The status header is used by other programs to read the spam status. Don't modify the part before tests=... | ||
+ | add_header all Status _YESNO_, hits=_HITS_ required=_REQD_ tests=_TESTSSCORES(,)_ autolearn=_AUTOLEARN_ | ||
+ | add_header all Details version=_VERSION_ _REPORT_ | ||
+ | add_header all Pyzor _PYZOR_ | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | === Configure service === | ||
+ | Edit file <code>/etc/default/spamassassin</code> and change the following line<syntaxhighlight lang="shell"> | ||
+ | OPTIONS="--create-prefs --max-children 5 -u vmail --listen /run/spamd.socket" | ||
+ | </syntaxhighlight>Create file <code>/etc/spamassassin/spamc.conf</code> with the following content<syntaxhighlight lang="text"> | ||
+ | --socket /run/spamd.socket | ||
+ | </syntaxhighlight>It's now time to enable the Spamassassin service<syntaxhighlight lang="console"> | ||
+ | $ sudo systemctl enable spamassassin.service | ||
+ | $ sudo systemctl start spamassassin.service | ||
+ | </syntaxhighlight> | ||
+ | |||
+ | === Cron === | ||
+ | Spamassassin authors publish updated rules on a daily basis. To stay up-to-date, edit file <code>/etc/default/spamassassin</code> and set option<syntaxhighlight lang="shell"> | ||
+ | CRON=1 | ||
+ | </syntaxhighlight> | ||
== Integrate with exim == | == Integrate with exim == | ||
<syntaxhighlight lang="console"> | <syntaxhighlight lang="console"> | ||
− | + | $ sudo apt install sa-exim | |
</syntaxhighlight> | </syntaxhighlight> | ||
Line 52: | Line 107: | ||
Edit the following setting | Edit the following setting | ||
− | <syntaxhighlight lang=" | + | <syntaxhighlight lang="properties"> |
SAspamcUser: vmail | SAspamcUser: vmail | ||
</syntaxhighlight> | </syntaxhighlight> | ||
By defauld ''sa-exim'' is disabled. Remove the following lines to enable it | By defauld ''sa-exim'' is disabled. Remove the following lines to enable it | ||
− | <syntaxhighlight lang=" | + | <syntaxhighlight lang="properties"> |
#---------------------------------------------------------------------- | #---------------------------------------------------------------------- | ||
# Remove or comment out the following line to enable sa-exim | # Remove or comment out the following line to enable sa-exim | ||
SAEximRunCond: 0 | SAEximRunCond: 0 | ||
#---------------------------------------------------------------------- | #---------------------------------------------------------------------- | ||
− | </syntaxhighlight> | + | </syntaxhighlight>Other parameter that I change<syntaxhighlight lang="properties"> |
− | + | SApermreject: 10.0 | |
− | You can now restart exim to take you settings into account | + | </syntaxhighlight>You can now restart exim to take you settings into account |
<syntaxhighlight lang="console"> | <syntaxhighlight lang="console"> | ||
− | + | $ sudo systemctl restart exim4.service | |
</syntaxhighlight> | </syntaxhighlight> | ||
== Integrate with dovecot == | == Integrate with dovecot == | ||
+ | SpamAssassin is able to learn from it's mistakes. By using the plugin ''dovecot-antispam'', we train SpamAssassin by just moving email in or out of the spam folder. | ||
+ | |||
+ | First install it with this command | ||
<syntaxhighlight lang="console"> | <syntaxhighlight lang="console"> | ||
− | + | $ sudo apt install dovecot-antispam | |
</syntaxhighlight> | </syntaxhighlight> | ||
− | + | Then in file <code>/etc/dovecot/conf.d/20-imap.conf</code>, modify the option ''mail_plugins'' and add ''antispam'' to the list | |
<syntaxhighlight lang="shell"> | <syntaxhighlight lang="shell"> | ||
protocol imap { | protocol imap { | ||
Line 83: | Line 141: | ||
</syntaxhighlight> | </syntaxhighlight> | ||
− | + | Create file <code>/etc/dovecot/conf.d/90-antispam.conf</code> | |
<syntaxhighlight lang="shell"> | <syntaxhighlight lang="shell"> | ||
− | + | plugin { | |
− | + | ################## | |
− | + | # GENERIC OPTIONS | |
− | + | ||
− | + | # Debugging options | |
− | + | # Uncomment to get the desired debugging behaviour. | |
+ | # Note that in some cases stderr debugging will not be as | ||
+ | # verbose as syslog debugging due to internal limitations. | ||
+ | # | ||
+ | # antispam_debug_target = syslog | ||
+ | # antispam_debug_target = stderr | ||
+ | # antispam_verbose_debug = 1 | ||
+ | |||
+ | antispam_backend = pipe | ||
+ | |||
+ | antispam_trash_pattern_ignorecase = trash;Deleted Items;Deleted Messages | ||
+ | antispam_spam_pattern_ignorecase = Spam;Junk | ||
+ | |||
+ | |||
+ | ########################### | ||
+ | # BACKEND SPECIFIC OPTIONS | ||
+ | # | ||
+ | |||
+ | #===================== | ||
+ | # pipe plugin | ||
+ | # | ||
+ | |||
+ | # temporary directory | ||
+ | antispam_pipe_tmpdir = /tmp | ||
+ | |||
+ | # spam/not-spam argument (default unset which will is not what you want) | ||
+ | antispam_pipe_program_spam_arg = -r | ||
+ | antispam_pipe_program_notspam_arg = -k | ||
+ | |||
+ | # binary to pipe mail to | ||
+ | antispam_pipe_program = /usr/bin/spamassassin | ||
+ | } | ||
</syntaxhighlight> | </syntaxhighlight> | ||
+ | And finally, reload Dovecot | ||
+ | <syntaxhighlight lang="console"> | ||
+ | $ sudo systemctl restart dovecot.service | ||
+ | </syntaxhighlight> | ||
− | [[Category: | + | == Integrate in Munin == |
+ | There is a plugin in [[Munin]] to get statistics on the ham/spam values from Spamassassin. To activate it, run the following command<syntaxhighlight lang="console"> | ||
+ | $ sudo ln -s /usr/share/munin/plugins/spamstats /etc/munin/plugins/ | ||
+ | </syntaxhighlight>Then create file <code>/etc/munin/plugin-conf.d/spamstats</code><syntaxhighlight lang="ini"> | ||
+ | [spamstats] | ||
+ | group adm | ||
+ | env.logfile mail.log | ||
+ | </syntaxhighlight>Finally, restart the Munin node<syntaxhighlight lang="console"> | ||
+ | $ sudo systemctl restart munin-node.service | ||
+ | </syntaxhighlight>After 5 minutes, you should see your new graph in Munin. | ||
+ | [[Category:Email Server]] | ||
+ | [[Category:Linux Server]] | ||
+ | [[Category:Munin]] |
Latest revision as of 06:22, 3 May 2018
SpamAssassin is a spam detection software intended to be run on your mail server. It rank mail using several criteria criteria that can be put in the following families
- DNS Whitelist/Blacklist: does the server that sent you the email sent spam before?
- URI Blacklist: does the body of the message contain links to some bad sites?
- Distributed Spam Hashes: does someone reported the same message as spam already?
- Bayesian Filter: compare email to your past spam and ham
- SPF/DKIM: check is the from email address that you see is legitimate
- Static Rules: a lot of manually crafted rules by SpamAssassin contributors
Prerequisites
This article is part of the emails series. It is assumed that you already covered Dovecot and Exim.
Optional prerequisites:
- nftables is used as a firewall here. You can however replace it by any firewall you use.
- Munin allows you to monitor the spam/ham ratio of your installation.
Install
$ sudo apt install spamassassin
Configure
After changing config in /etc/spamassassin/
, don't forget tell SpamAssassin to reload config
$ sudo service spamassassin reload
Bayesian filter
To reach a good efficiency, SpamAssassin Bayesian filter need to be trained with both spam and ham messages. You can use your actual mailbox for that but note the following points:
- Be sure that the folders you use for training contain only spam or ham. If a folder contain a mix of them, SpamAssassin will learn wrong info and produce bad quality results
- To be effective you need between 1000 and 5000 messages each of both spam and ham.
- You need to have more ham than spam to train. Otherwise, SpamAssassin might become biased toward spam.
$ sudo -u vmail sa-learn --spam --progress --dir /var/maildir/<username>/Maildir/.Spam/cur/
$ sudo -u vmail sa-learn --ham --progress --dir /var/maildir/<username>/Maildir/cur/
To check the status of the database, you can run
$ sudo -u vmail sa-learn --dump magic
Pyzor
Install
$ sudo apt install pyzor
Firewall
Assuming that you configured nftables as described, you can edit file /etc/nftables/main_config.conf
and add
# Pyzor (Spamassassin)
add element inet main tcp_port_out { 24441 }
add element inet main udp_port_out { 24441 }
and activate it using
$ sudo /etc/nftables/reload_main.conf
Razor
Install
$ sudo apt install razor
Firewall
Assuming that you configured nftables as described, you can edit file /etc/nftables/main_config.conf
and add
# Razor (Spamassassin)
add element inet main tcp_port_out { 2703 }
and activate it using
$ sudo /etc/nftables/reload_main.conf
Configure
$ sudo -u vmail razor-admin -create
$ sudo -u vmail razor-admin -register
Register successful. Identity stored in /var/maildir/.razor/identity-xo4OkrHieL
Report Headers
SpamAssassin can had headers in the messages it scan. It will help you investigate things in case you get false-positive are false-negative.
Add the following lines to /etc/spamassassin/local.cf
# The status header is used by other programs to read the spam status. Don't modify the part before tests=...
add_header all Status _YESNO_, hits=_HITS_ required=_REQD_ tests=_TESTSSCORES(,)_ autolearn=_AUTOLEARN_
add_header all Details version=_VERSION_ _REPORT_
add_header all Pyzor _PYZOR_
Configure service
Edit file /etc/default/spamassassin
and change the following line
OPTIONS="--create-prefs --max-children 5 -u vmail --listen /run/spamd.socket"
Create file /etc/spamassassin/spamc.conf
with the following content
--socket /run/spamd.socket
It's now time to enable the Spamassassin service
$ sudo systemctl enable spamassassin.service
$ sudo systemctl start spamassassin.service
Cron
Spamassassin authors publish updated rules on a daily basis. To stay up-to-date, edit file /etc/default/spamassassin
and set option
CRON=1
Integrate with exim
$ sudo apt install sa-exim
Configuration is stored in /etc/exim4/sa-exim.conf
.
Edit the following setting
SAspamcUser: vmail
By defauld sa-exim is disabled. Remove the following lines to enable it
#----------------------------------------------------------------------
# Remove or comment out the following line to enable sa-exim
SAEximRunCond: 0
#----------------------------------------------------------------------
Other parameter that I change
SApermreject: 10.0
You can now restart exim to take you settings into account
$ sudo systemctl restart exim4.service
Integrate with dovecot
SpamAssassin is able to learn from it's mistakes. By using the plugin dovecot-antispam, we train SpamAssassin by just moving email in or out of the spam folder.
First install it with this command
$ sudo apt install dovecot-antispam
Then in file /etc/dovecot/conf.d/20-imap.conf
, modify the option mail_plugins and add antispam to the list
protocol imap {
# Space separated list of plugins to load (default is global mail_plugins).
mail_plugins = $mail_plugins antispam
}
Create file /etc/dovecot/conf.d/90-antispam.conf
plugin {
##################
# GENERIC OPTIONS
# Debugging options
# Uncomment to get the desired debugging behaviour.
# Note that in some cases stderr debugging will not be as
# verbose as syslog debugging due to internal limitations.
#
# antispam_debug_target = syslog
# antispam_debug_target = stderr
# antispam_verbose_debug = 1
antispam_backend = pipe
antispam_trash_pattern_ignorecase = trash;Deleted Items;Deleted Messages
antispam_spam_pattern_ignorecase = Spam;Junk
###########################
# BACKEND SPECIFIC OPTIONS
#
#=====================
# pipe plugin
#
# temporary directory
antispam_pipe_tmpdir = /tmp
# spam/not-spam argument (default unset which will is not what you want)
antispam_pipe_program_spam_arg = -r
antispam_pipe_program_notspam_arg = -k
# binary to pipe mail to
antispam_pipe_program = /usr/bin/spamassassin
}
And finally, reload Dovecot
$ sudo systemctl restart dovecot.service
Integrate in Munin
There is a plugin in Munin to get statistics on the ham/spam values from Spamassassin. To activate it, run the following command
$ sudo ln -s /usr/share/munin/plugins/spamstats /etc/munin/plugins/
Then create file /etc/munin/plugin-conf.d/spamstats
[spamstats]
group adm
env.logfile mail.log
Finally, restart the Munin node
$ sudo systemctl restart munin-node.service
After 5 minutes, you should see your new graph in Munin.