Difference between revisions of "Spamassassin"

From wiki
(Learning from dovecot)
(`pyzor discover` command was removed https://github.com/SpamExperts/pyzor/commit/50f2bf5aa47ed863de78c413ff7114f5e54f5a9b)
 
(18 intermediate revisions by the same user not shown)
Line 1: Line 1:
{{WIP}}
+
SpamAssassin is a spam detection software intended to be run on your mail server. It rank mail using several criteria criteria that can be put in the following families
 +
* DNS Whitelist/Blacklist: does the server that sent you the email sent spam before?
 +
* URI Blacklist: does the body of the message contain links to some bad sites?
 +
* Distributed Spam Hashes: does someone reported the same message as spam already?
 +
* Bayesian Filter: compare email to your past spam and ham
 +
* SPF/DKIM: check is the ''from'' email address that you see is legitimate
 +
* Static Rules: a lot of manually crafted rules by SpamAssassin contributors
 +
 
 +
== Prerequisites ==
 +
This article is part of the [[Emails/Complete|emails]] series. It is assumed that you already covered [[Dovecot]] and [[Exim]].
 +
 
 +
Optional prerequisites:
 +
* [[nftables]] is used as a firewall here. You can however replace it by any firewall you use.
 +
* [[Munin]] allows you to monitor the spam/ham ratio of your installation.
  
 
== Install ==
 
== Install ==
  
 
<syntaxhighlight lang="console">
 
<syntaxhighlight lang="console">
# apt install spamassassin
+
$ sudo apt install spamassassin
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Line 11: Line 24:
 
After changing config in <code>/etc/spamassassin/</code>, don't forget tell SpamAssassin to reload config
 
After changing config in <code>/etc/spamassassin/</code>, don't forget tell SpamAssassin to reload config
 
<syntaxhighlight lang="console">
 
<syntaxhighlight lang="console">
# service spamassassin reload
+
$ sudo service spamassassin reload
</syntaxhighlight>
 
 
 
=== Report Headers ===
 
 
 
SpamAssassin can had headers in the messages it scan. It will help you investigate things in case you get false-positive are false-negative.
 
 
 
Add the following lines to <code>/etc/spamassassin/local.cf</code>
 
<syntaxhighlight lang="shell">
 
add_header all Status _YESNO_, hits=_HITS_ tests=_TESTSSCORES(,)_ autolearn=_AUTOLEARN_
 
add_header all Details required=_REQD_ version=_VERSION_ _REPORT_
 
add_header all Pyzor _PYZOR
 
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Line 33: Line 35:
  
 
<syntaxhighlight lang="console">
 
<syntaxhighlight lang="console">
# su vmail -s /bin/sh -c "sa-learn --spam --progress --dir /var/maildir/<username>/Maildir/.Spam/cur/"
+
$ sudo -u vmail sa-learn --spam --progress --dir /var/maildir/<username>/Maildir/.Spam/cur/
# su vmail -s /bin/sh -c "sa-learn --ham --progress --dir /var/maildir/<username>/Maildir/cur/"
+
$ sudo -u vmail sa-learn --ham --progress --dir /var/maildir/<username>/Maildir/cur/
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Line 40: Line 42:
  
 
<syntaxhighlight lang="console">
 
<syntaxhighlight lang="console">
# su vmail -s /bin/sh -c "sa-learn --dump magic"
+
$ sudo -u vmail sa-learn --dump magic
 
</syntaxhighlight>
 
</syntaxhighlight>
  
 +
=== Pyzor ===
 +
 +
==== Install ====
 +
<syntaxhighlight lang="console">
 +
$ sudo apt install pyzor
 +
</syntaxhighlight>
 +
 +
==== Firewall ====
 +
{{nftables/config|category=Pyzor (Spamassassin)|tcp_port_in=|udp_port_in=|tcp_port_out=24441|udp_port_out=24441|user_out=}}
 +
 +
=== Razor ===
 +
 +
==== Install ====
 +
<syntaxhighlight lang="console">
 +
$ sudo apt install razor
 +
</syntaxhighlight>
 +
 +
==== Firewall ====
 +
{{nftables/config|category=Razor (Spamassassin)|tcp_port_in=|udp_port_in=|tcp_port_out=2703|udp_port_out=|user_out=}}
 +
 +
==== Configure ====
 +
<syntaxhighlight lang="console">
 +
$ sudo -u vmail razor-admin -create
 +
$ sudo -u vmail razor-admin -register
 +
Register successful.  Identity stored in /var/maildir/.razor/identity-xo4OkrHieL
 +
</syntaxhighlight>
 +
 +
=== Report Headers ===
 +
 +
SpamAssassin can had headers in the messages it scan. It will help you investigate things in case you get false-positive are false-negative.
 +
 +
Add the following lines to <code>/etc/spamassassin/local.cf</code>
 +
<syntaxhighlight lang="shell">
 +
# The status header is used by other programs to read the spam status. Don't modify the part before tests=...
 +
add_header all Status _YESNO_, hits=_HITS_ required=_REQD_ tests=_TESTSSCORES(,)_ autolearn=_AUTOLEARN_
 +
add_header all Details version=_VERSION_ _REPORT_
 +
add_header all Pyzor _PYZOR_
 +
</syntaxhighlight>
 +
 +
=== Configure service ===
 +
Edit file <code>/etc/default/spamassassin</code> and change the following line<syntaxhighlight lang="shell">
 +
OPTIONS="--create-prefs --max-children 5 -u vmail --listen /run/spamd.socket"
 +
</syntaxhighlight>Create file <code>/etc/spamassassin/spamc.conf</code> with the following content<syntaxhighlight lang="text">
 +
--socket /run/spamd.socket
 +
</syntaxhighlight>It's now time to enable the Spamassassin service<syntaxhighlight lang="console">
 +
$ sudo systemctl enable spamassassin.service
 +
$ sudo systemctl start spamassassin.service
 +
</syntaxhighlight>
 +
 +
=== Cron ===
 +
Spamassassin authors publish updated rules on a daily basis. To stay up-to-date, edit file <code>/etc/default/spamassassin</code> and set option<syntaxhighlight lang="shell">
 +
CRON=1
 +
</syntaxhighlight>
 
== Integrate with exim ==
 
== Integrate with exim ==
  
 
<syntaxhighlight lang="console">
 
<syntaxhighlight lang="console">
# apt install sa-exim
+
$ sudo apt install sa-exim
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Line 52: Line 107:
  
 
Edit the following setting
 
Edit the following setting
<syntaxhighlight lang="apache">
+
<syntaxhighlight lang="properties">
 
SAspamcUser: vmail
 
SAspamcUser: vmail
 
</syntaxhighlight>
 
</syntaxhighlight>
  
 
By defauld ''sa-exim'' is disabled. Remove the following lines to enable it
 
By defauld ''sa-exim'' is disabled. Remove the following lines to enable it
<syntaxhighlight lang="apache">
+
<syntaxhighlight lang="properties">
 
#----------------------------------------------------------------------
 
#----------------------------------------------------------------------
 
# Remove or comment out the following line to enable sa-exim
 
# Remove or comment out the following line to enable sa-exim
 
SAEximRunCond: 0
 
SAEximRunCond: 0
 
#----------------------------------------------------------------------
 
#----------------------------------------------------------------------
</syntaxhighlight>
+
</syntaxhighlight>Other parameter that I change<syntaxhighlight lang="properties">
 
+
SApermreject: 10.0
You can now restart exim to take you settings into account
+
</syntaxhighlight>You can now restart exim to take you settings into account
 
<syntaxhighlight lang="console">
 
<syntaxhighlight lang="console">
# service exim4 restart
+
$ sudo systemctl restart exim4.service
 
</syntaxhighlight>
 
</syntaxhighlight>
  
 
== Integrate with dovecot ==
 
== Integrate with dovecot ==
  
 +
SpamAssassin is able to learn from it's mistakes. By using the plugin ''dovecot-antispam'', we train SpamAssassin by just moving email in or out of the spam folder.
 +
 +
First install it with this command
 
<syntaxhighlight lang="console">
 
<syntaxhighlight lang="console">
# apt install dovecot-antispam
+
$ sudo apt install dovecot-antispam
 
</syntaxhighlight>
 
</syntaxhighlight>
  
In file <code>/etc/dovecot/conf.d/20-imap.conf</code>, modify the option ''mail_plugins'' and add ''antispam'' to the list
+
Then in file <code>/etc/dovecot/conf.d/20-imap.conf</code>, modify the option ''mail_plugins'' and add ''antispam'' to the list
 
<syntaxhighlight lang="shell">
 
<syntaxhighlight lang="shell">
 
protocol imap {
 
protocol imap {
Line 83: Line 141:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Edit file <code>/etc/dovecot/conf.d/90-antispam.conf</code> and set or edit the following options
+
Create file <code>/etc/dovecot/conf.d/90-antispam.conf</code>  
 
<syntaxhighlight lang="shell">
 
<syntaxhighlight lang="shell">
  antispam_backend = pipe
+
plugin {
  antispam_trash_pattern_ignorecase = trash;Deleted Items;Deleted Messages
+
    ##################
  antispam_spam_pattern_ignorecase = Spam;Junk
+
    # GENERIC OPTIONS
  antispam_pipe_program_spam_arg = -r
+
 
  antispam_pipe_program_notspam_arg = -k
+
    # Debugging options
  antispam_pipe_program = /usr/bin/spamassassin
+
    # Uncomment to get the desired debugging behaviour.
 +
    # Note that in some cases stderr debugging will not be as
 +
    # verbose as syslog debugging due to internal limitations.
 +
    #
 +
    # antispam_debug_target = syslog
 +
    # antispam_debug_target = stderr
 +
    # antispam_verbose_debug = 1
 +
 
 +
    antispam_backend = pipe
 +
 
 +
    antispam_trash_pattern_ignorecase = trash;Deleted Items;Deleted Messages
 +
    antispam_spam_pattern_ignorecase = Spam;Junk
 +
 
 +
 
 +
    ###########################
 +
    # BACKEND SPECIFIC OPTIONS
 +
    #
 +
 
 +
    #=====================
 +
    # pipe plugin
 +
    #
 +
 
 +
    # temporary directory
 +
    antispam_pipe_tmpdir = /tmp
 +
 
 +
    # spam/not-spam argument (default unset which will is not what you want)
 +
    antispam_pipe_program_spam_arg = -r
 +
    antispam_pipe_program_notspam_arg = -k
 +
 
 +
    # binary to pipe mail to
 +
    antispam_pipe_program = /usr/bin/spamassassin
 +
}
 
</syntaxhighlight>
 
</syntaxhighlight>
  
 +
And finally, reload Dovecot
 +
<syntaxhighlight lang="console">
 +
$ sudo systemctl restart dovecot.service
 +
</syntaxhighlight>
  
[[Category:Install]]
+
== Integrate in Munin ==
 +
There is a plugin in [[Munin]] to get statistics on the ham/spam values from Spamassassin. To activate it, run the following command<syntaxhighlight lang="console">
 +
$ sudo ln -s /usr/share/munin/plugins/spamstats /etc/munin/plugins/
 +
</syntaxhighlight>Then create file <code>/etc/munin/plugin-conf.d/spamstats</code><syntaxhighlight lang="ini">
 +
[spamstats]
 +
group adm
 +
env.logfile mail.log
 +
</syntaxhighlight>Finally, restart the Munin node<syntaxhighlight lang="console">
 +
$ sudo systemctl restart munin-node.service
 +
</syntaxhighlight>After 5 minutes, you should see your new graph in Munin.
 +
[[Category:Email Server]]
 +
[[Category:Linux Server]]
 +
[[Category:Munin]]

Latest revision as of 06:22, 3 May 2018

SpamAssassin is a spam detection software intended to be run on your mail server. It rank mail using several criteria criteria that can be put in the following families

  • DNS Whitelist/Blacklist: does the server that sent you the email sent spam before?
  • URI Blacklist: does the body of the message contain links to some bad sites?
  • Distributed Spam Hashes: does someone reported the same message as spam already?
  • Bayesian Filter: compare email to your past spam and ham
  • SPF/DKIM: check is the from email address that you see is legitimate
  • Static Rules: a lot of manually crafted rules by SpamAssassin contributors

Prerequisites

This article is part of the emails series. It is assumed that you already covered Dovecot and Exim.

Optional prerequisites:

  • nftables is used as a firewall here. You can however replace it by any firewall you use.
  • Munin allows you to monitor the spam/ham ratio of your installation.

Install

$ sudo apt install spamassassin

Configure

After changing config in /etc/spamassassin/, don't forget tell SpamAssassin to reload config

$ sudo service spamassassin reload

Bayesian filter

To reach a good efficiency, SpamAssassin Bayesian filter need to be trained with both spam and ham messages. You can use your actual mailbox for that but note the following points:

  • Be sure that the folders you use for training contain only spam or ham. If a folder contain a mix of them, SpamAssassin will learn wrong info and produce bad quality results
  • To be effective you need between 1000 and 5000 messages each of both spam and ham.
  • You need to have more ham than spam to train. Otherwise, SpamAssassin might become biased toward spam.
$ sudo -u vmail sa-learn --spam --progress --dir /var/maildir/<username>/Maildir/.Spam/cur/
$ sudo -u vmail sa-learn --ham --progress --dir /var/maildir/<username>/Maildir/cur/

To check the status of the database, you can run

$ sudo -u vmail sa-learn --dump magic

Pyzor

Install

$ sudo apt install pyzor

Firewall

Assuming that you configured nftables as described, you can edit file /etc/nftables/main_config.conf and add

# Pyzor (Spamassassin)
add element  inet main  tcp_port_out { 24441 }
add element  inet main  udp_port_out { 24441 }

and activate it using

$ sudo /etc/nftables/reload_main.conf

Razor

Install

$ sudo apt install razor

Firewall

Assuming that you configured nftables as described, you can edit file /etc/nftables/main_config.conf and add

# Razor (Spamassassin)
add element  inet main  tcp_port_out { 2703 }

and activate it using

$ sudo /etc/nftables/reload_main.conf

Configure

$ sudo -u vmail razor-admin -create
$ sudo -u vmail razor-admin -register
Register successful.  Identity stored in /var/maildir/.razor/identity-xo4OkrHieL

Report Headers

SpamAssassin can had headers in the messages it scan. It will help you investigate things in case you get false-positive are false-negative.

Add the following lines to /etc/spamassassin/local.cf

# The status header is used by other programs to read the spam status. Don't modify the part before tests=...
add_header all Status _YESNO_, hits=_HITS_ required=_REQD_ tests=_TESTSSCORES(,)_ autolearn=_AUTOLEARN_
add_header all Details version=_VERSION_ _REPORT_
add_header all Pyzor _PYZOR_

Configure service

Edit file /etc/default/spamassassin and change the following line

OPTIONS="--create-prefs --max-children 5 -u vmail --listen /run/spamd.socket"

Create file /etc/spamassassin/spamc.conf with the following content

--socket /run/spamd.socket

It's now time to enable the Spamassassin service

$ sudo systemctl enable spamassassin.service
$ sudo systemctl start spamassassin.service

Cron

Spamassassin authors publish updated rules on a daily basis. To stay up-to-date, edit file /etc/default/spamassassin and set option

CRON=1

Integrate with exim

$ sudo apt install sa-exim

Configuration is stored in /etc/exim4/sa-exim.conf.

Edit the following setting

SAspamcUser: vmail

By defauld sa-exim is disabled. Remove the following lines to enable it

#----------------------------------------------------------------------
# Remove or comment out the following line to enable sa-exim
SAEximRunCond: 0
#----------------------------------------------------------------------

Other parameter that I change

SApermreject: 10.0

You can now restart exim to take you settings into account

$ sudo systemctl restart exim4.service

Integrate with dovecot

SpamAssassin is able to learn from it's mistakes. By using the plugin dovecot-antispam, we train SpamAssassin by just moving email in or out of the spam folder.

First install it with this command

$ sudo apt install dovecot-antispam

Then in file /etc/dovecot/conf.d/20-imap.conf, modify the option mail_plugins and add antispam to the list

protocol imap {
  # Space separated list of plugins to load (default is global mail_plugins).
  mail_plugins = $mail_plugins antispam
}

Create file /etc/dovecot/conf.d/90-antispam.conf

plugin {
    ##################
    # GENERIC OPTIONS

    # Debugging options
    # Uncomment to get the desired debugging behaviour.
    # Note that in some cases stderr debugging will not be as
    # verbose as syslog debugging due to internal limitations.
    #
    # antispam_debug_target = syslog
    # antispam_debug_target = stderr
    # antispam_verbose_debug = 1

    antispam_backend = pipe

    antispam_trash_pattern_ignorecase = trash;Deleted Items;Deleted Messages
    antispam_spam_pattern_ignorecase = Spam;Junk


    ###########################
    # BACKEND SPECIFIC OPTIONS
    #

    #=====================
    # pipe plugin
    #

    # temporary directory
    antispam_pipe_tmpdir = /tmp

    # spam/not-spam argument (default unset which will is not what you want)
    antispam_pipe_program_spam_arg = -r
    antispam_pipe_program_notspam_arg = -k

    # binary to pipe mail to
    antispam_pipe_program = /usr/bin/spamassassin
}

And finally, reload Dovecot

$ sudo systemctl restart dovecot.service

Integrate in Munin

There is a plugin in Munin to get statistics on the ham/spam values from Spamassassin. To activate it, run the following command

$ sudo ln -s /usr/share/munin/plugins/spamstats /etc/munin/plugins/

Then create file /etc/munin/plugin-conf.d/spamstats

[spamstats]
group adm
env.logfile mail.log

Finally, restart the Munin node

$ sudo systemctl restart munin-node.service

After 5 minutes, you should see your new graph in Munin.