Spamassassin: Difference between revisions

From wiki
Firewall rules for pyzor + razor
`pyzor discover` command was removed https://github.com/SpamExperts/pyzor/commit/50f2bf5aa47ed863de78c413ff7114f5e54f5a9b
 
(10 intermediate revisions by the same user not shown)
Line 7: Line 7:
* Static Rules: a lot of manually crafted rules by SpamAssassin contributors
* Static Rules: a lot of manually crafted rules by SpamAssassin contributors


{{WIP}}
== Prerequisites ==
This article is part of the [[Emails/Complete|emails]] series. It is assumed that you already covered [[Dovecot]] and [[Exim]].
 
Optional prerequisites:
* [[nftables]] is used as a firewall here. You can however replace it by any firewall you use.
* [[Munin]] allows you to monitor the spam/ham ratio of your installation.


== Install ==
== Install ==


<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
# apt install spamassassin
$ sudo apt install spamassassin
</syntaxhighlight>
</syntaxhighlight>


Line 19: Line 24:
After changing config in <code>/etc/spamassassin/</code>, don't forget tell SpamAssassin to reload config
After changing config in <code>/etc/spamassassin/</code>, don't forget tell SpamAssassin to reload config
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
# service spamassassin reload
$ sudo service spamassassin reload
</syntaxhighlight>
 
=== Report Headers ===
 
SpamAssassin can had headers in the messages it scan. It will help you investigate things in case you get false-positive are false-negative.
 
Add the following lines to <code>/etc/spamassassin/local.cf</code>
<syntaxhighlight lang="shell">
# The status header is used by other programs to read the spam status. Don't modify the part before tests=...
add_header all Status _YESNO_, hits=_HITS_ required=_REQD_ tests=_TESTSSCORES(,)_ autolearn=_AUTOLEARN_
add_header all Details version=_VERSION_ _REPORT_
add_header all Pyzor _PYZOR
</syntaxhighlight>
</syntaxhighlight>


Line 42: Line 35:


<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
# sudo -u vmail sa-learn --spam --progress --dir /var/maildir/<username>/Maildir/.Spam/cur/
$ sudo -u vmail sa-learn --spam --progress --dir /var/maildir/<username>/Maildir/.Spam/cur/
# sudo -u vmail sa-learn --ham --progress --dir /var/maildir/<username>/Maildir/cur/
$ sudo -u vmail sa-learn --ham --progress --dir /var/maildir/<username>/Maildir/cur/
</syntaxhighlight>
</syntaxhighlight>


Line 49: Line 42:


<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
# sudo -u vmail sa-learn --dump magic
$ sudo -u vmail sa-learn --dump magic
</syntaxhighlight>
</syntaxhighlight>


Line 56: Line 49:
==== Install ====
==== Install ====
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
# apt install pyzor  
$ sudo apt install pyzor  
</syntaxhighlight>
</syntaxhighlight>


==== Firewall ====
==== Firewall ====
{{nftables/config|category=Pyzor (Spamassassin)|tcp_port_in=|udp_port_in=|tcp_port_out=24441|udp_port_out=24441|user_out=}}Configure<syntaxhighlight lang="console">
{{nftables/config|category=Pyzor (Spamassassin)|tcp_port_in=|udp_port_in=|tcp_port_out=24441|udp_port_out=24441|user_out=}}
# sudo -u vmail pyzor discover
downloading servers from http://pyzor.sourceforge.net/cgi-bin/inform-servers-0-3-x
# sudo -u vmail pyzor ping
public.pyzor.org:24441  (200, 'OK')
</syntaxhighlight>


=== Razor ===
=== Razor ===
Line 71: Line 59:
==== Install ====
==== Install ====
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
# apt install razor  
$ sudo apt install razor  
</syntaxhighlight>
</syntaxhighlight>


Line 79: Line 67:
==== Configure ====
==== Configure ====
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
# sudo -u vmail razor-admin -create
$ sudo -u vmail razor-admin -create
# sudo -u vmail razor-admin -register
$ sudo -u vmail razor-admin -register
Register successful.  Identity stored in /var/maildir/.razor/identity-xo4OkrHieL
Register successful.  Identity stored in /var/maildir/.razor/identity-xo4OkrHieL
</syntaxhighlight>
</syntaxhighlight>


=== Report Headers ===
SpamAssassin can had headers in the messages it scan. It will help you investigate things in case you get false-positive are false-negative.
Add the following lines to <code>/etc/spamassassin/local.cf</code>
<syntaxhighlight lang="shell">
# The status header is used by other programs to read the spam status. Don't modify the part before tests=...
add_header all Status _YESNO_, hits=_HITS_ required=_REQD_ tests=_TESTSSCORES(,)_ autolearn=_AUTOLEARN_
add_header all Details version=_VERSION_ _REPORT_
add_header all Pyzor _PYZOR_
</syntaxhighlight>
=== Configure service ===
Edit file <code>/etc/default/spamassassin</code> and change the following line<syntaxhighlight lang="shell">
OPTIONS="--create-prefs --max-children 5 -u vmail --listen /run/spamd.socket"
</syntaxhighlight>Create file <code>/etc/spamassassin/spamc.conf</code> with the following content<syntaxhighlight lang="text">
--socket /run/spamd.socket
</syntaxhighlight>It's now time to enable the Spamassassin service<syntaxhighlight lang="console">
$ sudo systemctl enable spamassassin.service
$ sudo systemctl start spamassassin.service
</syntaxhighlight>
=== Cron ===
Spamassassin authors publish updated rules on a daily basis. To stay up-to-date, edit file <code>/etc/default/spamassassin</code> and set option<syntaxhighlight lang="shell">
CRON=1
</syntaxhighlight>
== Integrate with exim ==
== Integrate with exim ==


<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
# apt install sa-exim
$ sudo apt install sa-exim
</syntaxhighlight>
</syntaxhighlight>


Line 93: Line 107:


Edit the following setting
Edit the following setting
<syntaxhighlight lang="apache">
<syntaxhighlight lang="properties">
SAspamcUser: vmail
SAspamcUser: vmail
</syntaxhighlight>
</syntaxhighlight>


By defauld ''sa-exim'' is disabled. Remove the following lines to enable it
By defauld ''sa-exim'' is disabled. Remove the following lines to enable it
<syntaxhighlight lang="apache">
<syntaxhighlight lang="properties">
#----------------------------------------------------------------------
#----------------------------------------------------------------------
# Remove or comment out the following line to enable sa-exim
# Remove or comment out the following line to enable sa-exim
SAEximRunCond: 0
SAEximRunCond: 0
#----------------------------------------------------------------------
#----------------------------------------------------------------------
</syntaxhighlight>
</syntaxhighlight>Other parameter that I change<syntaxhighlight lang="properties">
 
SApermreject: 10.0
You can now restart exim to take you settings into account
</syntaxhighlight>You can now restart exim to take you settings into account
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
# service exim4 restart
$ sudo systemctl restart exim4.service
</syntaxhighlight>
</syntaxhighlight>


Line 116: Line 130:
First install it with this command
First install it with this command
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
# apt install dovecot-antispam
$ sudo apt install dovecot-antispam
</syntaxhighlight>
</syntaxhighlight>


Line 127: Line 141:
</syntaxhighlight>
</syntaxhighlight>


Edit file <code>/etc/dovecot/conf.d/90-antispam.conf</code> and set or edit the following options
Create file <code>/etc/dovecot/conf.d/90-antispam.conf</code>  
<syntaxhighlight lang="shell">
<syntaxhighlight lang="shell">
  antispam_backend = pipe
plugin {
  antispam_trash_pattern_ignorecase = trash;Deleted Items;Deleted Messages
    ##################
  antispam_spam_pattern_ignorecase = Spam;Junk
    # GENERIC OPTIONS
  antispam_pipe_program_spam_arg = -r
 
  antispam_pipe_program_notspam_arg = -k
    # Debugging options
  antispam_pipe_program = /usr/bin/spamassassin
    # Uncomment to get the desired debugging behaviour.
    # Note that in some cases stderr debugging will not be as
    # verbose as syslog debugging due to internal limitations.
    #
    # antispam_debug_target = syslog
    # antispam_debug_target = stderr
    # antispam_verbose_debug = 1
 
    antispam_backend = pipe
 
    antispam_trash_pattern_ignorecase = trash;Deleted Items;Deleted Messages
    antispam_spam_pattern_ignorecase = Spam;Junk
 
 
    ###########################
    # BACKEND SPECIFIC OPTIONS
    #
 
    #=====================
    # pipe plugin
    #
 
    # temporary directory
    antispam_pipe_tmpdir = /tmp
 
    # spam/not-spam argument (default unset which will is not what you want)
    antispam_pipe_program_spam_arg = -r
    antispam_pipe_program_notspam_arg = -k
 
    # binary to pipe mail to
    antispam_pipe_program = /usr/bin/spamassassin
}
</syntaxhighlight>
</syntaxhighlight>


And finally, reload Dovecot
And finally, reload Dovecot
<syntaxhighlight lang="console">
<syntaxhighlight lang="console">
# service dovecot restart
$ sudo systemctl restart dovecot.service
</syntaxhighlight>
</syntaxhighlight>


== Integrate in Munin ==
There is a plugin in [[Munin]] to get statistics on the ham/spam values from Spamassassin. To activate it, run the following command<syntaxhighlight lang="console">
$ sudo ln -s /usr/share/munin/plugins/spamstats /etc/munin/plugins/
</syntaxhighlight>Then create file <code>/etc/munin/plugin-conf.d/spamstats</code><syntaxhighlight lang="ini">
[spamstats]
group adm
env.logfile mail.log
</syntaxhighlight>Finally, restart the Munin node<syntaxhighlight lang="console">
$ sudo systemctl restart munin-node.service
</syntaxhighlight>After 5 minutes, you should see your new graph in Munin.
[[Category:Email Server]]
[[Category:Email Server]]
[[Category:Linux Server]]
[[Category:Linux Server]]
[[Category:Munin]]

Latest revision as of 06:22, 3 May 2018

SpamAssassin is a spam detection software intended to be run on your mail server. It rank mail using several criteria criteria that can be put in the following families

  • DNS Whitelist/Blacklist: does the server that sent you the email sent spam before?
  • URI Blacklist: does the body of the message contain links to some bad sites?
  • Distributed Spam Hashes: does someone reported the same message as spam already?
  • Bayesian Filter: compare email to your past spam and ham
  • SPF/DKIM: check is the from email address that you see is legitimate
  • Static Rules: a lot of manually crafted rules by SpamAssassin contributors

Prerequisites

This article is part of the emails series. It is assumed that you already covered Dovecot and Exim.

Optional prerequisites:

  • nftables is used as a firewall here. You can however replace it by any firewall you use.
  • Munin allows you to monitor the spam/ham ratio of your installation.

Install

$ sudo apt install spamassassin

Configure

After changing config in /etc/spamassassin/, don't forget tell SpamAssassin to reload config

$ sudo service spamassassin reload

Bayesian filter

To reach a good efficiency, SpamAssassin Bayesian filter need to be trained with both spam and ham messages. You can use your actual mailbox for that but note the following points:

  • Be sure that the folders you use for training contain only spam or ham. If a folder contain a mix of them, SpamAssassin will learn wrong info and produce bad quality results
  • To be effective you need between 1000 and 5000 messages each of both spam and ham.
  • You need to have more ham than spam to train. Otherwise, SpamAssassin might become biased toward spam.
$ sudo -u vmail sa-learn --spam --progress --dir /var/maildir/<username>/Maildir/.Spam/cur/
$ sudo -u vmail sa-learn --ham --progress --dir /var/maildir/<username>/Maildir/cur/

To check the status of the database, you can run

$ sudo -u vmail sa-learn --dump magic

Pyzor

Install

$ sudo apt install pyzor

Firewall

Assuming that you configured nftables as described, you can edit file /etc/nftables/main_config.conf and add

# Pyzor (Spamassassin)
add element  inet main  tcp_port_out { 24441 }
add element  inet main  udp_port_out { 24441 }

and activate it using

$ sudo /etc/nftables/reload_main.conf

Razor

Install

$ sudo apt install razor

Firewall

Assuming that you configured nftables as described, you can edit file /etc/nftables/main_config.conf and add

# Razor (Spamassassin)
add element  inet main  tcp_port_out { 2703 }

and activate it using

$ sudo /etc/nftables/reload_main.conf

Configure

$ sudo -u vmail razor-admin -create
$ sudo -u vmail razor-admin -register
Register successful.  Identity stored in /var/maildir/.razor/identity-xo4OkrHieL

Report Headers

SpamAssassin can had headers in the messages it scan. It will help you investigate things in case you get false-positive are false-negative.

Add the following lines to /etc/spamassassin/local.cf

# The status header is used by other programs to read the spam status. Don't modify the part before tests=...
add_header all Status _YESNO_, hits=_HITS_ required=_REQD_ tests=_TESTSSCORES(,)_ autolearn=_AUTOLEARN_
add_header all Details version=_VERSION_ _REPORT_
add_header all Pyzor _PYZOR_

Configure service

Edit file /etc/default/spamassassin and change the following line

OPTIONS="--create-prefs --max-children 5 -u vmail --listen /run/spamd.socket"

Create file /etc/spamassassin/spamc.conf with the following content

--socket /run/spamd.socket

It's now time to enable the Spamassassin service

$ sudo systemctl enable spamassassin.service
$ sudo systemctl start spamassassin.service

Cron

Spamassassin authors publish updated rules on a daily basis. To stay up-to-date, edit file /etc/default/spamassassin and set option

CRON=1

Integrate with exim

$ sudo apt install sa-exim

Configuration is stored in /etc/exim4/sa-exim.conf.

Edit the following setting

SAspamcUser: vmail

By defauld sa-exim is disabled. Remove the following lines to enable it

#----------------------------------------------------------------------
# Remove or comment out the following line to enable sa-exim
SAEximRunCond: 0
#----------------------------------------------------------------------

Other parameter that I change

SApermreject: 10.0

You can now restart exim to take you settings into account

$ sudo systemctl restart exim4.service

Integrate with dovecot

SpamAssassin is able to learn from it's mistakes. By using the plugin dovecot-antispam, we train SpamAssassin by just moving email in or out of the spam folder.

First install it with this command

$ sudo apt install dovecot-antispam

Then in file /etc/dovecot/conf.d/20-imap.conf, modify the option mail_plugins and add antispam to the list

protocol imap {
  # Space separated list of plugins to load (default is global mail_plugins).
  mail_plugins = $mail_plugins antispam
}

Create file /etc/dovecot/conf.d/90-antispam.conf

plugin {
    ##################
    # GENERIC OPTIONS

    # Debugging options
    # Uncomment to get the desired debugging behaviour.
    # Note that in some cases stderr debugging will not be as
    # verbose as syslog debugging due to internal limitations.
    #
    # antispam_debug_target = syslog
    # antispam_debug_target = stderr
    # antispam_verbose_debug = 1

    antispam_backend = pipe

    antispam_trash_pattern_ignorecase = trash;Deleted Items;Deleted Messages
    antispam_spam_pattern_ignorecase = Spam;Junk


    ###########################
    # BACKEND SPECIFIC OPTIONS
    #

    #=====================
    # pipe plugin
    #

    # temporary directory
    antispam_pipe_tmpdir = /tmp

    # spam/not-spam argument (default unset which will is not what you want)
    antispam_pipe_program_spam_arg = -r
    antispam_pipe_program_notspam_arg = -k

    # binary to pipe mail to
    antispam_pipe_program = /usr/bin/spamassassin
}

And finally, reload Dovecot

$ sudo systemctl restart dovecot.service

Integrate in Munin

There is a plugin in Munin to get statistics on the ham/spam values from Spamassassin. To activate it, run the following command

$ sudo ln -s /usr/share/munin/plugins/spamstats /etc/munin/plugins/

Then create file /etc/munin/plugin-conf.d/spamstats

[spamstats]
group adm
env.logfile mail.log

Finally, restart the Munin node

$ sudo systemctl restart munin-node.service

After 5 minutes, you should see your new graph in Munin.