Self Hosted DMARC reports

You've implemented DMARC for some reason (deliverability) and now you'd like to read the reports to see whats up. Somehow, the only solutions people talk about, even when you ask specifically about a self hosted option, are things like Dmarcian, Postmarkapp and Fraudmarc.

There are a a couple of solutions out there, but some are insane in regards to dependencies. Do you want to go through some AWS + Docker shit ? I didn't even bother reading about Fraudmarc "community edition" beyond the diagram. There's parsedmarc which seems really nice, but in my case, I'm not ready to invest in more servers (parsedemarc requires Splunk, Elastisearch and/or Kafka), it's way too much for what I expect out of a Dmarc report.

Techsneeze has published a couple of scripts to save the day. I'm writing this because their pages aren't so easy to find (too many "free DMARC reporting services" are all over the search results). But also, I'll try to gather everything in one place to get things working.

For this, you've already configured DMARC and you are already receiving reports. With the following, you will need to give access to the email account where you receive reports to a script. In my case, I have the public DMARC report email, which receives reports from other providers, I then forward those emails to another account that is unknown to the outside world, this way I can avoid issues with the script trying to read spam.

As usual, this stuff was done on Debian stable (Jessie), [insert disclaimer].

Parse

The dmarcts-report-parser script will take care of magically going through the emails, extracting data from zipped (or gunzipped) XML files and uploading them to a database. This can be done from any machine, as long as it has access to the Database server.

Prepare the database

The database tables will be created automatically by the script. All we need to do here is go to your Mysql/MariaDB server and create an empty database and a user. [citation needed]

You might need to add some options to your database server, I did. I had to edit /etc/mysql/mariadb.conf.d/50-server.cnf and add to the [mysqld] section:

innodb_large_prefix = on innodb_file_format = barracuda innodb_file_per_table = 1

You might need to restart the server (not reload)

systemctl restart mariadb

Or

/etc/init.d/mysql restart

I don't know, I'm getting old.

Install some libs

apt-get install libmail-imapclient-perl libmime-tools-perl \ libxml-simple-perl libclass-dbi-mysql-perl \ libio-socket-inet6-perl libio-socket-ip-perl libperlio-gzip-perl \ libmail-mbox-messageparser-perl unzip

Get the code

We'll assume the code goes in /usr/local/, of course, you can install it wherever you want...

cd /usr/local/ git clone https://github.com/techsneeze/dmarcts-report-parser.git

Configure the parser

We just need to add the database credentials as well as the email account details to connect to, via IMAP. Note that this script will check emails from the Inbox (or a specified IMAP folder) and move the emails to a folder called "processed" once a report is dealt with.

cd /usr/local/dmarcts-report-parser/ cp dmarcts-report-parser.conf.sample dmarcts-report-parser.conf

There are a few options that can be changed, in my case, I just want to connect to a a mailbox that contains the report emails and upload the data to a database. Hence, all I changed was the $db... stuff and the $imap... variables.

If you're already here, and you want to change other settings, the configuration file is self-explanatory and very basic. You might need/want to change $imapreadfolder to point to the folder containing DMARC reports.

Parse !

cd /usr/local/dmarcts-report-parser/ ./dmarcts-report-parser.pl -i

And that's it ! Well, you might have some errors to fix, but perhaps with the few tips here, you might not even.

Crontab It

Once you're setup, it's possible that you might want to automate this, why not set up a crontab ?

If so, edit /etc/crontab:

0 5 * * * user /usr/local/dmarcts-report-parser/dmarcts-report-parser.pl -i

Note, you might want to adjust permissions so that a specific user can do this, instead of root. In that case, you should pick a user, and let's say:

chown -R user:user /usr/local/dmarcts-report-parser/ chmod 750 /usr/local/dmarcts-report-parser/

View

The Techsneeze people have also written and provided a PHP script to extract stuff from the database and print a web page with the reports. Simple and efficient. You could always write your own thing if you need, as everything is in the database. I really like that they've separated the 2 parts.

Install the Viewer

We'll do the same to install dmarcts-report-viewer, however, this should be done on a web server and will need access to the database. Again, I used /usr/local/, you can do whatever makes you happy.

cd /usr/local/ git clone https://github.com/techsneeze/dmarcts-report-viewer.git

Configure Viewer

The configuration is like the parser, but only needs the database part, so... simpler.

cd /usr/local/dmarcts-report-viewer/ cp dmarcts-report-viewer-config.php.sample dmarcts-report-viewer-config.php

Then edit dmarcts-report-viewer-config.php and that's it.

Configure Apache

All Apache needs is to open the file /usr/local/dmarcts-report-viewer/dmarcts-report-viewer.php and that file must be in the same directory as the config file (dmarcts-report-viewer-config.php).

In my case, I created an Apache config and included that in the Virtual Host that will need access to this. This is the part that you'll need to adapt to however you've set up your life.

I created and edited the file /etc/apache2/conf-available/dmarcts-report-viewer.conf with:

Alias /dmarc "/usr/local/dmarcts-report-viewer/" <Directory /usr/local/dmarcts-report-viewer/> AllowOverride All Require all granted </Directory>

In a Virtual Host, the one I use for accessing other things like Munin, Awstats and other such things, I added:

Include /etc/apache2/conf-available/dmarcts-report-viewer.conf

And then, I just need to visit that VirtualHost/dmarc/dmarcts-report-viewer.php and voilà, it's magic.

Clean up permissions

You might need to adjust the following to whatever is the web user that needs access to this file (probably www-data, which is my case):

chown root:www-data /usr/local/dmarcts-report-viewer/ chmod 750 /usr/local/dmarcts-report-viewer/

That should work, if it doesn't.... well, I turned off comments because humans are horrible with their spam bots.