When a few people started calling for the death penalty for spammers, my intial reaction was that it seemed extreme. But the more I’ve had to deal with spam, the more I’ve come around to that point of view. Spammers may not kill any one person, but considering the lost time they cost email users collectively, they are effectively mass murderers, and should be treated accordingly. But that’s not what I’m here to write about.
One of the many slimy things spammers do is forge return addresses, so if a victim email system accepts a message then finds it undeliverable, it will bounce it to the forged address (which is the correct behavior for the email system based on the RFCs).
This causes problems for the victim of the forgery. The victim may then manually or automatically lodge a complaint against the system that bounced the message, even though that system was behaving correctly. This can result in the system being blacklisted, or being disconnected by the ISP.
I got tired of answering bounce-spam complaints that were sent to my ISP, and decided to do something about it. Unfortunately there is no low-computational-cost way to accurately determine whether the message being bounced was spam. (These are messages that already got past various blacklists, greylisting, etc.) I concluded that the best way to deal with the problem was to break my mail system so that it never sends bounce messages (thus violating the RFC).
The system in question is running Qmail. I used to be quite enamored of Qmail, as it was very easy to configure. However, the code is difficult to read and understand, and it is no longer maintained by its author. I’m in the process of migrating to Postfix.
I found the point in the code where Qmail decides to send a bounce message, and changed it in what I thought was a fairly obvious way, to not send the bounce message. I tested it, and it seemed to work. Legitimate mail still got through, but mail I sent into the system in an effort to elicit a bounce message did not do so.
Months later I keep having problems with the /var partition filling up. The most obvious culprit is the maillog. Because of the volume of spam, it grows to about 2 GiB each week, and the log rotation tries to keep four weeks of logs. Every time the /var partition filled up, I deleted the old maillogs, but the interval between occurences kept decreasing.
On Saturday I was doing a full backup of the system, and noticed that while other partitions were backing up over the network at rates over 1 GiB/minute, the /var partition’s rate was under 1 GiB/hour. Something was clearly wrong.
To make a long story short, my patch to Qmail was preventing the bounce messages from being sent over the network to other MTAs, but was not preventing them from accumulating in the mail queue. There were over a million orphan files in /var/qmail/queue/bounce, and about 80,000 related files in of the other hash bucket directories. The Ext3 file system in its default configuration is incredibly slow when dealing with directories containing tens of thousands of files. Various programs that purport to fix Qmail queues did not notice these orphaned bounce files or clean them up.
I’ll have to do one of three things:
- Study the Qmail code more and try to fix my patch.
- Write a tool to run as a cron job to clean up orphaned bounces in the queue.
- Compete my migration to Postfix.