Cloud based SMTP filtering for spam and virus signatures detection
Comendo came to us looking to improve the reliability, efficiency and accuracy of their spam filter integrated with the MTA (Mail Transfer Agent) from MessageSystems. The original filter did not have good test coverage and did not meet performance requirements. We also created an analytics toolkit to gather and disect relevant data.
During our long-term work relations we also developed a virus filtering service that allowed to identify new virus signatures in spam traffic.
The client offers a comprehensive email service to their corporate and govt customers serving over 10k domains. The solution is based on the MTA from MessageSystems — Ecelerity.
To counter spam, the client uses its own anti-spam daemon which scans an SMTP flow reaching a processing rate of over 50 messages per second. The original daemon implementation didn’t meet performance requirements.
We took over the codebase and created a scalable, semi-self learning spam filter compatible with any MTA. New content based filtering algorithms have been added.
In the MTA, filter
client & daemon
To accelerate development speed & optimise computation
In the central database to store email signatures
Data handling, analytics and self-learning features
Communication between the database and 3rd party software
In later releases, we made improvements to scalability, architecture, the central database and compatibility with 3rd party software. We added several sophisticated spam-filtering algorithms, an advanced admin toolset and detailed statistics. We also improved compatibility with other daemons such as a virus filter.
Today the spam filter has a processing rate of over 50 messages per second and is fully scalable. The system can update itself dynamically without interrupting the flow of emails.
We delivered 4 key outcomes:
The system can handle high peaks with a wider stack of content based algorithms. Its CPU and memory consumption has been reduced
The newly created daemon has been test-driven developed which allowed to introduce changes to the filtering algorithms quicker
The system learns from earlier spam-attacks, with a growing central signature database.
Puppet was introduced to ease off the deployment process
We’d love to discuss your project