SMTP Server Cage Match
Last week I released smtproutes: A simple, Sinatra inspired, SMTP routing server.
Read What if SMTP and Sinatra Had a Baby?
Out of curiosity, I wanted to benchmark smtproutes vs. some other libraries that one might use for building SMTP-backed-web-services.
In this post I compare the performance of:
- smtproutes (https://github.com/bcoe/smtproutes)
- Lamson (http://lamsonproject.org/)
- Pymilter (http://bmsi.com/python/milter.html)
“Three approaches to SMTP scripting enter, one approach to scripting SMTP leaves!”
The Test
I was interested in seeing the number of messages that each server could process per second, under various loads.
- I created a mailing script which repeatedly mailed an 8kb message for the first benchmark and an 88kb message for the second, testing how each server dealt with small and relatively large email messages respectively.
- I created a mail processing script for each of the servers. This received the messages and kept a running total of the number processed per second.
- I tested each mail server with 1, 4, 8, and 16 concurrent mailing scripts, to simulate various loads on the server.
- All the benchmarks were run in an Ubuntu Maverick VMware Image, on my Intel Core 2 Duo 2.26Ghz Macbook Pro.
The Results
Conclusions
Pymilter is painfully slow. Why? there’s a lot of OS level overhead that comes with receiving email via sendmail. I’m sure a Unix guru could configure sendmail in such a way that Pymilter performs much more admirably (I’m not that Guru).
Lamson is fast, it outperforms smtproutes when there are a small number of concurrent connections (a single SMTP connection sending a lot of messages to the server).
smtproutes, which uses an underlying process pool, excels when there are many concurrent connections to the server. With 16 concurrent mail scripts running, it more than doubles Lamson’s throughput.
So What?
The performance of Lamson and smtproutes helps validate the argument I’ve been putting forward in my last few blog posts (Writing a Secure SMTP Server in Python, What if SMTP and Sinatra Had a Baby?):
Running a lightweight SMTP server, independent of all the standard underpinnings of a captital ‘E’ email server, makes a ton of sense for a wide variety of applications that can be built using Email as a Service (EaaS, yep I’m going to keep using this horrible acronym).
— Ben (@benjamincoe)
