Crowdsourcing my OS Bridge Talk Proposal

  2009-03-30


Update: Submitted my proposal, but please still leave me comments! I can still edit my proposal.

Web server performance has been a hot topic for some time now as Apache hasn’t been the only viable option for a few years.

I want to do a talk at OS Bridge doing practical comparisons between the popular web site hosting options available today. There’s plenty of requests per second comparisons between the major HTTP servers out there, so I want to go beyond and compare a wide variety of metrics in a wide variety of environments.

Here’s what I’m thinking so far, but I’d love your feedback! I’m really excited about the possibility of presenting at a large conference like OS Bridge and want to make my talk as exciting and useful as possible.

Metrics

  • Requests per second (serially* and concurrent)
  • Memory usage
  • Features (extensibility, modularity, etc)
  • Community (support options, tools/modules available, etc)
  • Portability
  • Reliability (can I crash it? how well does it handle load?)
  • Ease of setup and maintenance (floods of patches? hideous config file?)

Web Servers

The usual suspects (will definitely test):

  • Apache 2.2 (prefork and worker mpms)
  • Nginx 0.6.x (latest stable)
  • Lighttpd 1.4.x (latest stable)

Maybes:

  • Nginx 0.5.x (legacy)
  • Nginx 0.7.x (latest dev)
  • Cherokee (not sure if its really popular enough yet to bother)
  • CherryPy and some other language specific or just less known servers would be fun.
  • IIS – It’d be really fun if I could find someone willing to help me run my tests on IIS. I definitely won’t be setting it up myself though!

Test Cases

This is where things become really complicated. I don’t want to just download static files from each server and pretend like thats a real world use case, but at the same time the various environments I could come up with are nearly limitless.

I want to strive to cover the most common environments** as well as a few variations that produce interesting results.

  • WordPress (mod-php where applicable and FastCGI)
  • Drupal (mod-php where applicable and FastCGI)
  • Bugzilla, Redmine, and Trac (to test a few non-trivial apps from 3 popular languages)
  • trivial Django app (mod-python & mod-wsgi on Apache, FastCGI everywhere)
  • trivial Ruby on Rails app (need advice from RoR devs on common deployment methods)
  • trivial Zend Framework app (or is there a more popular php framework?)
  • something from the Java world?
  • maybe something fairly obscure like Seaside?

Conclusion

Wow, that seems like a lot to cover thoroughly, but I think the research will be a lot of fun to do.

Please let me know your thoughts either in the comments, on twitter (@schmichael), on IRC (schmichael again), or via e-mail.



* Its interesting to compare serial to parallel req/sec to give people an idea of how well the technology will scale on a single server. I’ll make sure to note good serial performance can often be turned into good parallel performance by using a load balancing proxy and multiple instances of your application server.
** All Linux based for simplicity unless someone volunteers to run the tests on other platforms.