The full list of sessions is up (or here), and I’m very excited just to be attending the conference. Congratulations to all of the speakers!

Some of the sessions I’m particularly excited to attend are:

• An Introduction to Machine Learning by John Melesky – A Portland Python User Group regular and really smart guy in general. Looking forward to getting my feet wet with machine learning.
• Drop ACID and think about data by Bob Ippolito – I think the Portland Python crew that went to Pycon helped convince Bob to bring his presentation to OS Bridge, and I’m glad they did! I work with non-relational databases everyday, but I still design my apps with a relational data model in mind. This should be interesting.
• The Linux Kernel Development model by Greg Kroah-Hartman – Its Greg KH! Come on!
• How Idealist.org uses technology to change the world by Michel Pelletier – Another Portland Pythoneer and really smart guy. Eager to here more about open source at work in an organization. Its always fun to watch Michel present anyway.
• Web Testing with Windmill by Mikeal Rogers – Hardly a day goes by that I don’t tell myself I’m going to improve my test coverage and skills. Windmill looks like an excellent testing tool to help me accomplish that.

There are many many more sessions I’m excited about, but I’m too sick of copying and pasting to mention anymore right now.

So sign up to attend the Open Source Bridge Conference in Portland, OR this June!

The Problem

A memory leak. Apache is eating up memory so quickly that I need to restart it every couple days or risk my entire server grinding to a halt as it starts swapping wildly. I’ve poured over log files and pmap output, but I still can’t figure out where the problem lies. Curse you monolithic in-process architecture!

Actually I know what my problem is, I’m running a mess of modules:

• ssl – 2 certificates on 2 ports
• php5 – blerg, who doesn’t have to run this?
• suphp – I suspect this is my problem, but I can’t prove it. A client’s 3rd party web application requires it, but I think its easily replaceable with FastCGI.
• wsgi – No complaints. Python apps are out-of-process thankfully.
• proxy – Again no complaints. Can’t imagine how this module could cause any problems except it does proxy some large (multi-megabyte, not huge) POSTs at times. I can’t imagine a memory leak could slip into this module without a lot of people noticing.

Solution A: Apache+FastCGI

I love the idea of putting each web application in its own process and letting Apache just act as an HTTP router. FastCGI seems to have all the features I need, and I’m not really worried about the CPU overhead incurred by IPC.

However, there are 2 competing FastCGI modules for Apache, and I have no idea what to choose. Anecdotally the official mod_fastcgi is buggy and fastcgi.com is a spam infested wasteland. However, I’ve found no authoritative source saying: “fastcgi is dead, long live fcgid!” (Lame excuse, I know.)

Solution B: Lighttpd

I know Lighty is the darling of Rails sites, but whenever I stop by its site I’m greeted with a list of recently fixed security bugs, and now it seems as though they’re rewriting the core!

I’m sure Lighty is a high quality intelligently engineered project, but it seems to be the definition of immature. Not necessarily bad (in fact it usually means its progressing quickly!), but perhaps not as reliable as good old workhorses like Apache.

Solution C: Cherokee

I’ve been following Cherokee for some time now and running it locally on my workstation. I love the web interface. I’m usually a very anti-webmin, pro-vim kind of guy, but I’m sick of editing Apache’s config files. I do it about once a month and therefore it always takes lots of double-checking the docs. I don’t know why, but its configuration has just never felt natural to me.

However, the lead Cherokee developer’s bravado is by the most off-putting aspect of the project. He mocks modwsgi and posts simplistic benchmarks showing Cherokee to be the fastest web server, but meanwhile Cherokee churns out numerous bug patch releases in-between feature releases and has yet to reach 1.0 status.

It seems like an excellent project technically, but I’m afraid there will be negative consequences for the lead developers hubris. (I’m not meaning to insult the guy. He’s probably a far better hacker than I’ll ever be. Self-promotion just makes me uncomfortable.)

Solution D: nginx

I don’t know much about nginx except that it works. Basically all I’ve heard about it is:

• It works.
• Its fast. Really fast.

While “working” is definitely my primary objective, nginx seems a bit bare bones for me. I just don’t think I’m the target demographic. I’d kind of like for my web server to handle spawning and kill of FastCGI processes.

nginx feels like git to me. Those who know it: use it and love it. Those who don’t: stand in fear and awe of its unbridled power.

…or maybe its just a nice simple barebones HTTP server…

Conclusions?

I think Solution A: FastCGI is the most sensible. Apache has always served me well, and the memory leak is most likely due to that shoddy suphp module.

Moving my web applications to FastCGI is also the best way to prepare to move to one of these 2nd generation web servers.

However, I’m getting kind of sick of Apache, and the ambiguousness of which FastCGI solution to choose is fairly annoying.

So dear lazyweb, for your everyday web developer consultant looking to run a bunch of PHP and Python web applications, what HTTP server stack should I use? (Debian Lenny packages are a plus.)

Regardless, I find it hilarious that they don’t use IIS to host their web site.

According to the Server field of the response headers they’re using Apache/2.2.3 on CentOS!

CustomLog /srv/hzzah/log/access.log "0.0.0.0 %l %u %t \"%r\" %>s %b"

No more IP addresses logged! Only if you cause an error (404 or 500) will your IP address be logged.

So no javascript, css, or images from remote sites; no cookies; no ads. Can you get any more private?

I did just add jQuery and a bit of JavaScript, so if web2.0 makes you queasy, you’ve been warned.

I should mention that hzzah’s primary goal is for me to have fun with Django, not privacy. So while I’ll definitely warn you if I make any changes, right now its not a top priority to pursue extreme privacy.

1. The client actually needed a subset of the features most blogs offer, so I wouldn’t really have anything to contribute back to an existing project.
2. Blogs are one of the simplest content driven web applications in existence. Wikis are just a bit simpler perhaps. At any rate, creating a blog app is an excellent way to learn a framework.

Python Deployment Decisions

In the past I’ve used CherryPy as my framework and a simple mod_proxy configuration to run the applications behind Apache. Django considers its built-in web server a development tool only, so I figured it was time to explore the myriad of Python web app deployment alternatives: mod_python, FastCGI, modwsgi. I’m sure there are many more, but I’d say those are the big 3.

I had tried to deploy Python web applications on DreamHost using FastCGI before and entered the hell that is deploying Python web apps on shared hosts. So FastCGI wasn’t my first choice this time.

I had also tried mod_python for deploying CherryPy apps on my Linode before and for whatever reason just found mod_proxy to be much easier to setup and manage.

I was kind of eager to try out modwsgi because its been getting a lot of attention lately, so I downloaded the source and compiled it on my Debian Etch server.

Deploying a Django App via modwsgi

modwsgi was quite easy to setup as long as you follow the instructions in their wiki for Django integration. I was hit by bug #3762, but the modwsgi documentation got me through it. (For what its worth the attached wsgi.patch also worked, but I don’t really want to run a patched version of Django.)

One big problem I ran into was sqlite3 gave me OperationalError: unable to open database file whenever I did anything that would write to the database. My database file was owned by www-data (the Apache process owner) and had the permissions 664.

I switched to PostgreSQL, ran syncdb, and everything worked beautifully.

My wsgi script file /srv/spam/eggs/eggs.wsgi:

import os, sys
sys.path.append('/srv/spam')
sys.path.append('/srv/spam/eggs')
os.environ['DJANGO_SETTINGS_MODULE'] = 'eggs.wsgi_settings'
 
import django.core.handlers.wsgi
 
_application = django.core.handlers.wsgi.WSGIHandler()
 
def application(environ, start_response):
    environ['PATH_INFO'] = environ['SCRIPT_NAME'] + environ['PATH_INFO']
    return _application(environ, start_response)

Note I use wsgi_settings instead of my usual settings file. wsgi_settings just imports my main settings file and changes some to their production values.

My Django application actually drops into the /blog/ and /accounts/ folders under a VirtualHost otherwise occupied by static files and some PHP scripts. modwsgi made this easy by putting this in my existing VirtualHost:

WSGIScriptAliasMatch /(blog|accounts)/.* /srv/spam/eggs/eggs.wsgi
 
# A simple Alias directive handles my static files
Alias /static/ /srv/spam/eggs/static/

Bottom Line

I highly recommend using modwsgi for deploying Python web applications. sqlite3 may work for you. In my case its probably best I use PostgreSQL for a number of reasons.