The state of Python WSGI deployment

Like Ian Bicking, I've been thinking about deployment a lot. The landscape in Python web serving has been quite dull and littered with some bad design decisions. The author of mod_wsgi, Graham Dumpleton should have started writing a *NIX only clone of Apache/mod_wsgi without Apache a long time ago. I'm sure it's a lot of work and there's a lot to do considering his code quality standard. But how lame is it to have Apache in the stack when most everyone uses NGINX for static file serving and Apache only for WSGI apps?

Then there is uWSGI. It's somewhat promising, but again an unfortunate design decision was made to make plugins for multiple platforms (NGINX, Apache ..). Why not focus on a nice configuration, speed and stability and let reverse proxy servers handle the rest? (I haven't really bothered with CherryPy and Cherokee.)

We have all the pieces of the puzzle; deployment should be automated, fast and reliable. I shouldn't need a sysadmin. We need Heroku for WSGI! But it should do even more for us.

Wishlist

Easy to get up and going locally with a Django project - ideally one or two commands until there's an activated virtualenv and my preferred Django project skeleton is inside TextMate.
A directory of deployment configuration where I define the caching schema, number of workers, staging and live URL's, perhaps super user accounts.
Command line tools to load and dump data from all sources (PostgreSQL, redis, CouchDB, user uploaded files etc.). Some S3 backup points perhaps. Something like the Bundle feature in Heroku.
Easy migration strategy for scaling to multiple servers.
Easy inspection of data and node health.
Easy destroy/create of nodes.
Easy local testing - both quick for code-alt+tab-refresh action and a full stack VirtualBox/VMWare deployment.
Easy CDN management at the application level - Environments should be aware of how to serve static media.
Easy to venture off the beaten path - this is the enemy of automation!

Gunicorn looks promising for a WSGI server. Silver Lining looks promising for server management. I'll update this entry as I have stuff to show off. I'm bound to scratch this itch in one way or another - or at least discover something nice.

UPDATE 15.03.2010

I've been piecing together an automation tool for cloud deployment. It's called kraftwerk and it's ideal for managing many simple WSGI app deployments.

Here's my sales pitch:

# kraftwerk create-node myserver
> Looks like you haven't used kraftwerk before ... [some init, EC2/Rackspace credentials etc.]
> Waiting for node ...
> Edit /etc/hosts ? [y/n] y
> root SSH access is now ready
# kraftwerk init myproject
> myproject virtualenv is ready
# cd myproject
# source bin/activate
# bin/pip install django
# kraftwerk setup-project --node myserver
> config validates
> site is ready!

http://bitbucket.org/jokull/kraftwerk/src/

Currently it assumes:

You're doing WSGI webapps and your happy to run them under gunicorn behind NGINX
Specific places for static files
You're cool with "the cloud"

Major planned features: data load/dump for all services (SQL, redis, files ...) to and from nodes-project combinations (for backup and transfer). Some sort of CDN integration perhaps.