Have you ever wondered how hard it would be to tack a RESTful HTTP interface on top of a fast key/value database like TokyoCabinet?
Probably not, but I did: tc-rest.tar.gz
Components:
- TokyoCabinet – my favorite persistent key/value database
- pytc – a wonderful Python wrapper for TC
- fapws3 – a fast libev based HTTP/WSGI server
- simpleson – (or Python >= 2.6) for encapsulating HTTP responses
- okapi – a fantastic little static HTML page for testing HTTP APIs
Getting TokyoCabinet+pytc to work inside a virtualenv was a bit tricky, so check out my run.sh script if you’re having trouble getting it to start.
Once you get it started, load okapi in your browser:
http://localhost:8080/static/okapi.html
And then create a database by doing a POST like:
http://localhost:8080/foo/
And finally store/get keys and values using GET and POST requests like:
http://localhost:8080/foo/bar/
http://localhost:8080/foo/baz/
Doing a GET request to a database URL lists keys.
At any rate, I had big dreams for building a system where you would store JSON, specify indexes on certain keys, and the server would maintain those indexes for you by creating ad hoc TokyoCabinet databases.
Instead I ended up wasting most of my time learning how to write a low-level WSGI app. I should have just used CherryPy or Django from the beginning, but I had never written a pure WSGI app before. It was a good lesson even if it meant not getting some of my features implemented.
I’ll probably keep playing with this idea, but the next version will probably be based on some existing framework. Parsing environ['PATH_INFO'] and running start_response(...) manually gets old fast.
fapws3 is pretty neat, but had lots of annoying rough edges. I had to manually create a README file because its setup.py expects one to exist. Then I had to manually allow DELETE HTTP methods in fapws/base.py, otherwise it would return an HTML error message for me! That was a bit shocking since I was working under the assumption fapws3 is just a low-level HTTP/WSGI server.
Update
- If you’re new to TokyoCabinet, I posted my presentation on it that I did at Portland Python meetup.
- Someone want to benchmark this for me? Might be interesting since its made with the fastest libs available in Python for their respective tasks. I’m just feeling lazy at this point.
