MemoryMapFile Convenience Class for Python

My obsession with mmap hasn’t died, but while Python’s mmap module is a wonderful low level library it’s a bit hard for a newcomer to use properly.

I’ve started toying with a convenience wrapper class for mmap.mmap (at least the Unix version):

My original goal was to automatically grow the mmap whenever the user attempts to write beyond the current size of the mmap file, but that’s going to take carefully wrapping quite a few methods (write, __setitem__, and maybe get/read methods too).

If it becomes useful, I may use it in mmstats.

Feedback welcome!

Update: Discovered the hard way (segfaults) that resizing mmaps is tricky: the region can be moved but data will be copied. However, any existing pointers (from ctypes..from_buffer in my case) will now point to freed memory and segfault upon use.

tl;dr – If at all possible, precompute the size of your mmap before using it.

This entry was posted in Open Source, Python, Technology and tagged . Bookmark the permalink.