Python Module of the Week

Categories:

Recommended

Python provides several modules for storing data. There are basically two aspects to persistence: converting the in-memory object back and forth into a format for saving it, and working with the storage of the converted data.

1.1 Serializing Objects

Python includes two modules capable of converting objects into a transmittable or storable format (serializing): pickle and json. It is most common to use pickle, since there is a fast C implementation and it is integrated with some of the other standard library modules that actually store the serialized data, such as shelve. Web-based applications may want to examine json, however, since it integrates better with some of the existing web service storage applications.

1.2 Storing Serialized Objects

Once the in-memory object is converted to a storable format, the next step is to decide how to store the data. A simple flat-file with serialized objects written one after the other works for data that does not need to be indexed in any way. But Python includes a collection of modules for storing key-value pairs in a simple database using one of the DBM format variants.

The simplest interface to take advantage of the DBM format is provided by shelve. Simply open the shelve file, and access it through a dictionary-like API. Objects saved to the shelve are automatically pickled and saved without any extra work on your part.

One drawback of shelve is that with the default interface you can’t guarantee which DBM format will be used. That won’t matter if your application doesn’t need to share the database files between hosts with different libraries, but if that is needed you can use one of the classes in the module to ensure a specific format is selected (Specific Shelf Types).

If you’re going to be passing a lot of data around via JSON anyway, using json and anydbm can provide another persistence mechanism. Since the DBM database keys and values must be strings, however, the objects won’t be automatically re-created when you access the value in the database.

Category:

Attribution

Doug Hellmann. Python Module of the Week (2.x). https://pymotw.com/2/

VP Flipbook Maker

Convert your work to digital flipbook with VP Online Flipbook Maker! You can also create a new one with the tool. Try it now!