To Pickle, Or Not To Pickle? — Quickle!

Pickling in Python can be dangerous.

A pickled gherkin, also known as a deli pickle. Public domain.
A pickled gherkin, a.k.a. a deli pickle. Public domain.

That’s where Quickle comes in — as long as you’re using Python 3.8 or later…

The Python standard library has a package for translating between Python objects and files called pickle. Technically, this process is called serializing or deserializing, depending on the direction. This can be a really handy way to save work in Python.

But, as the documentation for pickle says,

Warning: The pickle module is not secure. Only unpickle data you trust.

It is possible to construct malicious pickle data which will execute arbitrary code during unpickling. Never unpickle data that could have come from an untrusted source, or that could have been tampered with.

Consider signing data with hmac if you need to ensure that it has not been tampered with.

Safer serialization formats such as json may be more appropriate if you are processing untrusted data. See Comparison with json.

Pink warning box from the Python documentation for pickle.

Quickle prevents the possibility of executing arbitrary code upon deserializing, and natively supports a wide range of builtin Python types (unlike msgpack or json). Quickle is also faster than pickle, according to the developer’s website.

It’s easy to install, using either conda or pip:

 # How to install quickle using conda:
conda install -c conda-forge quickle

# How to install quickle using pip:
pip install quickle

Quickle uses dumps to serialize, and loads to deserialize, Python objects, respectively — just like pickle — but it is also possible to create an Encoder (or Decoder) for a more efficient implementation.

The source code for quickle is available from Jim Crist-Harif’s GitHub repo.

Author