The pickle module implements binary protocols for serializing and de-serializing a Python object structure. "Pickling" converts an object hierarchy into a byte stream; "unpickling" reconstructs the object from the byte stream. Unlike JSON, pickle handles almost any Python type but is Python-specific and unsafe with untrusted data.
Freezing Python objects to disk
What you will learn: how pickle serializes almost any Python object to bytes and back, preserving types JSON cannot, and how to save to files.
How to read this tab: Always open pickle files in binary mode (wb/rb). pickle keeps tuples, sets, and datetimes exactly.
pickle saves a Python object — a list, a dict, a custom class instance, almost anything — to a file or byte stream, so you can load it back exactly as it was later. Think of it as freezing a Python object to disk and thawing it on demand.
pickle preserves Python types JSON loses. Tuples stay tuples (JSON makes them lists), sets stay sets, datetimes stay datetimes, and custom class instances round-trip exactly. The trade-off: pickle is Python-only, while JSON is universal across languages.
Pickling and unpickling
import pickle
data = {
"name": "Priya",
"scores": [85, 92, 78],
"active": True,
"nested": {"a": (1, 2, 3)},
}
# dumps -> bytes (the "s" means string/bytes, like json)
pickled = pickle.dumps(data)
print(type(pickled)) # <class 'bytes'>
# loads -> reconstruct the object
restored = pickle.loads(pickled)
print(restored == data) # True — identical structure
print(restored["nested"]["a"]) # (1, 2, 3) — even the tuple is preserved
# pickle handles types JSON cannot:
import datetime
obj = {
"when": datetime.datetime.now(), # datetime — JSON can't do this
"coords": (1, 2), # tuple stays a tuple (JSON makes lists)
"unique": {1, 2, 3}, # set — JSON can't do this
}
print(pickle.loads(pickle.dumps(obj))) # all types preserved exactlyUnlike JSON, pickle keeps tuples as tuples (not lists), sets as sets, datetimes as datetimes, and can even serialize custom class instances. The trade-off: pickle files are readable only by Python, while JSON is universal across languages.
Always use binary mode for pickle. Open with wb to write and rb to read — never w/r. Pickle data is binary; text mode will corrupt it. The conventional extension is .pkl, though it is not required.
Saving to and loading from files
import pickle
cache = {"results": [1, 2, 3], "computed_at": "2026-06-17"}
# dump -> write to a file (binary mode is required)
with open("cache.pkl", "wb") as f: # 'wb' — write binary
pickle.dump(cache, f)
# load -> read from a file
with open("cache.pkl", "rb") as f: # 'rb' — read binary
loaded = pickle.load(f)
print(loaded) # {'results': [1, 2, 3], 'computed_at': '2026-06-17'}
# Pickle a custom class instance
class Model:
def __init__(self, weights):
self.weights = weights
m = Model([0.1, 0.5, 0.9])
with open("model.pkl", "wb") as f:
pickle.dump(m, f)
# Later — the class definition must be importable to unpickle
with open("model.pkl", "rb") as f:
restored = pickle.load(f)
print(restored.weights) # [0.1, 0.5, 0.9]Pickle data is binary — open files with "wb" and "rb", never "w"/"r". The conventional extension is .pkl or .pickle, though the extension is not required.
✅ Beginner tab complete
- I can pickle.dumps an object to bytes and loads it back
- I know pickle preserves tuples, sets, datetimes, custom classes
- I use binary mode (wb/rb) for pickle files
- I know pickle files are readable only by Python
The security risk and pickle vs JSON
What you will learn: why unpickling untrusted data is dangerous, how to sign pickles if needed, and when to choose pickle vs JSON.
How to read this tab: This is the most important section: NEVER unpickle data from an untrusted source. It can execute arbitrary code.
NEVER unpickle data from an untrusted source. Unpickling can execute arbitrary code the moment you load it — this is inherent to how pickle reconstructs objects, not a bug. Pickle is safe only for data your own program created and stored where only it can write. For anything from the outside world, use JSON.
The security risk — read this carefully
Unpickling data executes code embedded in the pickle. A malicious pickle can run arbitrary commands the moment you load it. Never unpickle data from an untrusted source.
import pickle
# WHY pickle is dangerous: unpickling can execute arbitrary code.
# A crafted pickle can run any command when loaded — no warning.
# This is not a bug; it is inherent to how pickle reconstructs objects
# (it can call __reduce__ which can invoke any callable).
# THE RULE:
# - NEVER pickle.load() data received over a network, from a user
# upload, from a URL, or any source you do not fully control.
# - Pickle is safe ONLY for data your own program created and stored
# somewhere only your program can write.
# If you must exchange data with the outside world, use JSON:
import json
safe = json.loads(untrusted_string) # JSON cannot execute code
# If you need pickle's power but worry about tampering, sign the data:
import hmac, hashlib, secrets
key = secrets.token_bytes(32)
def safe_dump(obj):
payload = pickle.dumps(obj)
sig = hmac.new(key, payload, hashlib.sha256).digest()
return sig + payload
def safe_load(blob):
sig, payload = blob[:32], blob[32:]
expected = hmac.new(key, payload, hashlib.sha256).digest()
if not hmac.compare_digest(sig, expected):
raise ValueError("Tampered or forged pickle — refusing to load")
return pickle.loads(payload) # only after verifying the signaturepickle for internal storage, JSON for exchange. Use pickle for your own program caching and state where you control both ends. Use JSON for APIs, config files, and anything crossing a trust boundary or read by other languages — it is safe and universal.
pickle vs JSON — choosing
| pickle | JSON | |
|---|---|---|
| Readable by | Python only | Any language |
| Types | Almost any Python object | dict, list, str, num, bool, null |
| Human-readable | No (binary) | Yes (text) |
| Safe with untrusted data | No — never | Yes |
| Tuples / sets / datetime | Preserved | Lost / unsupported |
| Use for | Internal caching, app state | APIs, config, data exchange |
✅ Intermediate tab complete
- I NEVER unpickle data from an untrusted source
- I know unpickling can execute arbitrary code
- I use JSON for data crossing a trust boundary or other languages
- I can sign a pickle with hmac if I must verify integrity
Protocols, __getstate__, and alternatives
What you will learn: pickle protocols 0-5, controlling serialization with __getstate__/__setstate__/__reduce__, and ecosystem alternatives.
How to read this tab: Read when pickling objects with live resources (sockets, files) that need custom save/restore logic.
Protocols, __reduce__, and alternatives
pickle supports several wire protocols, numbered 0 through 5. Higher protocols are more efficient and support more types; protocol 5 (PEP 574, Python 3.8+) adds out-of-band data for zero-copy serialization of large buffers. pickle.DEFAULT_PROTOCOL is the safe default, while pickle.HIGHEST_PROTOCOL gives the most efficient format your Python version supports.
import pickle
print(pickle.HIGHEST_PROTOCOL) # 5 on modern Python
print(pickle.DEFAULT_PROTOCOL) # 5 (3.8+)
# Choose a protocol explicitly (e.g. for cross-version compatibility)
data = {"x": 1}
blob = pickle.dumps(data, protocol=4) # protocol 4 for older readers
# __reduce__ / __getstate__ / __setstate__ control pickling of a class
class Connection:
def __init__(self, host):
self.host = host
self.socket = "live-socket-object" # can't be pickled
def __getstate__(self):
# Return what to pickle — exclude the unpicklable socket
state = self.__dict__.copy()
del state["socket"]
return state
def __setstate__(self, state):
# Restore, then re-establish the excluded resource
self.__dict__.update(state)
self.socket = None # reconnect on demand
c = pickle.loads(pickle.dumps(Connection("db.example.com")))
print(c.host, c.socket) # db.example.com None
# What cannot be pickled: open files, sockets, lambdas, generators,
# locks, and database connections — anything tied to runtime state.
# pickle.dumps(lambda x: x) # PicklingError
# Alternatives in the ecosystem:
# - json: cross-language, safe, text
# - marshal: internal CPython use only, NOT stable across versions
# - dill: third-party, extends pickle (lambdas, more types)
# - joblib: efficient for large NumPy arrays (ML model persistence)For controlling serialization of complex objects, the pickling protocol offers a hierarchy of hooks: __getstate__ and __setstate__ customize what state is saved and how it is restored (the common case, ideal for excluding live resources like sockets or file handles), while __reduce__ gives complete low-level control over reconstruction. The recommended practice for any object holding runtime-bound resources is to implement __getstate__ to exclude them and __setstate__ to re-establish or defer them. For the specific and common case of persisting machine-learning models with large NumPy arrays, joblib is more efficient than raw pickle; for cross-language or untrusted contexts, JSON remains the correct and safe choice.
✅ Expert tab complete
- I know pickle has protocols 0-5; higher is more efficient
- I can use __getstate__/__setstate__ to exclude unpicklable resources
- I know open files, sockets, lambdas, and generators cannot be pickled
- I know joblib for large arrays and dill for extended types