Python

The data model and dunder methods

Everything in Python is an object — including integers, functions, and classes. Python's behaviour for built-in operations is defined by dunder methods (double-underscore): __add__ for +, __len__ for len(), __iter__/__next__ for iteration, __enter__/__exit__ for context managers (with statement). Implementing these protocols lets user-defined classes integrate seamlessly with built-in syntax.

class Vector:
    def __init__(self, x, y): self.x, self.y = x, y
    def __add__(self, other): return Vector(self.x+other.x, self.y+other.y)
    def __repr__(self): return f"Vector({self.x}, {self.y})"
    def __len__(self): return 2
    def __eq__(self, other): return self.x==other.x and self.y==other.y

v1 = Vector(1, 2)
v2 = Vector(3, 4)
print(v1 + v2)    # Vector(4, 6) — uses __add__
print(len(v1))    # 2 — uses __len__
print(v1 == v1)   # True — uses __eq__

Generators and lazy evaluation

A generator is a function that uses yield instead of return. Calling a generator function returns a generator object — an iterator that produces values lazily, one at a time. Generators are memory-efficient for large sequences: a generator for all integers uses O(1) memory, where a list would use O(n).

def fibonacci():
    a, b = 0, 1
    while True:          # infinite sequence
        yield a
        a, b = b, a + b  # never stores the sequence

gen = fibonacci()
for _ in range(10):
    print(next(gen), end=" ")  # 0 1 1 2 3 5 8 13 21 34

# Generator expressions — like list comprehensions but lazy
big_squares = (x**2 for x in range(10_000_000))  # O(1) memory
first = next(big_squares)  # 0 — computed on demand

Decorators

A decorator is a function that takes a function as input and returns a modified function. The @decorator syntax is sugar for func = decorator(func). Decorators enable cross-cutting concerns — caching, timing, authentication, validation — without modifying the underlying function.

import functools, time

def timer(func):
    @functools.wraps(func)      # preserve function metadata
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        elapsed = time.perf_counter() - start
        print(f"{func.__name__}: {elapsed:.4f}s")
        return result
    return wrapper

@timer
def slow_sum(n):
    return sum(range(n))

slow_sum(10_000_000)   # slow_sum: 0.1234s

Context managers and the with statement

A context manager implements __enter__ and __exit__. The with statement calls __enter__ on entry and __exit__ on exit — even if an exception occurs. This guarantees cleanup. File handles, database connections, locks, and network sockets are all context managers in Python's standard library.

# File — automatically closed on exit
with open("data.csv", "r") as f:
    content = f.read()
# f is closed here, even if an exception occurred

# contextlib.contextmanager: decorator-based context manager
from contextlib import contextmanager

@contextmanager
def timer(label):
    import time
    start = time.perf_counter()
    yield
    print(f"{label}: {time.perf_counter()-start:.4f}s")

with timer("my block"):
    result = sum(range(10_000_000))

Standard library highlights

Virtual environments and packaging

venv: python -m venv .venv creates an isolated Python environment with its own packages, preventing version conflicts between projects. source .venv/bin/activate (Unix) or .venv\Scripts\activate (Windows) activates it. Always use a venv per project.

pip: pip install requests installs from PyPI. pip freeze > requirements.txt records dependencies. uv (2024): a Rust-based Python package manager 10–100× faster than pip, rapidly becoming the standard. Poetry and PDM: dependency management + build tools.

CPython internals: the reference implementation

CPython (github.com/python/cpython) is the reference implementation of Python. Python is formally specified by the CPython implementation and the Python Language Reference (docs.python.org/3/reference/). The compilation pipeline: source code → tokeniser → parser → AST → compile to bytecode (.pyc in __pycache__) → CPython virtual machine (a stack-based VM) executes bytecode. The bytecode format is documented in the dis module: import dis; dis.dis(function) shows the bytecode instructions.

import dis

def add(x, y):
    return x + y

dis.dis(add)
# Bytecode output (Python 3.13):
#   LOAD_FAST    0 (x)
#   LOAD_FAST    1 (y)
#   BINARY_OP    0 (+)
#   RETURN_VALUE

Memory management: reference counting + cyclic GC

CPython uses reference counting as its primary memory management strategy. Every Python object has a reference count. When the count reaches 0, the object is freed immediately. Reference counting is deterministic and low-latency — unlike tracing GC, it doesn't require stop-the-world pauses. However, reference counting cannot handle reference cycles (object A points to B, B points to A). Python's cyclic garbage collector (gc module) periodically detects and collects cycles — it runs by default in three generations (gen 0, 1, 2) with collection frequencies configurable via gc.set_threshold().

PEP process and language governance

Python evolves through the PEP (Python Enhancement Proposal) process. Any Python developer can write a PEP; significant ones are reviewed by the Steering Council (a five-person elected body established in PEP 13 following Guido van Rossum's resignation as BDFL in July 2018). Key PEPs: PEP 8 (style guide), PEP 20 (Zen of Python), PEP 484 (type hints), PEP 572 (walrus operator), PEP 634 (structural pattern matching), PEP 703 (no-GIL / free-threaded CPython, accepted 2023, experimental in 3.13).

Performance: CPython vs. PyPy vs. Cython

PyPy: a JIT-compiled Python implementation. Typically 4–10× faster than CPython on pure Python code. Excellent for compute-heavy Python without C extensions. Cython: a superset of Python that compiles to C. Adding type annotations to a Cython file can yield 50–100× speedups for numerical code. Used internally by pandas, scikit-learn, and scipy. Numba: a JIT compiler for NumPy-heavy code; @numba.jit can achieve near-C performance for numerical loops without rewriting in C.