Interpretation

Three interpreter architectures

Interpreters exist on a spectrum of implementation complexity, roughly ordered by execution speed:

1. Tree-walking interpreters parse the source into an AST and directly evaluate each node recursively. Simple to implement. Slow for loops (re-evaluates the same nodes). Used in early implementations and educational contexts. Ruby's first implementation (MRI 1.8 and earlier) was tree-walking.

2. Bytecode VMs compile source to a compact instruction set designed for a virtual stack machine or register machine, then execute that bytecode in a dispatch loop. CPython, Ruby's YARV (since 1.9), Lua's VM, and the JVM are bytecode VMs. Faster than tree-walking because the bytecode is already parsed, and dispatch loops are well-optimised by CPU branch predictors.

3. JIT-compiling VMs start as bytecode VMs and selectively compile hot code paths to native machine code during execution. The JVM's HotSpot JIT (Oracle), V8 (Google), SpiderMonkey (Mozilla), and LuaJIT are JIT compilers. Can approach or match ahead-of-time compiled performance for long-running programs.

CPython bytecode in detail

CPython's virtual machine is a stack machine — operations push and pop values from an evaluation stack. The bytecode for x = 1 + 2 is approximately:

LOAD_CONST   1       # push 1 onto the stack
LOAD_CONST   2       # push 2 onto the stack
BINARY_OP    +       # pop two values, add, push result (3)
STORE_NAME   x       # pop result, store as 'x'

You can inspect CPython bytecode directly:

import dis

def add(x, y):
    return x + y

dis.dis(add)
# Output:
#   RESUME          0
#   LOAD_FAST       0 (x)
#   LOAD_FAST       1 (y)
#   BINARY_OP       0 (+)
#   RETURN_VALUE

JIT compilation: V8 and HotSpot

V8 (Google, open source) uses a tiered compilation strategy. Initially, JavaScript functions are interpreted or compiled by a baseline compiler called Sparkplug — fast compilation, modest optimisation. If a function is called frequently (a "hot" function), the optimising compiler Maglev or Turbofan takes over and produces heavily optimised machine code, using type information gathered from previous executions. If the type assumptions turn out to be wrong (a function that was always called with integers is suddenly called with strings), the optimised code is deoptimised — thrown away — and the function falls back to interpreted execution.

The JVM's HotSpot uses a similar strategy: methods start interpreted, and frequently-called methods are JIT-compiled at increasing optimisation levels (C1 — client, fast compile; C2 — server, heavy optimise). HotSpot also performs aggressive speculative optimisations — for example, inlining a virtual method call if only one implementation has ever been seen. If a second implementation is loaded later, the inlined code is deoptimised.

Interpretation vs. compilation: a formal comparison

Threaded code and direct threading

A naive bytecode dispatch loop uses a switch statement over opcode values. Each iteration: read opcode, branch to handler, execute, repeat. The branch predictor typically mispredicts the target (many possible opcodes), causing pipeline stalls. Direct threading replaces the switch with a table of function pointers (or computed gotos in GCC): instead of dispatching on an opcode integer, each bytecode instruction stores the address of its handler directly. At the end of each handler, execution jumps directly to the next handler — no central dispatch loop. CPython 3.11+ uses specialising adaptive interpreter (PEP 659): frequently-executed instructions are replaced in-place with specialised versions optimised for the observed operand types, approaching direct threading in performance.

Inline caching

A key optimisation in dynamic language VMs is inline caching — caching the result of a dynamic lookup (property access, method resolution) at the call site. When obj.method() is executed, the VM must look up method in obj's class hierarchy. Instead of doing this every call, the VM stores the resolved method pointer at the call site and uses it directly on subsequent calls with the same object type. If the type changes, the cache is invalidated. V8 uses polymorphic inline caches (PICs) that can store multiple (type, method) pairs. Hidden classes (V8's internal representation of object shapes) enable property access close to compiled-language speeds.

Garbage collection and the GIL

CPython uses reference counting as its primary garbage collection mechanism — each object maintains a count of references to it; when the count reaches zero, the object is immediately freed. A cyclic garbage collector runs periodically to collect reference cycles (objects that reference each other, keeping both counts above zero despite being unreachable).

CPython's Global Interpreter Lock (GIL) is a mutex that protects CPython's internal state. Only one thread may execute Python bytecode at a time. This simplifies memory management (reference count updates are atomic because only one thread runs) but prevents true parallelism on multi-core hardware for CPU-bound Python threads. The GIL does not affect I/O-bound concurrency (threads release the GIL while waiting for I/O). Python 3.13 introduced an experimental free-threaded build (PEP 703) that removes the GIL at the cost of additional per-object locking.