Undefined behaviour (UB) in C is any operation whose result the C standard explicitly does not define. The compiler is permitted to assume UB never occurs, enabling optimisations that produce wrong results or security vulnerabilities in code that triggers it. Common UB: buffer overflows, use-after-free, signed integer overflow, reading uninitialised variables, and null pointer dereference.
In most languages, accessing an out-of-bounds array throws an exception or crashes predictably. In C, it's undefined behaviour — the compiler can do anything: the wrong value silently, a crash sometime later, or nothing at all right now but corruption that surfaces hours later as a different crash. This is why 70% of Microsoft and Google security CVEs come from memory safety issues in C and C++ code.
Common undefined behaviour: examples
#include <stdio.h>
#include <stdlib.h>
int main(void) {
/* UB 1: Buffer overflow — writing past array bounds */
int arr[5];
arr[5] = 99; /* UB: index 5 doesn't exist (valid: 0–4) */
arr[-1] = 0; /* UB: negative index */
/* UB 2: Use-after-free */
int *p = malloc(sizeof(int));
free(p);
*p = 42; /* UB: p points to freed memory */
/* UB 3: Null pointer dereference */
int *null_p = NULL;
/* *null_p = 5; */ /* UB: crashes on most systems, but not defined */
/* UB 4: Signed integer overflow */
int x = 2147483647; /* INT_MAX */
int y = x + 1; /* UB: signed overflow is not defined in C */
/* The compiler may assume this can't happen and optimise aggressively */
/* NOTE: unsigned overflow IS defined — it wraps (mod 2^n) */
/* UB 5: Reading uninitialised variable */
int uninit;
/* printf("%d\n", uninit); */ /* UB: indeterminate value */
/* UB 6: Dangling pointer after function return */
/* int* bad = bad_function(); */ /* returns address of local — UB to use */
return 0;
}Detection tools: compile with warnings, run with sanitisers
# Compiler warnings — always use these:
gcc -Wall -Wextra -Wpedantic -std=c17 program.c -o program
# AddressSanitizer (ASan): detects buffer overflows, use-after-free, memory leaks
gcc -fsanitize=address -g program.c -o program_asan
./program_asan # reports exact location of memory errors at runtime
# UndefinedBehaviorSanitizer (UBSan): detects signed overflow, null deref, etc.
gcc -fsanitize=undefined -g program.c -o program_ubsan
./program_ubsan
# Both together (recommended for development):
gcc -fsanitize=address,undefined -g -O1 program.c -o program_san
# Valgrind: slower but very thorough memory analysis (heap only)
valgrind --leak-check=full --error-exitcode=1 ./programWhy the compiler assumes UB never happens
The C standard gives compilers a key guarantee: programs with undefined behaviour have no required behaviour. This is not an accidental loophole — it's intentional, enabling critical optimisations. If the compiler sees if (p != NULL) *p = x;, it can prove that p is non-null after the branch and eliminate the check. If the compiler sees signed arithmetic i + 1 > i, it can assume this is always true (because signed overflow is UB, so the compiler assumes it never wraps). This enables loop optimisations that would otherwise require per-iteration overflow checks. The problem: code that "works" with optimisations off can silently break with -O2 because UB-based optimisations take effect.
/* Classic example: the compiler may remove this null check due to UB reasoning */
void process(int *p) {
/* If the compiler sees *p = 42 later unconditionally,
it can infer p is non-null (null deref would be UB)
and thus remove this check entirely at -O2 */
if (p == NULL) return;
*p = 42;
}
/* Signed overflow UB optimisation: this loop may become infinite */
/* with -O2 because the compiler assumes i+1 > i is always true */
/* (signed overflow wrapping around would be UB) */
void loop_example(int n) {
for (int i = 0; i <= n; i++) {
/* if n == INT_MAX, i+1 overflows — UB */
/* optimiser may emit: for (int i = 0; true; i++) */
}
}
/* Safe: use unsigned or explicit bounds check */
void loop_safe(unsigned int n) {
for (unsigned int i = 0; i <= n; i++) {
/* unsigned overflow is defined (wraps mod 2^32) */
}
}UINT_MAX + 1 == 0 — unsigned arithmetic wraps modulo 2^n. This is defined by the C standard. INT_MAX + 1 — signed arithmetic overflow is undefined behaviour. The compiler assumes it won't happen. This distinction matters enormously for security code and loop termination.-O flag can change the observable behaviour. The only guarantee is that code without UB works correctly across all conforming implementations.-Wall -Wextra catches many issues statically (at compile time). AddressSanitizer and UBSan detect issues at runtime — they can't find bugs in code paths that aren't exercised. Both are complementary. Use static analysis (clang-tidy, Coverity, cppcheck) for code-path-independent analysis.The nasal demons problem: UB is not "implementation-defined"
The C standard distinguishes three categories of behaviour. Defined behaviour: the standard specifies exactly what happens. Implementation-defined behaviour: the result is left to the implementation but must be documented and consistent (e.g. the size of int). Undefined behaviour: the standard imposes no requirements — colloquially, "nasal demons" (from a famous comp.std.c post by John F. Woods: "it is legal for such a program to make demons fly out of your nose"). UB is not the same as "implementation-defined" — implementations are not required to define anything for UB. The compiler is free to assume UB never occurs and optimise accordingly, producing results that are neither the "wrapping" behaviour nor a crash — but some third, non-obvious outcome.
Sanitiser implementation: shadow memory and compile-time instrumentation
AddressSanitizer (ASan) works by maintaining a shadow memory region — for every 8 bytes of application memory, 1 byte of shadow memory tracks whether the application bytes are addressable and initialised. Every memory access in the instrumented program is preceded by a shadow memory check. Heap metadata (redzones) before and after each allocation and in freed memory enable detection of overflows and use-after-free. The slowdown is typically 2–5x (vs 10–30x for valgrind, which uses a different binary translation approach). UBSan instruments arithmetic operations and pointer dereferences with lightweight checks that call a runtime handler when triggered — typically 10–20% overhead.
ISO/IEC 9899:2018 (C17). Annex J.2: Undefined behaviour (lists all 200+ UB forms). Lattner, C. (2011). "What Every C Programmer Should Know About Undefined Behavior." blog.llvm.org. Seacord, R. C. (2013). Secure Coding in C and C++. Addison-Wesley. NSA/CISA (2022). "Software Memory Safety." — Recommends memory-safe languages over C/C++ for new systems code.