thecodex.expert · The Codex Family of Knowledge
Software Engineering

Testing

Tests aren't overhead — they're the only way to move fast without breaking things. The question isn't whether to test, but what to test and how.

Testing pyramid Unit · Integration · E2E Last verified:
Canonical Definition

Software testing verifies that code behaves as intended under specified conditions. The testing pyramid describes three levels: unit tests (fast, isolated, numerous — test individual functions), integration tests (verify components work together), and end-to-end tests (verify full user flows, slow). TDD (test-driven development) writes failing tests before writing the implementation.

The Testing Pyramid

The testing pyramid, popularised by Mike Cohn and refined by Martin Fowler, guides the proportion of different test types. The principle: write many fast, isolated unit tests; fewer, slower integration tests; and very few full end-to-end tests.

▲ E2E tests (few — slow, expensive, fragile)
▲▲▲ Integration tests (some — medium speed)
▲▲▲▲▲▲▲ Unit tests (many — fast, cheap, isolated)
Unit tests

Test one function or class in complete isolation. Milliseconds each. Run on every save.

Integration tests

Test two or more components together — a service with a real database, or two modules interacting.

End-to-end tests

Test the full stack from the user's perspective — click a button, verify a database row exists.

Writing Good Unit Tests

A good unit test is fast (milliseconds), isolated (no network, no database, no filesystem), deterministic (same result every run), and focused (tests exactly one behaviour). The AAA pattern (Arrange, Act, Assert) gives tests a consistent structure.

Pythontest_pricing.py
import pytest

def apply_discount(price: float, discount_pct: float) -> float:
    """Apply a percentage discount. Returns discounted price."""
    if not 0 <= discount_pct <= 100:
        raise ValueError(f"Discount must be 0-100, got {discount_pct}")
    return round(price * (1 - discount_pct / 100), 2)

class TestApplyDiscount:
    # AAA: Arrange → Act → Assert
    def test_ten_percent_discount(self):
        # Arrange
        price = 100.0
        # Act
        result = apply_discount(price, 10)
        # Assert
        assert result == 90.0

    def test_zero_discount_unchanged(self):
        assert apply_discount(50.0, 0) == 50.0

    def test_full_discount_returns_zero(self):
        assert apply_discount(100.0, 100) == 0.0

    def test_rounding(self):
        assert apply_discount(10.0, 33) == 6.70  # 6.7000... rounded

    # Parametrize: run the same test with multiple inputs
    @pytest.mark.parametrize("discount,expected", [
        (0,   100.00),
        (10,   90.00),
        (25,   75.00),
        (50,   50.00),
        (100,   0.00),
    ])
    def test_various_discounts(self, discount, expected):
        assert apply_discount(100.0, discount) == expected

    # Test the error case
    def test_invalid_discount_raises(self):
        with pytest.raises(ValueError, match="Discount must be 0-100"):
            apply_discount(100.0, 150)

Integration Tests

Integration tests verify that components work correctly together. The key challenge: they're slower and require setup (database, message queue, external service). Two approaches: use a real database in a test container (Docker), or use an in-memory substitute for speed.

Pythontest_user_repository.py
import pytest

# Integration test: real SQLite database, not mocked
# pytest-sqlalchemy or similar sets up/tears down the DB

class TestUserRepository:
    @pytest.fixture(autouse=True)
    def setup_db(self, test_db):
        """Create tables before each test, drop after."""
        test_db.create_tables()
        yield
        test_db.drop_tables()

    def test_save_and_retrieve_user(self, test_db):
        repo = UserRepository(test_db)
        user = User(name="Alice", email="alice@example.com")

        saved = repo.save(user)
        retrieved = repo.find_by_email("alice@example.com")

        assert retrieved is not None
        assert retrieved.name == "Alice"
        assert retrieved.id == saved.id  # ID was assigned

    def test_find_by_email_returns_none_when_missing(self, test_db):
        repo = UserRepository(test_db)
        result = repo.find_by_email("missing@example.com")
        assert result is None

    def test_duplicate_email_raises(self, test_db):
        repo = UserRepository(test_db)
        repo.save(User(name="A", email="dup@example.com"))
        with pytest.raises(DuplicateEmailError):
            repo.save(User(name="B", email="dup@example.com"))

Test-Driven Development (TDD)

TDD inverts the usual order: write a failing test first, then write the minimum code to make it pass, then refactor. The cycle is Red → Green → Refactor.

Benefits: You only write code that's needed (no over-engineering). You get a test suite for free. You're forced to think about the API before implementation. You get immediate feedback that your code works.

Pythontdd_example.py
# TDD cycle: Red → Green → Refactor

# Step 1: RED — write a failing test
def test_fizzbuzz_3():
    assert fizzbuzz(3) == "Fizz"  # NameError: fizzbuzz not defined

# Step 2: GREEN — minimum code to pass
def fizzbuzz(n: int) -> str:
    return "Fizz"  # hardcoded! passes the test above

# Step 3: more tests force better implementation
def test_fizzbuzz_5():
    assert fizzbuzz(5) == "Buzz"

def test_fizzbuzz_15():
    assert fizzbuzz(15) == "FizzBuzz"

def test_fizzbuzz_1():
    assert fizzbuzz(1) == "1"

# Step 4: GREEN + REFACTOR — now write the real implementation
def fizzbuzz(n: int) -> str:
    if n % 15 == 0: return "FizzBuzz"
    if n % 3 == 0:  return "Fizz"
    if n % 5 == 0:  return "Buzz"
    return str(n)

# All 4 tests now pass — and you have full coverage

Mocking and Test Doubles

Unit tests should not hit networks, databases, or the clock. Mocks replace real dependencies with controlled substitutes. The key principle: mock at the boundary of your system, not inside it.

Pythontest_notification_service.py
from unittest.mock import Mock, patch, call

class NotificationService:
    def __init__(self, email_client, sms_client):
        self.email = email_client
        self.sms = sms_client

    def notify_user(self, user, message: str) -> None:
        self.email.send(to=user.email, body=message)
        if user.phone:
            self.sms.send(to=user.phone, body=message[:160])

def test_notifies_by_email_always():
    mock_email = Mock()
    mock_sms = Mock()
    service = NotificationService(mock_email, mock_sms)

    user = Mock(email="alice@example.com", phone=None)
    service.notify_user(user, "Your order shipped!")

    # Assert email was called with the right args
    mock_email.send.assert_called_once_with(
        to="alice@example.com",
        body="Your order shipped!"
    )
    # Assert SMS was NOT called (user has no phone)
    mock_sms.send.assert_not_called()

def test_also_sends_sms_when_phone_present():
    mock_email = Mock()
    mock_sms = Mock()
    service = NotificationService(mock_email, mock_sms)

    user = Mock(email="bob@example.com", phone="+919867800451")
    service.notify_user(user, "Order confirmed!")

    mock_email.send.assert_called_once()
    mock_sms.send.assert_called_once_with(to="+919867800451", body="Order confirmed!")
Testing pitfalls
Testing implementation, not behaviour. Tests that check that a specific internal function was called, or that a private variable has a specific value, break every time you refactor — even when the behaviour is unchanged. Test what the code does, not how it does it.
Flaky tests are worse than no tests. A test that sometimes passes and sometimes fails trains the team to ignore failures. Fix or delete flaky tests immediately. They erode trust in the entire test suite.
100% code coverage doesn't mean tested. Coverage measures which lines were executed, not whether the right things were asserted. You can have 100% coverage with tests that don't check any output. Coverage is a useful floor (90%+ is good), not a ceiling.
Mocking everything. Tests with a mock for every dependency test nothing but the wiring. If your test creates five mocks, consider whether you're testing real behaviour or just the plumbing. Integration tests with real dependencies often provide more confidence.

Sources

1
Beck, K. (2002). Test-Driven Development: By Example. Addison-Wesley. — The original TDD reference.
2
Fowler, M. (2018). "The Practical Test Pyramid." martinfowler.com/articles/practical-test-pyramid.html.
3
Freeman, S., & Pryce, N. (2009). Growing Object-Oriented Software, Guided by Tests. Addison-Wesley. — Integration testing and design for testability.
Source confidence: High Last verified: Primary source: Beck (2002) · Fowler martinfowler.com