1 The Problem
We have used pytest in every project — now we build one. Our framework discovers functions named test_*, runs each, catches any assertion failure without stopping the others, and reports passes and failures. It teaches introspection (code examining itself) and robust error handling — and demystifies the test runner you rely on.
Where this shows up: pytest, unittest, every test runner, plugin systems, task runners, anything that discovers and runs functions dynamically. Understanding how a framework finds and runs your code is deeply clarifying.
2 How to Think About It
Think about discover, run, report, before any code:
The plan — in plain English
1. Discover: look through a module for functions whose names start with test_. → 2. Run each in a try/except, so one failure does not stop the rest. → 3. A passing test simply returns; a failing one raises AssertionError. → 4. Report: count and list passes and failures. That is the whole framework.
3 The Build — explained part by part
Here is a complete testing framework. Each part is explained below.
Pythonmytest.py
import traceback
def run_tests(module):
# 1. DISCOVER: find every function named test_* in the module.
tests = [(name, obj) for name, obj in vars(module).items()
if name.startswith("test_") and callable(obj)]
passed, failed = 0, 0
failures = []
# 2. RUN each test, catching failures so others still run.
for name, test_func in tests:
try:
test_func()
passed += 1
print(f"PASS: {name}")
except AssertionError as e:
failed += 1
failures.append((name, str(e) or "assertion failed"))
print(f"FAIL: {name}")
except Exception as e:
failed += 1
failures.append((name, f"error: {e}"))
print(f"ERROR: {name}")
# 3. REPORT.
print(f"\n{passed} passed, {failed} failed")
return {"passed": passed, "failed": failed, "failures": failures}
# Example tests to run.
def test_addition():
assert 2 + 2 == 4
def test_string():
assert "ab".upper() == "AB"
def test_will_fail():
assert 1 == 2, "one is not two"
if __name__ == "__main__":
import sys
run_tests(sys.modules[__name__])
What each part does — in plain words
vars(module).items() — introspection: vars gives every name defined in the module and the object it points to. We are asking the module to describe itself.name.startswith("test_") and callable(obj) — the discovery rule: keep names beginning with
test_ that are functions. This is the exact convention pytest uses — and now you see it is just a name check.try: test_func() except AssertionError — run the test. If its
assert fails, Python raises AssertionError, which we catch — recording a failure but continuing to the next test. One failure never stops the suite.except Exception — a second catch for unexpected errors (not just failed asserts), so a buggy test is reported as an error rather than crashing the runner.
the report — tally passes and failures and print a summary. That count is exactly what pytest shows you at the end of a run.
Common mistakes — and how to avoid them
✗ Letting the first failure crash the runner — then later tests never run.
✓ Wrap each test in try/except so failures are isolated.
✗ Only catching AssertionError — a test with a real bug crashes instead of reporting.
✓ Also catch general exceptions and report them as errors.
✗ Running functions that are not tests — helpers get called by accident.
✓ Filter to names starting with
test_.4 Test & Prove Each Part
We test the framework itself — by giving it sample test functions and checking it reports the right pass/fail counts. (A test framework, tested.)
Passing tests are counted as passed
Failing tests are counted as failed
One failure does not stop the others running
Pythontest_mytest.py
import types
def run_tests(module):
tests = [(name, obj) for name, obj in vars(module).items()
if name.startswith("test_") and callable(obj)]
passed, failed = 0, 0
for name, test_func in tests:
try:
test_func()
passed += 1
except Exception:
failed += 1
return {"passed": passed, "failed": failed}
def make_module(**funcs):
"""Build a fake module holding the given test functions."""
mod = types.ModuleType("fake")
for name, func in funcs.items():
setattr(mod, name, func)
return mod
def test_counts_passes():
"""Two passing tests give 2 passed."""
mod = make_module(test_a=lambda: None, test_b=lambda: None)
assert run_tests(mod)["passed"] == 2
def test_counts_failures():
"""A failing test is counted as failed."""
def failing():
assert False
mod = make_module(test_x=failing)
assert run_tests(mod)["failed"] == 1
def test_continues_after_failure():
"""A failure does not stop a later passing test."""
def failing():
assert False
mod = make_module(test_1=failing, test_2=lambda: None)
result = run_tests(mod)
assert result["passed"] == 1
assert result["failed"] == 1
Run with pytest -v (using real pytest to test our pytest!). We build fake modules holding sample tests and confirm the counts. The continues-after-failure test proves the most important property: isolation between tests.
5 The Interface
runrun_tests(module)discover and run all test_*
Output
PASS: test_addition
PASS: test_string
FAIL: test_will_fail
2 passed, 1 failed6 Run It & Automate It
Run it locally
python3 mytest.pyWatch it discover and run the sample tests, reporting passes and failures — just like pytest.
Jenkins runs the tests automatically — each line explained below.
What you should see when it works
Terminala real run
PASS: test_addition
PASS: test_string
FAIL: test_will_fail
2 passed, 1 failedIf it breaks — how to fix it
🚨 The runner stops at the first failure.
Your try/except is around the whole loop, not each test. Put it inside the loop.
🚨 Non-test functions get run.
Tighten the filter to
name.startswith("test_") and callable.GroovyJenkinsfile
// Jenkinsfile — automated tests on every change.
pipeline {
agent any
stages {
stage('Get the code') { steps { checkout scm } } // download the code
stage('Install') { steps { sh 'python3 -m venv venv && . venv/bin/activate && pip install pytest' } } // test tool
stage('Test') { steps { sh '. venv/bin/activate && pytest -v' } } // run every test
}
post {
success { echo 'All tests passed.' }
failure { echo 'A test failed — look above.' }
}
}
🎯 Try this next — make it yours
- setup/teardown. Run code before and after each test. (Teaches: fixtures.)
- assertRaises. Add a helper to assert a function raises. (Teaches: testing errors.)
- Discover files. Find tests across multiple files. (Teaches: module loading.)
What you learned
You built a testing framework using introspection (vars to discover tests) and robust error handling (catching failures so tests stay isolated). pytest is no longer magic — it discovers test_* functions and runs them in try/except, exactly as you did. Related: unittest, Error Handling.