1 The Problem
We want a load balancer: spread incoming requests across a pool of backend servers (so no one server is overwhelmed), using round-robin, and skip servers that are unhealthy. It teaches distributing work and health checking — the core of how services scale to handle huge traffic.
Where this shows up: nginx, HAProxy, AWS ELB, Kubernetes services, any system serving traffic at scale. A load balancer is what sits in front of a fleet of servers and makes them act as one reliable service.
2 How to Think About It
Think about fair distribution with health, before any code:
The plan — in plain English
1. A pool of backend servers, each marked healthy or not. → 2. Round-robin: hand each new request to the next server in turn, cycling around. → 3. Skip unhealthy ones: if the next server is down, move to the following healthy one. → 4. Health checks mark servers up or down. This spreads load evenly and routes around failures.
3 The Build — explained part by part
Here is a complete load balancer with health checks. Each part is explained below.
Pythonbalancer.py
class LoadBalancer:
def __init__(self, servers):
self.servers = servers # list of server names
self.healthy = {s: True for s in servers}
self.current = 0
def mark_unhealthy(self, server):
self.healthy[server] = False
def mark_healthy(self, server):
self.healthy[server] = True
def next_server(self):
# Round-robin, skipping unhealthy servers.
for _ in range(len(self.servers)):
server = self.servers[self.current]
self.current = (self.current + 1) % len(self.servers)
if self.healthy[server]:
return server
return None # all servers are down
if __name__ == "__main__":
lb = LoadBalancer(["server1", "server2", "server3"])
print([lb.next_server() for _ in range(4)]) # rotates
lb.mark_unhealthy("server2")
print([lb.next_server() for _ in range(4)]) # skips server2
What each part does — in plain words
self.healthy = {s: True ...} — track each server’s health. All start healthy; checks can flip them.self.current — remembers whose turn is next, the heart of round-robin.
self.current = (self.current + 1) % len(...) — advance to the next server, wrapping around to 0 at the end. The modulo makes it a loop — an elegant way to cycle.
the for loop in next_server — try up to N servers (N = pool size). For each, advance the pointer and, if that server is healthy, return it. This skips unhealthy servers while still moving the round-robin forward.
return None — if we check every server and none is healthy, there is nowhere to send the request. A real balancer would return an error to the client.
mark_healthy / mark_unhealthy — health checks call these. Routing around a failed server is what keeps the overall service up even when individual servers die.
Common mistakes — and how to avoid them
✗ Not advancing the pointer when skipping — you get stuck on a healthy server forever.
✓ Always advance
current, then check health.✗ Looping forever when all servers are down — no exit condition.
✓ Limit the search to N tries (the pool size) and return None.
✗ Forgetting modulo — the pointer runs off the end of the list.
✓ Wrap with
% len(servers) to cycle.4 Test & Prove Each Part
We test round-robin distribution and that unhealthy servers are skipped.
Requests rotate through servers in order
Unhealthy servers are skipped
If all servers are down, None is returned
Pythontest_balancer.py
class LoadBalancer:
def __init__(self, servers):
self.servers = servers
self.healthy = {s: True for s in servers}
self.current = 0
def mark_unhealthy(self, server):
self.healthy[server] = False
def next_server(self):
for _ in range(len(self.servers)):
server = self.servers[self.current]
self.current = (self.current + 1) % len(self.servers)
if self.healthy[server]:
return server
return None
def test_round_robin():
"""Requests rotate through all servers."""
lb = LoadBalancer(["a", "b", "c"])
assert [lb.next_server() for _ in range(3)] == ["a", "b", "c"]
def test_skips_unhealthy():
"""An unhealthy server is skipped."""
lb = LoadBalancer(["a", "b", "c"])
lb.mark_unhealthy("b")
picks = [lb.next_server() for _ in range(4)]
assert "b" not in picks
def test_all_down_returns_none():
"""If every server is down, return None."""
lb = LoadBalancer(["a", "b"])
lb.mark_unhealthy("a")
lb.mark_unhealthy("b")
assert lb.next_server() is None
Run with pytest -v. The skip-unhealthy test proves the balancer routes around failures — the property that keeps a service alive when a server dies. The all-down test confirms it fails gracefully rather than looping forever.
5 The Interface
nextlb.next_server()pick a healthy server
Round-robin
server1, server2, server3, server1, ...healthmark_unhealthy(server)route around failures
After marking server2 down
server1, server3, server1, server3, ...6 Run It & Automate It
Run it locally
python3 balancer.pyWatch requests rotate, then mark a server unhealthy and see it skipped.
Jenkins runs the tests automatically — each line explained below.
What you should see when it works
Terminala real run
['server1', 'server2', 'server3', 'server1']
['server1', 'server3', 'server1', 'server3']If it breaks — how to fix it
🚨 It keeps returning the same server.
You are not advancing
current. Advance it every iteration.🚨 It hangs with all servers down.
Your loop has no bound. Try at most
len(servers) times, then return None.GroovyJenkinsfile
// Jenkinsfile — automated tests on every change.
pipeline {
agent any
stages {
stage('Get the code') { steps { checkout scm } } // download the code
stage('Install') { steps { sh 'python3 -m venv venv && . venv/bin/activate && pip install pytest' } } // test tool
stage('Test') { steps { sh '. venv/bin/activate && pytest -v' } } // run every test
}
post {
success { echo 'All tests passed.' }
failure { echo 'A test failed — look above.' }
}
}
🎯 Try this next — make it yours
- Weighted balancing. Send more traffic to bigger servers. (Teaches: weighted round-robin.)
- Least connections. Pick the server with the fewest active requests. (Teaches: another strategy.)
- Auto health checks. Periodically ping servers to update health. (Teaches: background monitoring.)
What you learned
You built a load balancer with round-robin distribution and health checks — the component that lets a fleet of servers act as one reliable, scalable service. Spreading load and routing around failures is core to every large system. Related: Object-Oriented Python, socket.