1 The Problem
We want a file sync tool (a tiny rsync): make a destination folder match a source, but efficiently — only copying files that are new or changed, skipping identical ones. The trick is detecting change with checksums. It teaches efficient change detection, the core of backup and sync tools.
Where this shows up: rsync, Dropbox, backup software, deployment tools, CDN propagation. Copying everything every time is wasteful; detecting and copying only what changed is what makes sync fast and practical.
2 How to Think About It
Think about how to know a file changed, before any code:
The plan — in plain English
1. For each file in the source, compute a checksum — a short fingerprint of its contents. → 2. Compare to the destination’s file: same fingerprint means identical — skip it. → 3. Different or missing means copy it. → Checksums let us detect real content changes without comparing whole files byte by byte.
3 The Build — explained part by part
Here is the complete sync tool. The change-detection logic is testable on its own. Each part is explained below.
Pythonfilesync.py
import hashlib
import shutil
from pathlib import Path
def checksum(path):
# A short fingerprint of a file's contents.
h = hashlib.md5()
h.update(Path(path).read_bytes())
return h.hexdigest()
def needs_copy(source, dest):
# Copy if the destination is missing or different.
if not Path(dest).exists():
return True
return checksum(source) != checksum(dest)
def sync(source_dir, dest_dir):
source = Path(source_dir)
dest = Path(dest_dir)
dest.mkdir(exist_ok=True)
copied = []
for src_file in source.glob("*"):
if src_file.is_file():
dest_file = dest / src_file.name
if needs_copy(src_file, dest_file):
shutil.copy2(src_file, dest_file)
copied.append(src_file.name)
return copied
if __name__ == "__main__":
copied = sync("source", "backup")
print(f"Copied {len(copied)} changed files: {copied}")
What each part does — in plain words
checksum(path) — read the file’s bytes and run them through MD5 to get a short fingerprint. Two files with the same contents have the same checksum; any change produces a different one.needs_copy(source, dest) — the decision: if the destination does not exist, copy. If it does, compare checksums — copy only if they differ. This is the efficiency: identical files are skipped.
source.glob("*") — go through every item in the source folder;
is_file() skips sub-folders.shutil.copy2 — copy the file and its metadata (timestamps). We only reach this for files that actually need copying.
copied list — we record which files we copied, so the tool can report exactly what changed — useful and reassuring.
Common mistakes — and how to avoid them
✗ Copying every file every time — slow and wasteful.
✓ Use checksums to skip unchanged files.
✗ Comparing only file size or name — misses same-size edits.
✓ Compare content checksums to catch every real change.
✗ Comparing whole files byte by byte — works but is slower than a checksum compare.
✓ A checksum is a compact, fast way to compare contents.
4 Test & Prove Each Part
We test the change-detection logic — the decision of whether to copy — which is the part that matters most.
A missing destination file needs copying
Identical files are skipped
Changed files are detected and copied
Pythontest_filesync.py
import hashlib
import os
import tempfile
from pathlib import Path
def checksum(path):
h = hashlib.md5()
h.update(Path(path).read_bytes())
return h.hexdigest()
def needs_copy(source, dest):
if not Path(dest).exists():
return True
return checksum(source) != checksum(dest)
def make_file(content):
path = os.path.join(tempfile.mkdtemp(), "f.txt")
Path(path).write_text(content)
return path
def test_missing_dest_needs_copy():
"""A non-existent destination always needs copying."""
src = make_file("hello")
assert needs_copy(src, "/tmp/does-not-exist-xyz") is True
def test_identical_skipped():
"""Identical files are not copied."""
src = make_file("same content")
dest = make_file("same content")
assert needs_copy(src, dest) is False
def test_changed_detected():
"""A different destination is detected and copied."""
src = make_file("new content")
dest = make_file("old content")
assert needs_copy(src, dest) is True
Run with pytest -v. We test needs_copy with real temp files of known content. The identical-skipped test proves the efficiency works; the changed-detected test proves we never miss a real change.
5 The Interface
INPUTsource + dest folderssync source to dest
Call
sync("source", "backup")OUTPUTcopied filesonly what changed
Returns
Copied 2 changed files: ['new.txt', 'updated.txt']
# identical files skipped6 Run It & Automate It
Run it locally
python3 filesync.pyRun it twice — the second run copies nothing because nothing changed. Change a file and it copies just that one.
Jenkins runs the tests automatically — each line explained below.
What you should see when it works
Terminala real run
Copied 2 changed files: ['report.pdf', 'notes.txt']If it breaks — how to fix it
🚨 It copies everything every run.
Your checksum comparison is wrong, or you skip the exists-check. Verify
needs_copy.🚨 It misses changed files.
Make sure you read the actual bytes for the checksum, not just metadata.
GroovyJenkinsfile
// Jenkinsfile — automated tests on every change.
pipeline {
agent any
stages {
stage('Get the code') { steps { checkout scm } } // download the code
stage('Install') { steps { sh 'python3 -m venv venv && . venv/bin/activate && pip install pytest' } } // test tool
stage('Test') { steps { sh '. venv/bin/activate && pytest -v' } } // run every test
}
post {
success { echo 'All tests passed.' }
failure { echo 'A test failed — look above.' }
}
}
🎯 Try this next — make it yours
- Recursive sync. Handle sub-folders too. (Teaches: walking trees.)
- Delete extras. Remove destination files no longer in source. (Teaches: two-way comparison.)
- Dry run. Show what would change without copying. (Teaches: safe previews.)