The shutil module offers high-level operations on files and collections of files, especially functions that copy and remove entire directory trees, move files, and create or extract archives. It complements os and pathlib, which handle individual path and file operations.
Whole-file and whole-folder operations
What you will learn: copying files (copy/copy2), copying entire directory trees, and the metadata differences between the copy functions.
How to read this tab: shutil is the "do it to the whole folder" module — the high-level counterpart to os and pathlib.
Where pathlib and os work with single files and paths, shutil does the big operations: copy a file with its metadata, copy or delete an entire folder tree, move things, and zip up directories. It is the module you reach for when "copy this whole folder" is the task.
copyfile (bytes) vs copy (+perms) vs copy2 (+timestamps). Default to copy2 for a faithful duplicate that keeps permissions and timestamps. Use copyfile when you only care about the contents. copytree duplicates an entire directory tree.
Copying files and directories
import shutil
# Copy a single file (contents + permissions)
shutil.copy("report.pdf", "backup/report.pdf")
# copy2 also preserves metadata (timestamps) — usually what you want
shutil.copy2("report.pdf", "backup/report.pdf")
# Copy an ENTIRE directory tree
shutil.copytree("project/", "project_backup/")
# copytree with options
shutil.copytree(
"project/", "project_backup/",
dirs_exist_ok=True, # don't fail if target exists (3.8+)
ignore=shutil.ignore_patterns("*.pyc", "__pycache__", ".git")
)
# Copy just the contents of a file object (low-level)
with open("source.txt", "rb") as src, open("dest.txt", "wb") as dst:
shutil.copyfileobj(src, dst) # efficient streaming copycopyfile copies only the bytes. copy copies bytes + permission bits. copy2 copies bytes + permissions + timestamps and other metadata. Default to copy2 when you want a faithful duplicate; use copyfile when you only care about contents.
rmtree is permanent and recursive — like rm -rf. shutil.rmtree("folder/") deletes the folder and everything inside, irreversibly. There is no recycle bin. Double-check the path before calling, especially if it is built from variables.
Moving and deleting
import shutil
# Move a file or directory (also used for renaming)
shutil.move("old_location/file.txt", "new_location/file.txt")
shutil.move("temp.txt", "archive/") # into a directory
# Delete an ENTIRE directory tree (files and subdirectories)
shutil.rmtree("build/") # like 'rm -rf build/'
# WARNING: rmtree is permanent and recursive — there is no undo.
# Safely handle errors during rmtree
shutil.rmtree("maybe_missing/", ignore_errors=True) # no error if absent
# To remove a SINGLE file, use os or pathlib instead:
import os
os.remove("single_file.txt") # or Path("f.txt").unlink()shutil.rmtree("folder/") deletes the entire folder and everything inside it, recursively and irreversibly — the equivalent of rm -rf. There is no recycle bin. Double-check the path before calling it, especially if the path is built from variables.
✅ Beginner tab complete
- I can copy a file with copy2 (preserving metadata)
- I can copy a whole directory tree with copytree
- I know copyfile (bytes) vs copy (bytes+perms) vs copy2 (bytes+perms+timestamps)
- I can use dirs_exist_ok=True to merge into an existing directory
Archives, disk usage, and which
What you will learn: zipping and extracting directories in one call, checking disk usage, and finding executables with which.
How to read this tab: make_archive and unpack_archive handle zip/tar in a single line — no need for zipfile/tarfile directly.
make_archive zips a whole directory in one call. shutil.make_archive("backup", "zip", "project/") creates backup.zip from project/. Formats include zip, tar, gztar (.tar.gz), bztar, xztar. unpack_archive extracts and auto-detects the format.
Creating and extracting archives
shutil can zip or tar an entire directory in one call, and extract archives just as easily — no need to handle the zipfile or tarfile modules directly for common cases.
import shutil
# Create a .zip of an entire directory
# make_archive(base_name, format, root_dir)
shutil.make_archive("backup", "zip", "project/")
# Creates backup.zip containing the contents of project/
# Other formats: 'tar', 'gztar' (.tar.gz), 'bztar', 'xztar'
shutil.make_archive("backup", "gztar", "project/") # backup.tar.gz
# See available formats on this system
print([name for name, desc in shutil.get_archive_formats()])
# ['bztar', 'gztar', 'tar', 'xztar', 'zip']
# Extract an archive
shutil.unpack_archive("backup.zip", "restored/")
shutil.unpack_archive("data.tar.gz", "extracted/") # format auto-detected
# A full backup-with-timestamp pattern
from datetime import datetime
stamp = datetime.now().strftime("%Y%m%d_%H%M%S")
archive = shutil.make_archive(f"backup_{stamp}", "zip", "important_data/")
print(f"Created {archive}")shutil.which checks if a tool is installed. shutil.which("ffmpeg") returns its path or None — perfect for verifying a required external program exists before using it. disk_usage reports total/used/free space.
Disk usage and finding executables
import shutil
# Disk usage of the filesystem containing a path
usage = shutil.disk_usage("/")
print(f"Total: {usage.total // (2**30)} GB")
print(f"Used: {usage.used // (2**30)} GB")
print(f"Free: {usage.free // (2**30)} GB")
# which — find an executable on PATH (like the Unix 'which' command)
print(shutil.which("python3")) # /usr/bin/python3 (or None if not found)
print(shutil.which("git")) # path to git, or None
# Useful for checking whether a required tool is installed:
if shutil.which("ffmpeg") is None:
print("ffmpeg is not installed")
# Get terminal size — for formatting CLI output
size = shutil.get_terminal_size()
print(f"Terminal: {size.columns} columns x {size.lines} lines")dirs_exist_ok=True (Python 3.8+) to merge into an existing directory instead.✅ Intermediate tab complete
- I can create a zip/tar of a directory with make_archive
- I can extract an archive with unpack_archive
- I can check free space with disk_usage
- I can find an executable on PATH with shutil.which
Copy fidelity, move semantics, and fast-copy
What you will learn: the full copy-function hierarchy, why move is sometimes instant and sometimes a full copy, and platform fast-copy syscalls.
How to read this tab: Read when copying large files or moving across filesystems — the performance characteristics matter.
copy functions, metadata fidelity, and platform limits
The family of copy functions in shutil forms a deliberate hierarchy of metadata fidelity, and understanding it matters when duplicates must be faithful. copyfile copies only data; copymode copies only permission bits; copystat copies permissions, timestamps, and flags but not data; copy combines copyfile + copymode; and copy2 combines copyfile + copystat for the most complete duplicate. Even copy2, however, cannot always preserve every attribute — extended attributes, ACLs, and resource forks are platform-dependent and may be lost.
import shutil, os
# copytree with a custom copy_function and ignore callable
def log_copy(src, dst, *, follow_symlinks=True):
print(f"copying {src} -> {dst}")
return shutil.copy2(src, dst, follow_symlinks=follow_symlinks)
shutil.copytree("src/", "dst/", copy_function=log_copy)
# rmtree with an error handler (e.g. to fix read-only files on Windows)
def handle_remove_readonly(func, path, exc_info):
os.chmod(path, 0o700) # make writable
func(path) # retry the removal
shutil.rmtree("readonly_dir/", onexc=handle_remove_readonly) # 3.12+ onexc
# Registering a custom archive format
def make_custom(base_name, base_dir, **kwargs):
... # custom archiving logic
# shutil.register_archive_format("myfmt", make_custom, [], "My format")
# move semantics: if dst is on a different filesystem, move falls back
# to copy2 + rmtree (not an atomic rename). On the SAME filesystem it
# uses os.rename (atomic and instant). This matters for large files:
# same fs -> instant metadata rename
# cross fs -> full byte copy then delete (slow, not atomic)
shutil.move("/tmp/bigfile", "/home/user/bigfile") # may be a full copy
# COPY_BUFSIZE controls the streaming buffer size for copyfileobj
# Larger buffers can speed up copies of very large files
with open("huge.bin","rb") as s, open("out.bin","wb") as d:
shutil.copyfileobj(s, d, length=16 * 1024 * 1024) # 16 MB bufferAn important performance and correctness subtlety lies in shutil.move: when source and destination are on the same filesystem, it performs an atomic os.rename — instantaneous regardless of file size, because only directory metadata changes. When they are on different filesystems (for example moving from /tmp to a home directory on a separate mount), it must fall back to copying every byte and then deleting the original, which is slow for large files and not atomic. For the common case of copying single large files, modern Python (3.8+) also uses platform-specific fast-copy syscalls like sendfile on Linux and fcopyfile on macOS under the hood, making copyfile and copy2 substantially faster than a naive read/write loop.
✅ Expert tab complete
- I know the copyfile/copymode/copystat/copy/copy2 fidelity hierarchy
- I know move is atomic on the same filesystem but a full copy across filesystems
- I know copy2/copyfile use fast-copy syscalls (sendfile/fcopyfile) on modern Python
- I can pass a custom copy_function or error handler to copytree/rmtree