Python Rotating Log Files with RotatingFileHandler

RotatingFileHandler limits an individual log file by size, keeping a configured number of backups.

Understanding the Concept

RotatingFileHandler limits an individual log file by size, keeping a configured number of backups. This is useful for bounded local diagnostics, not a guarantee of complete audit retention.

Avoid multiple processes writing to the same rotating file without coordinated logging.

How this helps diagnose real applications

First decide what question you are trying to answer: whether a particular branch runs, why an operation failed, or what contextual event should be recorded. Logging, interactive debugging, traceback inspection and controlled timing complement each other but produce different kinds of evidence.

Diagnostic output should preserve enough context to reproduce the problem without recording passwords, tokens, full personal records or untrusted payloads. Keep instrumentation focused so it does not hide the original behavior.

Worked Example

Read the code from the first line to the last. Identify which values are inputs, which operation is being demonstrated and what should happen before running it.

import logging
from logging.handlers import RotatingFileHandler
from tempfile import TemporaryDirectory
from pathlib import Path
with TemporaryDirectory() as folder:
    handler = RotatingFileHandler(Path(folder) / "app.log", maxBytes=120, backupCount=2)
    logger = logging.getLogger("rotation-demo")
    logger.addHandler(handler)
    for index in range(12):
        logger.warning("Test record number %02d", index)
    handler.close()
    logger.removeHandler(handler)
    print(sorted(path.name for path in Path(folder).iterdir()))

Walkthrough: What Each Part Does

The imports make logging, logging.handlers, tempfile, pathlib available. These are standard-library building blocks unless the example explicitly names an external tool.
The with or async with scope controls lifetime. Setup takes place on entry, and cleanup runs on exit even when an exception interrupts the body.
The loop repeats one well-defined unit of work. Check termination, ordering and whether an item may be processed more than once.
Observe which message or diagnostic field appears and which condition caused it. Modify the input deliberately to test the failure path.

This demonstration focuses on one practical behavior. Before changing several lines, vary one input or one configuration choice and check whether the result changes for the reason you expected.

Example Output

['app.log', 'app.log.1', 'app.log.2']

The output above was captured by running the example in the available Python environment. The order of concurrent events, durations and platform-dependent values can legitimately differ on your computer.

Important Behavior and Edge Cases

Expecting backupCount to prevent disk use from ever growing under all conditions, including other unrelated files.

Avoid multiple processes writing to the same rotating file without coordinated logging.

Practice Exercise

Lower maxBytes and observe when the base file and numbered backups change.

Write down the result you expect before running your changed code. Check at least one normal case and one boundary or failure case. If your example starts workers or opens resources, verify how they are cleaned up.

Further reading: official documentation and API reference.

Common Beginner Mistakes

Expecting backupCount to prevent disk use from ever growing under all conditions, including other unrelated files.
Removing the diagnostic message after seeing it once instead of reproducing the failure and checking the underlying cause.

Key Takeaways

RotatingFileHandler limits an individual log file by size, keeping a configured number of backups.
Avoid multiple processes writing to the same rotating file without coordinated logging.
Practice: Lower maxBytes and observe when the base file and numbered backups change.