Unexpected memory growth may come from caches, long-lived references, file handles or native allocations.
Understanding the Concept
Unexpected memory growth may come from caches, long-lived references, file handles or native allocations. Look for sustained growth under a repeatable workload rather than assuming garbage collection is broken.
A bounded cache and object ownership policy can fix retained-memory growth without manual GC.
How to interpret the measurement
Before optimizing, establish an objective measurement and a representative workload. Memory size, function-call counts, elapsed time and samples are different metrics: knowing one does not determine the others. Repeat observations before drawing a conclusion about a performance regression.
Profiling output is evidence for finding a bottleneck, not a performance guarantee. Check that the program still produces the correct result after any optimization and note version, input size and measurement conditions.
Worked Example
Read the code from the first line to the last. Identify which values are inputs, which operation is being demonstrated and what should happen before running it.
from collections import deque
history = deque(maxlen=3)
for index in range(10):
history.append(f"request-{index}")
print(len(history))
print(list(history))Walkthrough: What Each Part Does
- The imports make collections available. These are standard-library building blocks unless the example explicitly names an external tool.
- The loop repeats one well-defined unit of work. Check termination, ordering and whether an item may be processed more than once.
- The printed number or profile is diagnostic data. Repeat the observation and identify the unit before comparing values.
This demonstration focuses on one practical behavior. Before changing several lines, vary one input or one configuration choice and check whether the result changes for the reason you expected.
Example Output
3
['request-7', 'request-8', 'request-9']The output above was captured by running the example in the available Python environment. The order of concurrent events, durations and platform-dependent values can legitimately differ on your computer.
Important Behavior and Edge Cases
Keeping an unlimited list of processed records in a daemon and blaming the garbage collector for retaining it.
A bounded cache and object ownership policy can fix retained-memory growth without manual GC.
Practice Exercise
Compare an unbounded list with a bounded deque during 1,000 simulated events.
Write down the result you expect before running your changed code. Check at least one normal case and one boundary or failure case. If your example starts workers or opens resources, verify how they are cleaned up.
Further reading: official documentation and API reference.
Common Beginner Mistakes
- Keeping an unlimited list of processed records in a daemon and blaming the garbage collector for retaining it.
- Comparing results from different workloads or environments as if the measurements were directly interchangeable.
Key Takeaways
- Unexpected memory growth may come from caches, long-lived references, file handles or native allocations.
- A bounded cache and object ownership policy can fix retained-memory growth without manual GC.
- Practice: Compare an unbounded list with a bounded deque during 1,000 simulated events.