Python Generators vs Lists: Memory and Performance

Compare generator expressions with lists when processing sequences.

What You Will Learn

A list comprehension evaluates and stores every result immediately. A generator expression produces items only as its iterator advances, so it can reduce peak memory when values are consumed once. A generator is not automatically faster, and its results are exhausted after one pass.

sys.getsizeof() reports shallow object size, not full process memory use.

Many iterator tools are lazy. Creating the iterator does not necessarily perform the work; advancing it can consume the original source.

Step-by-Step Example

The example below makes the behavior visible with small input values that you can change safely.

import sys
numbers_list = [value * value for value in range(10000)]
numbers_generator = (value * value for value in range(10000))
print(len(numbers_list))
print(sys.getsizeof(numbers_list) > sys.getsizeof(numbers_generator))
print(next(numbers_generator))

How the Example Works

The imports provide the standard-library tools used in the demonstration: sys.
The variable numbers_list holds the result of [value * value for value in range(10000)]. Later lines use that value or inspect how it changes.
The variable numbers_generator holds the result of (value * value for value in range(10000)). Later lines use that value or inspect how it changes.
The print() calls make the return values or state visible so you can compare them with the explanation and check your prediction.

Expected output from the example:

10000
True
0

When You Would Use This

This pattern is useful in readers for CSV files, event streams, or paginated API results, where collecting everything in memory is unnecessary. Because iterator state advances when values are consumed, decide whether you need a reusable collection or a one-time pass. Keep files and connections open until their lazy consumers finish.

Practice: Modify the Code

Process 100,000 squares with sum() and compare a list comprehension to a generator expression.

Verify the changed output against the behavior described above, including the relevant invalid or boundary case.

For complete parameters and version details, consult the official Python documentation.

Common Beginner Mistakes

Calling len() on a generator or assuming it can be traversed repeatedly without recreating it.
Overlooking this documented constraint: a generator expression produces items only as its iterator advances, so it can reduce peak memory when values are consumed once.

Key Takeaways

Compare generator expressions with lists when processing sequences.
A list comprehension evaluates and stores every result immediately.
sys.getsizeof() reports shallow object size, not full process memory use.