Python Lazy Evaluation and Streaming Data

Use lazy evaluation to stream records without loading every item.

What You Will Learn

A generator postpones computation until a consumer requests the next value. This supports reading logs, CSV rows, and large streams in stages. Streaming controls memory growth, but it also moves errors and I/O work to the moment the iterator is consumed.

For file-backed generators, understand when the file will be closed and avoid iterating outside its context manager.

Many iterator tools are lazy. Creating the iterator does not necessarily perform the work; advancing it can consume the original source.

Step-by-Step Example

The example below makes the behavior visible with small input values that you can change safely.

def active_records(records):
    for record in records:
        if record['active']:
            yield record['name'].upper()

records = [{'name': 'ada', 'active': True}, {'name': 'ben', 'active': False}]
results = active_records(records)
print(next(results))
print(list(results))

How the Example Works

The function active_records defines reusable behavior; its body executes when it is called, not when it is declared.
The variable records holds the result of [{'name': 'ada', 'active': True}, {'name': 'ben', 'active': False}]. Later lines use that value or inspect how it changes.
The variable results holds the result of active_records(records). Later lines use that value or inspect how it changes.
The print() calls make the return values or state visible so you can compare them with the explanation and check your prediction.

Expected output from the example:

ADA
[]

When You Would Use This

This pattern is useful in readers for CSV files, event streams, or paginated API results, where collecting everything in memory is unnecessary. Because iterator state advances when values are consumed, decide whether you need a reusable collection or a one-time pass. Keep files and connections open until their lazy consumers finish.

Practice: Modify the Code

Make an iterator that reads and filters individual records instead of building a temporary list.

Verify the changed output against the behavior described above, including the relevant invalid or boundary case.

For complete parameters and version details, consult the official Python documentation.

Common Beginner Mistakes

Assuming the body of a generator runs when the generator function is called.
Overlooking this documented constraint: streaming controls memory growth, but it also moves errors and I/O work to the moment the iterator is consumed.

Key Takeaways

Use lazy evaluation to stream records without loading every item.
A generator postpones computation until a consumer requests the next value.
For file-backed generators, understand when the file will be closed and avoid iterating outside its context manager.