Connect simple iterators so each step does exactly one transformation.
What You Will Learn
A composable pipeline often consists of a reader, validator, filter and mapper. Separate these steps so they can be tested and reused. Use yield from to delegate to another iterable when appropriate; do not hide resource lifetime, exceptions or unbounded buffering behind a chain of generators.
Many iterator tools are lazy. Creating the iterator does not necessarily perform the work; advancing it can consume the original source.
Step-by-Step Example
The example below makes the behavior visible with small input values that you can change safely.
def nonempty(lines):
for line in lines:
if line.strip():
yield line.strip()
def lengths(words):
for word in words:
yield len(word)
words = ["sun", "", "mountain", "tree"]
print(list(lengths(nonempty(words))))How the Example Works
- The definitions nonempty, lengths separate the main steps into named components; calls to those components drive the example.
- The variable words holds the result of ['sun', '', 'mountain', 'tree']. Later lines use that value or inspect how it changes.
- The print() calls make the return values or state visible so you can compare them with the explanation and check your prediction.
For the chosen input, the expression list(lengths(nonempty(words))) prints [3, 8, 4]. Compare that result with the rule above before you change the argument.
Expected output from the example:
[3, 8, 4]When You Would Use This
This pattern is useful in readers for CSV files, event streams, or paginated API results, where collecting everything in memory is unnecessary. Because iterator state advances when values are consumed, decide whether you need a reusable collection or a one-time pass. Keep files and connections open until their lazy consumers finish.
Practice: Modify the Code
Add a stage that keeps only words longer than four characters.
Verify the changed output against the behavior described above, including the relevant invalid or boundary case.
For complete parameters and version details, consult the official Python documentation.
Common Beginner Mistakes
- Mixing file reading, data validation and business decisions in one opaque generator expression.
- Reusing a consumed pipeline and expecting previously yielded results to appear again.
Key Takeaways
- Connect simple iterators so each step does exactly one transformation.
- A composable pipeline often consists of a reader, validator, filter and mapper.
- Apply this operation only after checking the assumptions and input conditions shown in the example.