🧠 Python DeepCuts — 💡 Lazy Evaluation & Iterators Internals

Python is designed to avoid unnecessary work whenever possible.

Instead of generating every value upfront, Python often produces values only when they are needed. This approach is called lazy evaluation and is one of the reasons Python can efficiently handle large datasets and streaming workflows.

In this DeepCut, we’ll explore the iterator protocol, generators, and how Python executes lazy pipelines behind the scenes.

---

🧩 The Iterator Protocol

At the heart of lazy evaluation is the iterator protocol.

Every iterator implements:

• __iter__()
• __next__()

numbers = [1, 2, 3]

iterator = iter(numbers)

print(next(iterator))
print(next(iterator))
print(next(iterator))

iter() creates an iterator object.

next() requests the next available value.

This simple protocol powers much of Python’s iteration system.

---

🧠 How Iteration Ends

When an iterator runs out of values, it raises:

StopIteration

numbers = [1]

iterator = iter(numbers)

print(next(iterator))

try:
    print(next(iterator))
except StopIteration:
    print("Iterator exhausted")

This exception tells Python that iteration is complete.

---

🔄 What a for Loop Really Does

A for loop is essentially a repeated sequence of next() calls.

items = ["a", "b", "c"]

iterator = iter(items)

while True:
    try:
        item = next(iterator)
        print(item)
    except StopIteration:
        break

This is conceptually how Python executes every for loop.

---

🧬 Generators: Lazy Iterators

Generators are a convenient way to create iterators.

def count_to_three():
    yield 1
    yield 2
    yield 3

gen = count_to_three()

print(next(gen))
print(next(gen))
print(next(gen))

Unlike lists, generators do not create all values upfront.

Each value is produced only when requested.

---

🔍 Why Lazy Evaluation Matters

Consider:

squares = (x * x for x in range(1_000_000))

This does not create one million values immediately.

Instead:

print(next(squares))
print(next(squares))

Only the requested values are computed.

This dramatically reduces memory usage for large workloads.

---

⚠️ Generators Are Single-Use

Once consumed, a generator cannot be reused.

gen = (x for x in range(3))

for item in gen:
    print(item)

print(list(gen))

Output:

[]

The generator has already been exhausted.

---

🧠 Building Lazy Pipelines

Generators become especially powerful when chained together.

numbers = range(10)

evens = (x for x in numbers if x % 2 == 0)

squares = (x * x for x in evens)

print(list(squares))

Each stage processes values only when required. This creates highly efficient data-processing pipelines.

---

✅ Key Points

• Iterators implement iter() and next()
• StopIteration signals completion
• for loops are built on the iterator protocol
• Generators produce values lazily
• Lazy evaluation reduces memory consumption
• Generators are consumed as they iterate
• Generator pipelines enable efficient data processing

Lazy evaluation is one of Python’s most elegant features, allowing large computations to remain efficient and memory-friendly.

---

numbers = [1, 2, 3]

iterator = iter(numbers)

print(next(iterator))
print(next(iterator))
print(next(iterator))

numbers = [1]
iterator = iter(numbers)

print(next(iterator))

try:
    print(next(iterator))
except StopIteration:
    print("Iterator exhausted")

items = ["a", "b", "c"]

iterator = iter(items)

while True:
    try:
        item = next(iterator)
        print(item)
    except StopIteration:
        break

def count_to_three():
    yield 1
    yield 2
    yield 3

gen = count_to_three()

print(next(gen))
print(next(gen))
print(next(gen))

squares = (x * x for x in range(1_000_000))

print(next(squares))
print(next(squares))

gen = (x for x in range(3))

for item in gen:
    print(item)

print(list(gen))

numbers = range(10)

evens = (x for x in numbers if x % 2 == 0)

squares = (x * x for x in evens)

print(list(squares))