Python Generators

TL;DR

A generator function with the yield keyword keeps track of its state, and the next time it is called, it will continue from the same state.

For example, we can use yield to get even numbers in a function. How we can treat the function as an iterable:

def even_numbers(beginning: int, maximum: int):
    for i in range(beginning, maximum + 1):
        if i % 2 == 0:
            yield i
 
numbers = even_numbers(2, 10)
for number in numbers:
    print(number)
    
"""
2
4
6
8
10
"""

The same does not work using return:

def return_even(beginning: int, maximum: int):
    for i in range(beginning, maximum + 1):
        if i % 2 == 0:
            return i
 
numbers = return_even(2, 10)
for number in numbers:
    print(number)
"""
Traceback (most recent call last):
...
	for number in numbers:
                  ^^^^^^^
TypeError: 'int' object is not iterable

Explanation

When writing an iterator, such as in a class, we implement __iter__ and __next__. The issue with using them along with return is that they conjure up the entire series each time a new item is required. This can be a problem for memory, since we need the program to hold a possibly very long list.

Another instance where simply returning can be problematic is in Recursive reasoning, since a recursive function may recursively generate every item in a series many times over, especially the beginning.

The solution is to use generators, which will only return the next item in a series. They work mostly like normal functions, as they can be called and will return values, but the value a generator function returns differs from a normal function. A normal function should return the same value every time, given the same arguments. A generator function, on the other hand, should remember its current state and return the next item in the series, which may be different from the previous item.

Yield

The keyword yield is used to mark the value that the function returns. For example, the following function generates integers, starting from zero up to max_value:

Note

Unlike return, the yield keyword does not “close” the function.

 
def counter(max_value: int):
    number = 0
    while number <= max_value:
        yield number
        number += 1

We can use it along with next(), which retrieves the next value in an iterable:

    numbers = counter(10)
    print("First value:")
    print(next(numbers)) # prints 1
    print("Second value:")
    print(next(numbers)) # prints 2