GOF — Design Patterns: Elements of Reusable Object-Oriented Software

Program to an interface, not an implementation.

In Python, understanding a type means knowing the set of methods it provides, which is its interface.

The Typing Map

Since Python 3.8, there are four main approaches to typing and defining interfaces in Python, which complement each other and can be visualized in a “Typing Map”:

The four approaches are categorized based on when the type check happens (runtime vs. static) and how the type is determined (structural vs. nominal).

Runtime Checks (Done by the Python Interpreter)

Duck Typing (Structural): This is Python’s original and default typing approach. It checks an object’s structure (the methods it provides) at runtime. An object’s type is defined by its behavior, regardless of its formal class name.
Goose Typing (Nominal): This approach uses Abstract Base Classes (ABCs), available since Python 2.6. It performs type checks at runtime based on an object’s nominal type (the explicit class name or superclass).

Static Checks (Done by External Tools like MyPy)

Static Typing (Nominal): The traditional approach of languages like C and Java. Supported since Python 3.5 by the typing module and enforced by external type checkers compliant with PEP 484—Type Hints. It checks the nominal type of a variable statically (before runtime).
Static Duck Typing (Structural): Supported by the typing.Protocol class (new in Python 3.8), which is popular from languages like Go. It allows static type checkers to verify an object’s structure (methods) without requiring explicit inheritance.

Two kinds of Protocols

The term protocol in computer science can refer to commands (like HTTP in networking) or to an object protocol, which specifies the methods an object must provide to fulfill a certain role.

The sequence protocol is an example, demonstrated by the FrenchDeck example, which requires methods to behave like a sequence. This is an example of Dynamic Protocol.

It is often acceptable to implement only a part of a protocol. Implementing the __getitem__ special method alone is the key to the sequence protocol. This single method is enough to support:

Retrieving items by index (e.g., v[0], v[-1]).
Iteration (e.g., for c in v:).
The in operator (e.g., 'E' in v).

This is why a protocol is considered an informal interface in the style of Smalltalk.

With the adoption of PEP 544—Protocols: Structural subtyping (static duck typing)— in Python 3.8, the word “protocol” gained a second, formal meaning in Python. To be specific, two types are now recognized:

Dynamic Protocol:
- These are the informal protocols Python has always had.
- They are implicit, defined by convention, and described in the documentation (especially the “Data Model” chapter of The Python Language Reference).
Static Protocol:
- Introduced in Python 3.8 via PEP 544.
- They have an explicit definition as a subclass of typing.Protocol.

There are two main differences between them:

Completeness:
- An object can implement only part of a dynamic protocol and still be useful.
- To fulfill a static protocol, an object must provide every method declared in the protocol class.
Verification:
- Static protocols can be verified by static type checkers.
- Dynamic protocols can’t

Both kinds of protocols share the crucial characteristic that a class never needs to declare that it supports a protocol by name (i.e., through inheritance).

In addition to static protocols, Python also provides Abstract Base Classes (ABCs) as another way to define an explicit interface in code.

Programming Ducks

The discussion of dynamic protocols begins with two of the most important in Python: the sequence and iterable protocols. The Python interpreter is designed to handle objects that provide even a minimal implementation of these protocols.

Python Digs Sequences

The Python Data Model’s philosophy is to cooperate extensively with essential dynamic protocols. When it comes to sequences, the interpreter works hard to support even the simplest implementations.

The collections.abc.Sequence abstract base class (ABC) formalizes what a full sequence is expected to support, but built-in sequences like list and str do not rely on this ABC. The ABCs mostly formalize interfaces that the built-in objects and the interpreter already support implicitly.

A correct subclass of abc.Sequence must implement the abstract methods __getitem__ and __len__ (from abc.Sized).
The other methods in abc.Sequence (like __iter__ and __contains__) are concrete and provide default implementations.

The Python interpreter provides special fallback behavior for sequence-like objects:

Iteration Fallback: If an object is missing an __iter__ method, Python checks for a __getitem__ method. If found, it iterates over the object by repeatedly calling __getitem__ with integer indices starting from 0.
in Operator Fallback: If an object is missing the __contains__ method, Python makes the in operator work by performing a sequential scan using the iteration fallback (by calling __getitem__) to check if an item is present.

Let’s rewrite FrenchDeck class that we used in chapter 1 and create a new Vowels class:

class Vowels:
	def __getitem__(self, i):
		return 'AEIOU'[i]
 
 
import collections
Card = collections.namedtuple('Card', ['rank', 'suit'])
 
class FrenchDeck:
	ranks = [str(n) for n in range(2, 11)] + list('JQKA')
	suits = 'spades diamonds clubs hearts'.split()
	
	def __init__(self):
		self._cards = [Card(rank, suit) for suit in self.suits for rank in self.ranks]
	
	def __len__(self):
		return len(self._cards)
		
	def __getitem__(self, position):
		return self._cards[position]

The Vowels class (which only implements __getitem__) and the original FrenchDeck class (which implements both __getitem__ and __len__) are examples that rely on this special treatment by the interpreter, rather than inheriting from abc.Sequence.

The interpreter’s willingness to use __getitem__ as a fallback for both iteration and the in operator is considered an extreme form of duck typing for the iterable protocol. These behaviors are implemented directly in the interpreter’s core (mostly in C) and are not dependent on the concrete methods defined in the collections.abc.Sequence ABC. This dynamic nature is why static type checkers cannot fully deal with them.

Monkey Patching: Implementing a Protocol at runtime

Monkey patching is the practice of changing a module, class, or function dynamically (at runtime) to add features or fix issues.

The FrenchDeck class, which implements the immutable sequence protocol (__getitem__ and __len__), cannot be shuffled using the standard library function random.shuffle:

random.shuffle(x) is designed to shuffle a sequence in place by swapping items.
When attempting to shuffle a FrenchDeck instance, a TypeError: 'FrenchDeck' object does not support item assignment is raised.
This error occurs because random.shuffle requires the object to adhere to the mutable sequence protocol, which includes providing a __setitem__ method for item assignment. Since FrenchDeck is an immutable sequence, it is missing this method.

Because Python is a dynamic language, the necessary method can be added to the class at runtime:

A plain function, set_card, is defined to perform the item assignment by directly accessing the deck’s internal mutable list, _cards.
```
def set_card(deck, position, card):
    deck._cards[position] = card
```
This function is then attached to the FrenchDeck class as the special method __setitem__.
```
FrenchDeck.__setitem__ = set_card
```
The deck object can now be shuffled by random.shuffle because the class now implements the required method of the mutable sequence protocol.

This process is monkey patching: modifying a class’s behavior without altering its source code. While powerful, this type of patching creates a tight coupling with the patched code, often relying on internal, undocumented attributes (like _cards).

This example highlights the dynamic nature of protocols under duck typing. The random.shuffle function does not care about the object’s class; it only requires the object to implement the methods of the mutable sequence protocol. It works even if the object acquires the necessary methods (like __setitem__) after it was created.

Defensive Programming and “Fail Fast”

Defensive programming is a set of practices used to improve code safety and maintainability. In dynamically typed languages, the “fail fast” principle is key: raise a runtime error as soon as an invalid argument is detected, ideally at the start of a function. This makes bugs easier to trace and fix.

Fail Fast Strategies

Handling Iterable Arguments: When a function requires a sequence of items, avoid checking the argument’s type. Instead, immediately build a working copy, such as a list, from the argument:
```
def __init__(self, iterable):
    self._balls = list(iterable)
```
This approach is flexible, accepting any iterable that fits in memory. If the argument is not iterable, it fails immediately with a clear TypeError when list() is called, rather than blowing up later in a different method.
Checking for Mutable Sequences: If the data cannot be copied (e.g., it’s too large or needs to be changed in place, like in random.shuffle), a runtime check is necessary: isinstance(x, abc.MutableSequence).
Checking Iterability and Length:
- To obtain an iterator and fail fast if the argument is not iterable, call iter(x) immediately.
- To reject infinite generators or ensure a full sequence exists, call len(x) first. This is usually cheap and raises an error immediately for invalid arguments.

Type Hints can catch some problems, but are not a complete defense, as they are not enforced at runtime and may be bypassed when variables are tagged with the Any type. Fail fast is the final line of defense.

Defensive code leveraging duck types can also include logic to handle different types without explicit checks like isinstance() or hasattr(). This is often done using the EAFP (“It’s Easier to Ask Forgiveness Than Permission”) coding style, as demonstrated by the way collections.namedtuple handles the field_names argument (accepting either a single string or an iterable of strings):

try:
    field_names = field_names.replace(',', ' ').split() # Assume string
except AttributeError:
    pass # If it's not a string, it will raise AttributeError, and we assume it's iterable
 
field_names = tuple(field_names) # Convert to a tuple for safety and a private copy
 
if not all(s.isidentifier() for s in field_names):
    raise ValueError('field_names must all be valid identifiers')

This duck-typing approach is more expressive than static type hints. While the type hint for field_names might be Union[str, Iterable[str]], this doesn’t capture the requirement that the string must contain identifiers separated by commas or spaces. The EAFP block handles this logic dynamically.

Goose Typing

Bjorne Stroustrup, creator of C++

An abstract class represents an interface.

Since Python does not have an interface keyword, we use Abstract Base Classes (ABCs) to define interfaces.

The value of ABCs in duck-typed languages is that they define interfaces for explicit type checking at runtime, which complements the flexibility of duck typing. ABCs also introduce virtual subclasses: classes that do not inherit from an ABC but are recognized by isinstance() and issubclass() checks. This mechanism is known as Goose Typing.

Waterfowl and ABCs (Alex Martelli’s Essay)

Duck Typing is the practice of ignoring an object’s actual type and focusing only on whether it implements the required methods (its morphology and behavior). This often means avoiding isinstance() checks.

The need for Goose Typing arises because similar methods, like multiple draw() methods on unrelated classes (Artist, Gunslinger, Lottery), are not enough to guarantee abstract equivalence. Parallel evolution in programming can lead to “accidental similarities” that confuse duck typing.

Goose typing, loosely analogous to the biological approach of cladistics (focusing on inheritance from common ancestors), recommends:

Using isinstance(obj, cls) only when cls is an Abstract Base Class (its metaclass is abc.ABCMeta).
Useful existing ABCs can be found in collections.abc and the numbers module.

Advantages of ABCs

ABCs offer several conceptual and practical advantages:

register Class Method: This allows developers to declare that a class is a virtual subclass of an ABC, even if it was not developed with awareness of the ABC and does not inherit from it. The registered class only needs to meet the method name, signature, and semantic requirements of the ABC.
Automatic Recognition: Some ABCs, especially those defined by special methods, automatically recognize a class as a subclass without explicit registration or inheritance.
- Example: A class that implements __len__ is automatically recognized as a virtual subclass of abc.Sized.
```
>>> class Struggle:
...     def __len__(self): return 23
...
>>> from collections import abc
>>> isinstance(Struggle(), abc.Sized)
True
```

Summary of Goose Typing

Goose typing entails:

Subclassing from ABCs to explicitly declare that you are implementing a previously defined interface.
Runtime type checking using ABCs as the second argument for isinstance() and issubclass().

Inheriting from an ABC is a clear declaration of intent. The ability to register a virtual subclass (e.g., Sequence.register(FrenchDeck)) makes type checks more flexible than when checking against concrete classes, which limits polymorphism.

The use of isinstance and issubclass becomes more acceptable if you are checking against ABCs instead of concrete class because, otherwise, you would limit polymorphism—an essential feature of OOP. However, while ABCs make type checking more acceptable, excessive use of isinstance checks against ABCs can be a code smell if it’s used to implement branching logic (it’s usually not ok to have a chain of if/elif/elif with isinstance checks performing different actions depending on the type of object; for that, polymorphism should be used).

However, an isinstance check against an ABC is acceptable and useful when:

You must enforce an API contract (e.g., in a system with a plug-in architecture).
Checking against an existing, general-purpose ABC (like collections.abc.Sequence).

It is strongly advised not to define custom ABCs or metaclasses in production code, as they are tools for building broad frameworks and imposing ceremony in a practical language. Most developers should focus on correctly using the existing ABCs.

Subclassing an ABC

To explicitly follow the interface of a mutable sequence, the FrenchDeck2 class is declared as a subclass of collections.abc.MutableSequence:

from collections import namedtuple, abc
 
Card = namedtuple('Card', ['rank', 'suit'])
 
class FrenchDeck2(abc.MutableSequence):
	ranks = [str(n) for n in range(2, 11)] + list('JQKA')
	suites = 'spades diamonds clubs hearts'.split()
	
	def __init__(self):
		self._cards = [Card(rank, suit) for suit in self.suits for rank in self.ranks]
		
	def __len__(self):
		return len(self._cards)
		
	def __getitem__(self, position):
		return self._cards[position]
		
	def __setitem__(self, position, value):
		self._cards[position] = value
		
	def __delitem__(self, position):
		del self._cards[position]
		
	def insert(self, position, value):
		self._cards.insert(position, value)

Looking at this UML class diagram for the MutableSequence ABC and its superclasses from collections.abc, it’s easier to understand why FrenchDeck2 class is implemented in this way:

Here is a summary of the FrenchDeck2 class implementation:

The Sequence abstract base class requires the implementation of __len__ and __getitem__, which make the deck behave like a read-only sequence that supports indexing and iteration.
The MutableSequence abstract base class extends Sequence and additionally requires __setitem__, __delitem__, and insert, which make the deck mutable — allowing elements to be modified, removed, or inserted (for example, enabling the deck to be shuffled using random.shuffle):
1. __setitem__** (needed for shuffling and item assignment)
2. __delitem__ (needed for deleting items by index
3. insert (needed for inserting an item at a specific position)

Subclassing an Abstract Base Class (ABC) requires the concrete class to implement all of the ABC’s abstract methods. The implementation of these abstract methods is only checked at runtime when you try to create an instance of the subclass. If any abstract method is missing, a TypeError exception is raised.

The FrenchDeck2 class must implement __delitem__ and insert, even if the current examples don’t require their behavior, simply because the MutableSequence ABC requires them.

In return for implementing the abstract methods, FrenchDeck2 inherits many concrete methods (methods that already have an implementation) from the ABCs in the inheritance chain:

From collections.abc.Sequence: __contains__, __iter__, __reversed__, index, and count.
From collections.abc.MutableSequence: append, reverse, extend, pop, remove, and __iadd__ (which supports the += operator).

These concrete methods are implemented using the class’s public interface and do not rely on the internal details of the instance. A developer can choose to override these inherited methods with a more efficient implementation if needed (e.g., overriding the sequential scan of __contains__ with a faster binary search).

ABCs in the Standard Library

The standard library provides several Abstract Base Classes (ABCs), primarily in the collections.abc module, though others exist in packages like io and numbers. Note that the collections.abc module is implemented outside of the collections package—in Lib/_collections_abc.py—to reduce loading time. Every ABC relies on the core abc module, but it usually does not need to be imported directly unless a new ABC is being created.

The collections.abc module defines 17 ABCs, exhibiting extensive multiple inheritance:

These ABCs are organized into the following conceptual groups:

Foundation ABCs (Iterable, Container, Sized):
- Iterable supports iteration via __iter__.
- Container supports the in operator via __contains__.
- Sized supports len() via __len__.
- Every collection should either inherit from these or implement the compatible protocols.
Collection:
- Added in Python 3.6 to simplify subclassing by inheriting from Iterable, Container, and Sized. It has no methods of its own.
Main Collection Types (Sequence, Mapping, Set):
- These are the primary immutable collection types, and each has a corresponding mutable subclass (e.g., MutableSequence).
MappingView:
- In Python 3, the objects returned by dict methods like .items(), .keys(), and .values() implement the interfaces defined in this cluster (ItemsView, KeysView, and ValuesView). The ItemsView and KeysView also implement the rich interface of Set.
Iterator:
- Subclasses Iterable, reflecting the relationship discussed in the iterator protocol.
Utility ABCs (Callable, Hashable):
- These are not collections but are included to support runtime type checking for objects that must be callable or hashable.
- For callable detection, the built-in callable(obj) function is more convenient than isinstance(obj, Callable).

Caveats with `isinstance` on `Hashable` and `Iterable`

Using isinstance with the Hashable and Iterable ABCs can sometimes be misleading:

Hashable: If isinstance(obj, Hashable) returns True, it only confirms that the object’s class implements or inherits __hash__. However, the instance obj may still not be hashable (e.g., a tuple containing an unhashable item). The most accurate way to check hashability is to call hash(obj), which will raise a TypeError if the object is truly unhashable.
Iterable: Even if isinstance(obj, Iterable) returns False, Python may still be able to iterate over the object using the __getitem__ method with $0$ -based indices (the sequence protocol fallback). The only reliable way to determine if an object is iterable is to call iter(obj).

Defining and Using an ABC

Creating Abstract Base Classes (ABCs) is a tool primarily intended for building frameworks and defining extension points. However, ABCs also serve to provide flexibility (as discussed in Abstract Base Classes) and clarity in type hints for static typing.

To justify creating a new ABC, a context is established: an ad management framework named ADAM needs to support nonrepeating random-picking classes. An ABC is defined to clearly document the interface expected of these user-provided components.

The ABC is named Tombola, which is a machine designed to pick items randomly from a finite set without repeating, until exhausted. The Tombola ABC is defined by subclassing abc.ABC. It contains two abstract methods (must be implemented by subclasses) and two concrete methods (provided by the ABC). Let’s see all of them:

The two Abstract Method are:
- .load(iterable): Add items from an iterable into the container.
- .pick(): Remove one item at random and return it. Must raise LookupError when the instance is empty.
The two Concrete Method are
- .loaded(): Returns True if there is at least one item, relying on .inspect().
- .inspect(): Returns a tuple of all items, without changing the container’s contents.

Here is the structure of the Tombola ABC:

import abc
 
class Tombola(abc.ABC):
    @abc.abstractmethod
    def load(self, iterable):
        """Add items from an iterable."""
 
    @abc.abstractmethod
    def pick(self):
        """Remove item at random, returning it.
        This method should raise `LookupError` when the instance is empty.
        """
    
    def loaded(self):
	    """Return 'True' if there's at least 1 item, 'False' otherwise."""
	    return bool(self.inspect())
	    
	def inspect(self):
		"""Return a sorted tuple with the items currently inside."""
		items = []
		while True:
			try:
				items.append(self.pick())
			except LookupError:
				break
		self.load(items)
		return tuple(items)

Key ABC Implementation Rules

Declaration: To define an ABC, you must subclass abc.ABC.
Abstract Methods: An abstract method is marked with the @abc.abstractmethod decorator. Its body is often empty except for a docstring. Subclasses are forced to override any method marked with this decorator, even if the abstract method itself has an implementation (which is possible).
Concrete Methods in ABCs: An ABC can include concrete methods (like .loaded() and .inspect()). These methods must rely only on the interface defined by the ABC (i.e., other abstract or concrete methods).
- The .inspect() method demonstrates this by relying entirely on the abstract methods: it repeatedly calls .pick() until it raises LookupError, collects the items, and then calls .load() to put them back.
- The .loaded() method uses a less efficient one-liner: return bool(self.inspect()). Concrete subclasses are expected to override such inefficient methods with better implementations.

The abstract method .pick() explicitly requires implementers to raise LookupError when the container is empty. LookupError is chosen because it is the common ancestor for IndexError and KeyError, which are the likely exceptions raised by the underlying data structures of concrete implementations (sequences and mappings).

ABCs enforce the implementation of abstract methods at runtime, specifically when attempting to instantiate the subclass. Example of a Defective Subclass:

class Fake(Tombola):
    def pick(self):
        return 13
 
f = Fake()
# Traceback (most recent call last):
# ...
# TypeError: Can't instantiate abstract class Fake with abstract method load

The TypeError is raised not when the class is defined, but when instantiation is attempted, and the message clearly states that the class is considered abstract because it failed to implement the load abstract method.

Quartz 4

Explorer

13. Interfaces, Protocols, and ABCs

The Typing Map

Two kinds of Protocols

Programming Ducks

Python Digs Sequences

Monkey Patching: Implementing a Protocol at runtime

Defensive Programming and “Fail Fast”

Fail Fast Strategies

Goose Typing

Waterfowl and ABCs (Alex Martelli’s Essay)

Advantages of ABCs

Summary of Goose Typing

Subclassing an ABC

ABCs in the Standard Library

Caveats with `isinstance` on `Hashable` and `Iterable`

Defining and Using an ABC

Graph View

Table of Contents

Quartz 4

Explorer

13. Interfaces, Protocols, and ABCs

The Typing Map

Two kinds of Protocols

Programming Ducks

Python Digs Sequences

Monkey Patching: Implementing a Protocol at runtime

Defensive Programming and “Fail Fast”

Fail Fast Strategies

Goose Typing

Waterfowl and ABCs (Alex Martelli’s Essay)

Advantages of ABCs

Summary of Goose Typing

Subclassing an ABC

ABCs in the Standard Library

Caveats with isinstance on Hashable and Iterable

Defining and Using an ABC

Graph View

Table of Contents

Caveats with `isinstance` on `Hashable` and `Iterable`