Python protocols. When to use them in your projects to abstract and decoupling
What are Python Protocols and when to use them to complement or sustitute abstract classes and MixIns.
Python protocols. Defining a protocol and when to use it
Protocols in Python are a feature introduced in Python 3.8. Protocols provide a way of closing the gap between type hinting and the runtime type system, allowing us to do structural subtyping during typechacking. They annotate duck-typed during that typechecking.
Protocols are an alternative (or a complement) to inheritance, abstract classes and Mixins. While those could be great options for some use cases, the first ones may become complicate to follow as the class hierarchy increase in complexity. It usually requires intermediate objects that could became a problem for future new features and code manteiners. Mixins, however, could be a better option as they keep contained each functionality (e.g HashMixIn, ToDictMixIn.. ). The naming convention is usually xMixIn
or adding the suffix -able
/ -ible
.
So, why are Protocols useful in Python? Protocols provide a solid API that users can depend on, without relaying on a specific type. That allow us to write more composable and maintainable code.
How to define a Protocol
Let’s create an example on how to create API where an Explainable protocol
from pydantic import BaseModel
from typing import List, Optional, Tuple
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.figure
from abc import ABC
class BaseScorer(ABC, BaseModel):
@abstractmethod
def predict(self):
...
class MachineLearningScorer(BaseScorer):
weights: np.ndarray
learning_rate: float
epochs: int
batch_size: int
loss_function: str
optimizer: str
accuracy: Optional[float] = None
training_time: Optional[float] = None
class Config:
arbitrary_types_allowed = True
def predict(self) -> float:
# Not so smart
return 1.0
class Explainable(Protocol):
model: BaseScorer
def explain(self) -> Tuple[matplotlib.figure.Figure, plt.Axes] | str:
...
class CreditEvaluator(BaseModel):
model: BaseScorer
def explain(self) -> Tuple[matplotlib.figure.Figure, plt.Axes] | str:
print("Credit given due to several reasons")
We can now easily create a function that need Explainable models and asses their fairness:
Note how all Scorers based on BaseScorer
must implement a predict
method. However they could be explainable or not
Understanding interactions between ABC, MixIns and Protocols in Python
Here we have created an example where it is clear how those concepts interact, being both complementary and partially subsitutive.
from abc import ABC, abstractmethod
from typing import Protocol
# Abstract Base Class
class Document(ABC):
@abstractmethod
def display(self):
pass
# Mixin for search functionality
class SearchableMixin:
def search(self, query):
return f"Searching for '{query}' in {self.__class__.__name__}"
# Protocol for typing
class Displayable(Protocol):
def display(self):
pass
# Concrete implementation of a Document
class PdfDocument(Document, SearchableMixin):
def display(self):
return "Displaying PDF document content"
class WordDocument(Document, SearchableMixin):
def display(self):
return "Displaying Word document content"
def view_document(doc: Displayable):
print(doc.display())
# Usage
pdf = PdfDocument()
word = WordDocument()
view_document(pdf) # Works with PdfDocument
view_document(word) # Works with WordDocument
print(pdf.search("Python")) # Utilizing mixin method
Mixins are more tightly coupled to the classes they are mixed into. They are essentially a set of methods that a class can inherit from to gain certain functionalities. This inheritance implies a closer relationship between the mixin and the class, as the class directly incorporates the mixin’s methods into its own structure.
In our previous example, SearchableMixin
is a mixin because it directly contributes additional functionality (the search method) to the classes that inherit from it (PdfDocument and WordDocument). This functionality is integrated into the structure of these classes.
Protocols, on the other hand, are more generic and loosely coupled. They are used primarily for type checking, allowing Python to understand that certain classes are “compatible” or fulfill a specific interface, without those classes necessarily inheriting from a common ancestor.
Protocols don’t provide implementation,they only define a set of methods that implementing classes should have. In the example, Displayable
is a protocol because it is used to indicate that any object passed to view_document must have a display method. It doesn’t matter how the method is implemented or which class hierarchy the object belongs to.
Composition of Protocols
Protocols can be mixed without a problem. But imagine that in our firs example, you have an Explainable
protocol and you want also to create a Predictable
protocol.
To refer those protocols on a single type, you can’t do it with an Union
type alias, as that will match anything that satisfies at least one protocol, not both of them.
To achieve that functionality, you should make use of a composite protocol. Note that we need to subclass from Protocol, as subclassing from other Protocol do not automatically convert it in that.
from typing import runtime_checkable
class Predictable(Protocol):
def predict(self) -> list[int] | int | None:
...
@runtime_checkable
class ShapModel(Predictable, Explainable, Protocol):
...
You can use ShapModel
anywhere a model should support predict
and explain
methods, without having to inherit from any base model. Remember that the especification of the protocol in the class declaration is optional class NewModel(ShapModel)
or class NewModel
.
For making it possible to check if an object satisfies a protocol with isinstance()
, it is needed to use the runtime_checkable
protocol. It is calling a __hasattr__
method of each of the expected variables and functions of the protocol.
´
Stay updated on Python tips
Hopefully, this post has helped familiarize you with Protocols in Python, how to use them and their main advantages and enabling you to enjoy their benefits.
If you want to stay updated…