Python protocols. When to use them in your projects to abstract and decoupling

What are Python Protocols and when to use them to complement or sustitute abstract classes and MixIns.

Vibrant Python logo with the Protocols composability text.


Python protocols. Defining a protocol and when to use it

Protocols in Python are a feature introduced in Python 3.8. Protocols provide a way of closing the gap between type hinting and the runtime type system, allowing us to do structural subtyping during typechacking. They annotate duck-typed during that typechecking.

Protocols are an alternative (or a complement) to inheritance, abstract classes and Mixins. While those could be great options for some use cases, the first ones may become complicate to follow as the class hierarchy increase in complexity. It usually requires intermediate objects that could became a problem for future new features and code manteiners. Mixins, however, could be a better option as they keep contained each functionality (e.g HashMixIn, ToDictMixIn.. ). The naming convention is usually xMixIn or adding the suffix -able / -ible.

So, why are Protocols useful in Python? Protocols provide a solid API that users can depend on, without relaying on a specific type. That allow us to write more composable and maintainable code.


How to define a Protocol

Let’s create an example on how to create API where an Explainable protocol

from pydantic import BaseModel
from typing import List, Optional, Tuple
import numpy as np

import matplotlib.pyplot as plt
import matplotlib.figure
from abc import ABC

class BaseScorer(ABC, BaseModel):
    @abstractmethod
    def predict(self):
        ...

class MachineLearningScorer(BaseScorer):
    weights: np.ndarray
    learning_rate: float
    epochs: int
    batch_size: int
    loss_function: str
    optimizer: str
    accuracy: Optional[float] = None
    training_time: Optional[float] = None

    class Config:
        arbitrary_types_allowed = True
    
    def predict(self) -> float:
        # Not so smart
        return 1.0


class Explainable(Protocol):
    model: BaseScorer 
    
    def explain(self) -> Tuple[matplotlib.figure.Figure, plt.Axes] | str: 
        ...
        
class CreditEvaluator(BaseModel):
    model: BaseScorer
    
    def explain(self) -> Tuple[matplotlib.figure.Figure, plt.Axes] | str:
        print("Credit given due to several reasons")

We can now easily create a function that need Explainable models and asses their fairness:

def asses_fairness(model: Explainable):
    model.explain()

Note how all Scorers based on BaseScorer must implement a predict method. However they could be explainable or not


Understanding interactions between ABC, MixIns and Protocols in Python

Here we have created an example where it is clear how those concepts interact, being both complementary and partially subsitutive.

from abc import ABC, abstractmethod
from typing import Protocol

# Abstract Base Class
class Document(ABC):
    @abstractmethod
    def display(self):
        pass

# Mixin for search functionality
class SearchableMixin:
    def search(self, query):
        return f"Searching for '{query}' in {self.__class__.__name__}"

# Protocol for typing
class Displayable(Protocol):
    def display(self):
        pass

# Concrete implementation of a Document
class PdfDocument(Document, SearchableMixin):
    def display(self):
        return "Displaying PDF document content"

class WordDocument(Document, SearchableMixin):
    def display(self):
        return "Displaying Word document content"

def view_document(doc: Displayable):
    print(doc.display())

# Usage
pdf = PdfDocument()
word = WordDocument()

view_document(pdf)  # Works with PdfDocument
view_document(word) # Works with WordDocument

print(pdf.search("Python"))  # Utilizing mixin method

Mixins are more tightly coupled to the classes they are mixed into. They are essentially a set of methods that a class can inherit from to gain certain functionalities. This inheritance implies a closer relationship between the mixin and the class, as the class directly incorporates the mixin’s methods into its own structure.

In our previous example, SearchableMixin is a mixin because it directly contributes additional functionality (the search method) to the classes that inherit from it (PdfDocument and WordDocument). This functionality is integrated into the structure of these classes.

Protocols, on the other hand, are more generic and loosely coupled. They are used primarily for type checking, allowing Python to understand that certain classes are “compatible” or fulfill a specific interface, without those classes necessarily inheriting from a common ancestor.

Protocols don’t provide implementation,they only define a set of methods that implementing classes should have. In the example, Displayable is a protocol because it is used to indicate that any object passed to view_document must have a display method. It doesn’t matter how the method is implemented or which class hierarchy the object belongs to.


Composition of Protocols

Protocols can be mixed without a problem. But imagine that in our firs example, you have an Explainable protocol and you want also to create a Predictable protocol.

To refer those protocols on a single type, you can’t do it with an Union type alias, as that will match anything that satisfies at least one protocol, not both of them.

To achieve that functionality, you should make use of a composite protocol. Note that we need to subclass from Protocol, as subclassing from other Protocol do not automatically convert it in that.

from typing import runtime_checkable 

class Predictable(Protocol):
    def predict(self) -> list[int] | int | None:
        ...
    
@runtime_checkable
class ShapModel(Predictable, Explainable, Protocol):
    ...

You can use ShapModel anywhere a model should support predict and explain methods, without having to inherit from any base model. Remember that the especification of the protocol in the class declaration is optional class NewModel(ShapModel) or class NewModel.

For making it possible to check if an object satisfies a protocol with isinstance(), it is needed to use the runtime_checkable protocol. It is calling a __hasattr__ method of each of the expected variables and functions of the protocol.


´

Stay updated on Python tips

Hopefully, this post has helped familiarize you with Protocols in Python, how to use them and their main advantages and enabling you to enjoy their benefits.

If you want to stay updated…

Carlos Vecina
Carlos Vecina
Senior Data Scientist at Jobandtalent

Senior Data Scientist at Jobandtalent | AI & Data Science for Business

Related