Mypy: a static type checker for Python

Author

Ítalo Epifânio

Published

February 23, 2022

Introduction

The current article will show how useful the static code analyzer mypy can be and what problem it solves. For this purpose the concept of duck typing will be used to illustrate how type hints can be a great addition to a dynamic language and how mypy can increase code accuracy. The current article is divided into the following sections:

Duck typing

Python is a dynamic language that uses a concept called duck typing, which requires that type checking is deferred to runtime. Another particularity of duck typing is that it checks methods and properties instead of actually checking the object type. That’s why the slogan of this concept is:

If it walks like a duck and quacks like a duck, it must be a duck.

The next code illustrates how duck typing behaves by creating an actual ‘Duck’ class. This code exemplifies the slogan mentioned before.

class Duck:
    def walk(self):
        print('Duck walks')
    
    def quack(self):
        print('Duck quacks')
        
class Pigeon:
    def walk(self):
        print('Pigeon walks')

In the previous cell, the Duck and Pigeon classes were defined. Note that the pigeon object doesn’t quack.

If we define a method that simulates the duck’s behavior, the method will also accept the pigeon object and throw an error at runtime. The following code illustrates this behavior.

def walk_the_duck(duck):
    duck.walk()
    duck.quack()
    duck.walk()

The walk_the_duck method will behave as expect for a duck, as you can see in the next command execution.

duck = Duck()
walk_the_duck(duck)
Duck walks
Duck quacks
Duck walks

But when using the pigeon object, Python will throw an error on runtime since a pigeon doesn’t quack.

pigeon = Pigeon()
walk_the_duck(pigeon)
Pigeon walks
AttributeError: 'Pigeon' object has no attribute 'quack'

Note that the program prints Pigeon walks before throwing the error. This happens because the language tries to fit the object relative to the properties. This works for duck.walk() and fails once the code duck.quack() is executed. Remember: “if it walks like a duck and quacks like a duck, it must be a duck”. Since the Pigeon doesn’t quack it’s not a duck, which results in an error.

The next section will add a type Duck to the walk_the_duck method, exploring how Python handles typing and the duck typing concept together.

Type hint

Since Python 3.5 the language provides optional support to type hints. This feature was added to improve readability and doesn’t enforce the type notation, which means that even if the walk_the_duck method had an explicit type, the object will not be checked before runtime. For example, the next cell will throw the same error as before, even with the Duck type applied.

def walk_the_duck(duck: Duck):
    duck.walk()
    duck.quack()
    duck.walk()
walk_the_duck(pigeon)
Pigeon walks
AttributeError: 'Pigeon' object has no attribute 'quack'

Type hint was added to the Python language to improve code readbility, but the duck typing concept is still used in Python even when typing is applied. Third party tools can be used to ensure static typing and one of these tools is mypy.

Before showing how to use mypy, the next section explains the motivation and benefits of using static typing.

Why use static typing in Python?

Prior to Python 3.5, docstring was commonly used to indicate the types of the parameters. The following code shows an example of a code with docstring.

def natural_sum(x, y):
    """
    Args:
        x (int): first parameter of the sum
        y (int): second parameter of the sum
    Returns:
        Optional[int]: Sum of x and y if both bigger than zero
    """
    if x < 0 or y < 0:
        return None
    
    return x + y

The code above is documented for the users to know that the function natural_sum receives only integer parameters and can return None or int. Docstring is very helpful but once a developer forgets to update the docstring, it can lead to errors.

The introduction of type hint helped programmers to improve the readability of the code by avoiding docstring. However, it is not guaranteed that the type hint is used correctly (as mentioned in the Type hint section). By using static type tools, the type hint can be validated, ensuring that it has been set up correctly, and eliminating errors.

Another benefit of using static type tools is to reduce the number of necessary tests. For example, a test that checks for integer parameters in the function natural_sum doesn’t cover that float numbers can’t be accepted, so a new test is required to exclude float numbers. Static type checking helps developers to write more targeted tests, ignoring other data types like the float in our example.

Static type checking also helps to eliminate naive errors such as forgetting to handle a None return. Continuing with the natural_sum example, let’s use this function in a simple script:

def increment(value: int) -> int:
    return natural_sum(value, 1)

double_increment = increment(-1) + 1 # Error: None + 1

The increment function expects an integer and returns an integer but the natural_sum function can return None. In this case the programmer forgot to handle the None in the increment function. This code would throw an error at runtime. By using a static type checker the error would be catched before running the program and without having to write a test for it. This is why a static type checker can improve drastically the reliability of the code, avoiding subtle bugs and leading to more focused tests.

In summary, a static type usage can:

  • improve readability
  • lead to focused tests
  • avoid subtle bugs
  • improve reliability

Mypy

Mypy is a static type checker for Python. This tool makes writing statically typed code very easy, checking variables and functions in your program.

From the dynamic nature of Python it follows that the programmer only see errors when the code is attempted to run. When using mypy, the programmer finds bugs in the programs without even running the code.

Using mypy works as follows: first, create your program.py, then run mypy program.py.

#program.py (omitting the classes defined above)
pigeon = Pigeon()
def walk_the_duck(duck: Duck):
    duck.walk()
    duck.quack()
    duck.walk()
walk_the_duck(pigeon)

Mypy will check program.py and return an error:

error: Argument 1 to "walk_the_duck" has incompatible type "Pigeon"; expected "Duck"

This kind of validation increses the reliability in the code, avoiding errors, but also helps future developers to understand which data types are used, making the development more agile.

Mypy tricks and workarounds

This section will show some common problems and usage of mypy.

Type ignore

Sometimes it’s not worthy to fix a mypy error. You can always use #type: ignore comment to ignore the exception.

def foo() -> dict: 
    return 0 # type: ignore

The short example above will ignore that the function should return dict instead of an int.

Type any

Any type allow programmers to use any kind of type in the function which makes static checking less accurate but sometimes necessary. The next example explores a common use case of Any type.

from typing import Any

class MyObj:
    ...
    def __setattr__(self, name: str, value: Any):
        setattr(self, name, value)

Since MyObj is a generic object it can accept any type. For that reason it is common to use Any with this the __setattr__ method.

Note that in the code above the value property does not need to be specified as Any since mypy does this by default.

Incompatible types in assignment

Mypy inferes the variable type, consequently it will complain if you change the variable type. In the following example, mypy wouldn’t allow the variable test to be assigned to a str since the variable has type int.

test = "1"
test = 1

Mypy full error:

error: Incompatible types in assignment (expression has type "int", variable has type "str")

Local type inference

Sometimes mypy is unable to identify the type of some variable. For example, empty dictionaries like

global_dict = {}

will throws the following error:

Need type annotation for "global_dict" (hint: "global_dict: Dict[<type>, <type>] = ...")

In this cases you can specify a type for that variable using:

from typing import Dict
global_dict: Dict[str, str] = {}

Cannot assign to a method

Mypy doesn’t allow function overloading at runtime. In the example below, a on_btn_clicked function is overloaded and mypy complains.

from typing import Callable
    
class Button:
    def on_btn_clicked(self):
        pass
        
button = Button()
button.on_btn_clicked = lambda: print('clicked')

Mypy full error:

error: Cannot assign to a method

Conclusion

Python’s dynamic typing is one of its key features, but it can affect productivity when the codebase grows. Without type annotation, a basic understanding of the valid argument to a function or its return type becomes an issue.

Even if the whole code is documented in a docstring, a simple type change not updated on the docs can lead to a lot of errors, affecting code accuracy and creating imprecise codebases. Using mypy helps developers to keep track of the types and find subtle bugs (like forgetting to handle a None value).

Static checking can lead to a faster development environment and a stable codebase: - Most Integrated Development Enviroments (IDEs) use the static checking to improve auto-completion - Continuos Delivery (CI) can run automatic checks to investigate potential errors

Mypy was designed to be introduced slowly in any project, allowing programmers to work on legacy code while refactoring. Mypy usage can make your programs easier to understand, debug, and maintain. On the other hand, static type checks might give some illusion of safety. With all type hints in place and no analyzer errors, the application may still fall into type errors on runtime, for example when retrieving third party API data with an unexpected type.

Acknowledge

I would like to thank Immanuel Bayer, Oleksandr Pysarenko for their reviews and constructive suggestions. Their references and advices were very helpful in the development of this article.

References