Exploring the python type checking toolPosted by Daniel Moisset 11 months, 3 weeks ago Comments
Earlier this year PEP-484 was accepted, the
typing module was added to Python 3.5, and mypy moved into the umbrella of official python projects. Since it was a visible topic at the last Pycon.us, I decided to get some experience with it and see how it feels to use it.
I decided I’d take a working, mature, open-source project that wasn’t written by me and “convert” it to mypy. The questions I was trying to answer were:
- Does it help me discover actual bugs? (my guess was “probably not” being that this was an already mature tool, but I was also expecting to be surprised)
- Does the process of adding types help making the code base more understandable?
- How mature is mypy itself? Is it usable right now, does it have a lot of bugs?
- Is the type system flexible enough to express the kind of actual dynamic tricks that real developers like us use in actual, production python code? (that’s what motivated me to work on a already existing code base instead of writing a new one that would force me to “think in a statically-typed way” while building)
- Does it feel practical/usable?
- What other things that I didn’t expect can be learned from the experience?
The code sample that I chose for this is the pycodestyle tool (formerly known as “pep8”). It is relatively small, but it is also a popular tool. It was code that I’ve never seen before, but I’m familiar enough as a user with it to know what to expect inside.
This is a long post, so we’ll publish it in several parts.
- Part 1 (this one) gives a brief description about the tool, what it does, how to use it, and what was my project about
- Part 2 has a detailed report of my findings
- Part 3 has a summary and some conclusions
What is it?
For those unfamiliar with mypy, I’ll add a short introduction here. If you have already been following it, you can skip this section. For finer details read the official documentation which is quite good as a tutorial. My goal here is to give you a quick taste of what’s going on.
mypy is a type checking tool for python code, so it can detect inconsistencies in your code; a typical example would be adding a
str with an
int, but perhaps more realistic examples of errors we make are passing a file name to a function that actually expected a file object, or mistyping an attribute name of an object, or setting an attribute to an
int when it should be set to a function returning an
If you’re familiar with other statically typed languages like C or Java, there are some important differences:
- The type check phase is separate from the compiling/running phase. It’s something that you run whenever you like, probably before delivery and while developing. In that sense it feels more like running tests.
- Something that is partly a consequence of the above is that typing errors are not something you need to “fix” before running a program. You can run your Python program as always (or invoke them as from an interactive interpreter, or a test suite) even if they have type errors, which can be useful to debug a problem or if there is something that you know it looks like a type error but you know it’s a situation that can never happen.
- Using types is optional per function/method and you can choose how much or how little to use it. You can add type information to a few functions at a time, and only those will be checked. This make easier to apply stricter type rules to places of the code where it makes sense, and leave those parts where the cost/benefit of adding types is low unchecked.
- Even when adding types you can opt out at more granular levels. You can say that the type of a variable is Any which means “everything I do with this variable is OK, don’t check it”, or you can silence type errors in a single line.
In summary, it’s essentially an opt-in mechanism, designed that way to not interfere with the things that Python allows you to do and you like.
Another difference with some statically typed languages, but not all is that it can use types even where there aren’t type declarations. Local variable and object attribute types are inferred from context (with some exceptions), so you only specify types for function parameters and returns. It reduces the amount of work to use mypy (and the syntactic overhead), and function signatures are the place where type annotations make more sense also as a self-documenting property of the code.
To use it you can just install it with pip (you should do it inside a virtualenv/venv, which you probably are using anyway).
$ pip install mypy-lang
Note that you should install “mypy-lang”. There is a
mypy package in pip which is a completely unrelated library. Once you have that installed, you can run it on one or many python files:
$ mypy somescript.py $ mypy somescript.py somemodule.py $ mypy somepackage/*.py
Note that you use paths to python files, not import paths. You should run mypy from the same place that you run your code to make sure it finds imports in the same way that your code does.
Running it as it is won’t do much checking; it will ignore all the function bodies and only infer some types and check the main body of each module. You can get a few error messages for your module body, and also about imports it can’t find, or library modules that have no type information. When starting you probably don’t want a lot of warnings on imported modules so you can use the
-s command line flag to avoid following and checking imported modules.
The interesting stuff happens when you start adding type information to your functions. The type information is done with function annotations (those tags on parameters and results that exist since 3.0 times), following some conventions established in PEP484 which use the now standard
typing module (
typing is standard in Python 3.5 but if you’re using an older interpreter mypy provides its own backport). So you turn normal python function or method declarations like:
def trailing_blank_lines(physical_line, lines, line_number, total_lines):
from typing import List, Tuple SourcePosition = Tuple[int, int] CheckResult = Tuple[SourcePosition, str] def trailing_blank_lines(physical_line: str, lines: List[str], line_number: int, total_lines: int) -> CheckResult: # ...Function body goes here...
Some things to notice above are:
- You can use built-in types like
strin type declarations
- For container types you can use the
Tuplethat allow you to also specify element types. For example
Tuple[int, int]is the type of tuples with two integer components;
List[str]is the type of lists with string elements.
- You can create auxiliary definitions (like
CheckResult) to make declarations easier to read. I could have declared the above example as a function returning
Tuple[Tuple[int, int], str]but it’s harder to understand and less clear about purpose.
Once you’ve added annotations like those above, mypy can then detect many kinds of problems. A (non comprehensive) list of common errors that you can detect can be:
- If any code calls the above function with the wrong type/count/order of arguments, mypy will warn you about it.
- Inside the function body, misuses of the arguments (like doing
for i in range(physical_line)which is wrong because
physical_lineis a string) will be detected. Of course this ends up propagating to almost any value misuse inside the function, not necessarily an operation in the argument.
- The return types of the functions will be checked to match the declared type. If the function above has a
return 1, 2, "foo"instead of
return (1,2), "foo"you’ll get an error message after running mypy.
Working with libraries
The more function annotations you have, the more effective are the checks on function calls that mypy does. However, in most python code bases, a lot of your function calls will not be internal to your code base but to other library, possibly the python standard library or some other third party library you’re using.
In those cases it’s impractical and complicated to modify the third party code to add annotations. mypy provides an alternate mechanism that allows you to provide “stub files”, which are files with just a lot of type specifications for another module. Those files are written with python 3 syntax but with no function bodies (use
... instead), and saved with a
.pyi extension. Those are called typesheds
mypy provides typesheds for the most important (but not all) python standard library modules and some popular third party libraries. As its usage increases that coverage will grow, and it’s possible that module providers will start providing their own typesheds.
Working with legacy python
If you still have to support python 2.x, there are ways to do it adding the type annotations in specially formatted comments. The documentation covers it well and it’s not the scope of this article so I won’t cover the details.
With that general picture of what we’ll be seeing let’s start discussing my small project. I downloaded pycodestyle from its github repository. The pycodestyle tool is actually a single-file python script with about 2200 lines of code, 46 functions, 6 classes with 44 method definitions. There are other python files in the repo but related to installation and testing. It doesn’t depend on third party libraries, but uses many different components of the standard library (command-line parsing, regular expressions, the tokenizer, introspection via inspect)
After downloading the source, I created a virtualenv and installed mypy-lang with pip with no problems (at that point I used mypy 0.4.1; 0.4.2 is already available). Then I was able to run:
$ mypy pycodestyle.py
Which results in a few errors, but most of them uninteresting. The code is not annotated so mypy only checks the top level of the script. A lot of the errors are about imported library modules not having yet the “typesheds”, so you’ll see messages like:
pycodestyle.py:52: error: No library stub file for standard library module 'keyword' pycodestyle.py:60: error: No library stub file for standard library module 'optparse' ...
At this point it was quite reasonable to use the “silent” flag to avoid messages about external code:
$ mypy -s pycodestyle.py
If you are doing this on a large project you can later remove the
-s flag once you have some more typing information if you like, to make sure you haven’t missed large parts.
At this point I started iterating by doing different things:
- Fixing the errors that mypy was already detecting
- Finding some function definition(
def foo(...)) and adding types to it, which in turn helped mypy find more inconsistencies and report more issues.
After each of these changes, I re-ran mypy to make sure that I was moving in the right direction. Also, when I wanted to make sure that I wasn’t missing some function definitions that should be typed (or forgot to type one of the parameters) I used:
$ mypy -s --disallow-untyped-defs pycodestyle.py
which reported stuff like:
pycodestyle.py: note: In function "tabs_or_spaces": pycodestyle.py:134: error: Function is missing a type annotation pycodestyle.py: note: In function "tabs_obsolete": pycodestyle.py:153: error: Function is missing a type annotation
This is the end of Part 1 of this article. On the next one I’ll go through some of examples of where this process led me through, and the insight I gained regarding the tool. Finally, Part 3 will go for some general conclusions. See you in my next post !