Hi,
I wanted to share my experience with adding type hints to one of my projects.
TL;DR: There were some benefits, mypy discovered a couple of bugs, there are a
few
annoyances for me, and I am not really sold on it yet.
Please correct my mistakes or help me solve some problems if possible :)
(Sorry about the crappy formatting, I copied the md text verbatim from the same
post I
made on reddit. If you want to read all this and desire better layout you can
perhaps
read it there instead:
https://www.reddit.com/r/Python/comments/6d5haz/the_story_of_adding_type_hints_to_one_of_my/
)
Adding type hints to Tale
-
Intro
-
'Tale - an Interactive Fiction, MUD & mudlib framework' is
a medium sized Python project. At the time of writing (just after
releasing version 3.1 of the project),
it has about 15000 SLOC (~9500 in the tale library,
~3800 in the test suite, and ~2100 in the demo stories).
You can find it on Github; https://github.com/irmen/Tale
Tale started as a Python 2.x-and-3.x compatible library but I decided
I want to be able to use modern Python (3.5+) features and not worry
anymore about backwards compatibility, so I cleaned up and modernized the code
a while ago.
After finishing that, one of the major new Python features I still hadn't used
anywhere,
was _type hinting_. Because I wanted to learn to use it, I took Tale as
a test case and added type hints to all of the code.
I wanted to learn about the syntax,
how to apply it to an existing code base,
to see what benefits it gives,
and to discover what problems it introduces and what limitations it has.
Below are the results.
Benefits
The following things have proven to be beneficial to me:
- PyCharm (my IDE of choice) is giving more detailed warning messages and code
completion suggestions:
it adds inferred types or type hints to it.
- The MyPy tool (http://mypy.readthedocs.io/en/latest/index.html) is able to
statically
find type related errors in the code
at _compile time_ rather than having to run into them during _run time_.
It correctly identified several mistakes that were in the code that I had
not discovered yet and weren't caught by the unit tests.
- Mypy is pretty smart, I was quite surprised by its capability to infer
the type of things and how it propagates through the code.
- Type hinting is optional so you can mix code with and without type hints.
Things I'm not happy with
-
- It takes a lot of effort to add correct type hints everywhere.
Before this task is complete, mypy can and will report lots of
errors that are wrong or can be misleading.
- Mypy has bugs. One of the most obvious is that it doesn't know about
several modules and classes from the standard library.
- Shutting mypy up is done via a `# type: ignore` comment.
Tale now has about 60 of these...
Some of them can and should be removed once mypy gets smarter,
but I have no way of knowing which ones they are. How am I going to maintain
these?
It strongly reminds me of the 'suppress-warning' type of comments found
in other languages. This is problematic because it hides possible errors
in a way that is hard to find later.
- I really don't like these two major aspects of the type hint syntax:
1. Sometimes you have to use _strings_ rather than the type itself.
This is because the hints are parsed at the same time as all other code,
and sometimes you need "forward references" to types that are not yet
defined.
This can sometimes be fixed by rearranging the definition order
of your classes, but if classes reference each other (which is common)
this doesn't help. I find the most irritating that you have to
do this for the actual class that you're type hinting the methods of!
I understand the reason (the class is still being parsed, and is not
yet _defined_) but I find it very cumbersome.
2. Type hints for variables are often required to help mypy. Especially
if you're initializing names with empty collection types such as `[]` or
`{}`
which is common. Also you have to do this via a _comment_ such as `#
type:
List[str]`
The latter is improved in Python 3.6 (see PEP-526) but I want to
stick with 3.5 as a minimum for a while.
- Because type hints are parsed by Python itself, and because mypy
even parses the comments as well,
you'll have to import all types that you use in hints.
This causes a lot of extra imports in every module.
In my case this even led to some circular import problems that
were only fixable by changing the modules itself. (One can argue
that this is a good thing! Because circular references often
are a code smell)
Some types are only used in a _comment_, which
causes the IDE to warn me about unused imports that it wants to
clean up (but if I do this, it breaks mypy). PyCharm is not (yet)
smart enough to see that an import is really used
