The herculean term is perfect to describe this impressive achievement Chad.
Congratulations and thanks for the effort to make this happen. This will give Beam users not only improved functionality but as Robert mentioned help others to understand more quickly the internals of the python SDK. Maintenance is a small price to pay for the big wins of typing. Huge +1 On Wed, Oct 30, 2019 at 9:26 PM Chad Dombrova <chad...@gmail.com> wrote: > > >> >> Do you believe that a future mypy plugin could replace pipeline type checks >> in Beam, or are there limits to what it can do? > > > mypy will get us quite far on its own once we completely annotate the beam > code. That said, my PR does not include my efforts to turn PTransforms into > Generics, which will be required to properly analyze pipelines, so there's > still a lot more work to do. I've experimented with a mypy plugin to smooth > over some of the rough spots in that workflow and I will just say that the > mypy API has a very steep learning curve. > > Another thing to note: mypy is very explicit about function annotations. It > does not do the "implicit" inference that Beam does, such as automatically > detecting function return types. I think it should be possible to do a lot > of that as a mypy plugin, and in fact, since it has little to do with Beam it > could grow into its own project with outside contributors. > > -chad >
