Re: Python Static Typing: Next Steps

2020-05-07 Thread Ismaël Mejía
Awesome! Congrats to all involved in making the project achieve such milestone! On Thu, May 7, 2020 at 7:13 PM Luke Cwik wrote: > > Sweet. > > On Wed, May 6, 2020 at 5:05 PM Robert Bradshaw wrote: >> >> Just an update on this: we just merged >> https://github.com/apache/beam/pull/11620 which

Re: Python Static Typing: Next Steps

2020-05-07 Thread Luke Cwik
Sweet. On Wed, May 6, 2020 at 5:05 PM Robert Bradshaw wrote: > Just an update on this: we just merged > https://github.com/apache/beam/pull/11620 which enforces typechecking > for all files that currently pass. > > On Tue, Mar 3, 2020 at 1:12 PM Chad Dombrova wrote: > >> > >> This probably

Re: Python Static Typing: Next Steps

2020-05-06 Thread Robert Bradshaw
Just an update on this: we just merged https://github.com/apache/beam/pull/11620 which enforces typechecking for all files that currently pass. On Tue, Mar 3, 2020 at 1:12 PM Chad Dombrova wrote: >> >> This probably does not apply yet, does optional mean that opting-in for all >> or none of the

Re: Python Static Typing: Next Steps

2020-03-03 Thread Chad Dombrova
> > This probably does not apply yet, does optional mean that opting-in for > all or none of the precommit hooks? Would contributors have a choice of > which pre-commits they can opt-in too? > Yes, once the hook is installed, individual checks are opt-out and the design is clearly built around a

Re: Python Static Typing: Next Steps

2020-03-03 Thread Ahmet Altay
On Tue, Mar 3, 2020 at 11:53 AM Chad Dombrova wrote: > >>> +1. Also would not running dmypy would require all contributors to run a >> process all the time? I do not know if this is desired by existing >> contributors, and I am not sure if it will be friendly to new contributors. >> > >

Re: Python Static Typing: Next Steps

2020-03-03 Thread Chad Dombrova
> > >> +1. Also would not running dmypy would require all contributors to run a > process all the time? I do not know if this is desired by existing > contributors, and I am not sure if it will be friendly to new contributors. > Pre-commit git hooks are completely opt-in. Developers who choose

Re: Python Static Typing: Next Steps

2020-03-03 Thread Ahmet Altay
On Tue, Mar 3, 2020 at 11:10 AM Robert Bradshaw wrote: > Nice. > > I still think we should enable it in jenkins and get some experience > with it before we add it to the default pre-commit-hook territory. > +1. Also would not running dmypy would require all contributors to run a process all the

Re: Python Static Typing: Next Steps

2020-03-03 Thread Robert Bradshaw
Nice. I still think we should enable it in jenkins and get some experience with it before we add it to the default pre-commit-hook territory. On Tue, Mar 3, 2020 at 10:38 AM Chad Dombrova wrote: > > I tested out dmypy (the mypy daemon) last night and it was completing in > under a second after

Re: Python Static Typing: Next Steps

2020-03-03 Thread Chad Dombrova
I tested out dmypy (the mypy daemon) last night and it was completing in under a second after editing a file and rerunning (usually around 0.6s), which puts it into pre-commit-hook territory. -chad On Tue, Mar 3, 2020 at 1:54 AM Ismaël Mejía wrote: > +1 to do it by default. Great to see the

Re: Python Static Typing: Next Steps

2020-03-03 Thread Ismaël Mejía
+1 to do it by default. Great to see the typing work arrive to this maturity milestone. We can also refer to some of the mypy typing docs for newbies on the subject. On Tue, Mar 3, 2020 at 10:15 AM Kamil Wasilewski wrote: > > +1 for enabling mypy as a precommit job > > This however could be a

Re: Python Static Typing: Next Steps

2020-03-03 Thread Kamil Wasilewski
+1 for enabling mypy as a precommit job This however could be a good occasion to rework the current PythonLint job. Since yapf has been introduced, some of the checks made by pylint/flake are now unnecessary and could be dismantled. This would speed-up PythonLint quite a lot. I volunteer to help

Re: Python Static Typing: Next Steps

2020-03-02 Thread Robert Bradshaw
It seems people are conflating git pre-commit hooks (which IMHO should ideally be in the sub-second range, and run when an author does "git commit") with jenkins pre-commit tests (for which minutes is nothing compared to what we already do). I am +1 to adding mypy to the latter for sure, and think

Re: Python Static Typing: Next Steps

2020-03-02 Thread Udi Meiri
Off-topic: Python lint via pre-commit should be much faster. (I wrote my own modified-file-only lint in the past) On Mon, Mar 2, 2020 at 2:08 PM Kyle Weaver wrote: > > Python lint takes 4-5mins to complete. I think if the mypy analysis is > really on the order of 10s, the additional time won't

Re: Python Static Typing: Next Steps

2020-03-02 Thread Kyle Weaver
> Python lint takes 4-5mins to complete. I think if the mypy analysis is really on the order of 10s, the additional time won't matter and could always be enabled. +1 of course it would be nice to make mypy as fast as possible, but I don't think speed needs to be a blocker. The productivity gains

Re: Python Static Typing: Next Steps

2020-03-02 Thread Luke Cwik
Python lint takes 4-5mins to complete. I think if the mypy analysis is really on the order of 10s, the additional time won't matter and could always be enabled. On Mon, Mar 2, 2020 at 1:21 PM Chad Dombrova wrote: > I believe that mypy via pre-commit hook will be faster than 10s since it >> only

Re: Python Static Typing: Next Steps

2020-03-02 Thread Chad Dombrova
> > I believe that mypy via pre-commit hook will be faster than 10s since it > only applies to modified files. > Correct, with a few caveats: - pre-commit can be setup to only run if a python file changes. so modifying a java file won't trigger mypy to run. - if *any* python file

Re: Python Static Typing: Next Steps

2020-03-02 Thread Udi Meiri
Let's go forward with this and see. I volunteer to help as well. I believe that mypy via pre-commit hook will be faster than 10s since it only applies to modified files. On Mon, Mar 2, 2020 at 10:53 AM Robert Bradshaw wrote: > +1 > > We should enable this on jenkins, plus trivial instructions

Re: Python Static Typing: Next Steps

2020-03-02 Thread Robert Bradshaw
+1 We should enable this on jenkins, plus trivial instructions (ideally a one-liner tox command) to run it locally. Hopefully the errors will be easy enough for contributors to figure out (in particular local to and commensurate in complexity with the code that they're editing), and I agree it's

Re: Python Static Typing: Next Steps

2020-03-02 Thread Luke Cwik
+1 The typing information has really helped me several times figuring out that API contracts and expected types. On Mon, Mar 2, 2020 at 9:54 AM Pablo Estrada wrote: > I am in favor of enabling the test, and also am happy to start answering > questions too. > Thanks so much Chad for leading

Re: Python Static Typing: Next Steps

2020-03-02 Thread Pablo Estrada
I am in favor of enabling the test, and also am happy to start answering questions too. Thanks so much Chad for leading this. Best -P. On Mon, Mar 2, 2020 at 9:44 AM Chad Dombrova wrote: > Good news everyone! > We nearly have the full beam codebase passing in mypy. > > As we are now approaching

Python Static Typing: Next Steps

2020-03-02 Thread Chad Dombrova
Good news everyone! We nearly have the full beam codebase passing in mypy. As we are now approaching the zero-error event horizon, I'd like to open up a discussion around enabling mypy in the PythonLint job. Every day or so a PR is merged that introduces some new mypy errors, so enabling this