+1. Thanks!
On Wed, Dec 11, 2019 at 1:44 PM Pablo Estrada <pabl...@google.com> wrote: > > Love it. I've merged it, since it was already approved by Robert - and yes, > we don't want to hit a merge conflict. > > On Wed, Dec 11, 2019 at 1:36 PM Heejong Lee <heej...@google.com> wrote: >> >> Wow, this is a big step forward. As a long fan of strongly typed functional >> languages, I'm glad to see this change :) >> >> On Wed, Dec 11, 2019 at 9:44 AM Chad Dombrova <chad...@gmail.com> wrote: >>> >>> Hi all, >>> Robert has diligently reviewed the first batch of changes for this PR, and >>> all review notes are addressed and tests are passing: >>> https://github.com/apache/beam/pull/9915 >>> >>> Due to the number of file touched there's a short window of about one or >>> two days before a merge conflict arrives on master, and after resolving >>> that it usually takes another 1-2 days of pasting "Run Python PreCommit" >>> until they pass again, so it would be great to get this merged while the >>> window is open! Despite the number of files touched, the changes are >>> almost entirely type comments, so the PR is designed to be quite safe. >>> >>> -chad >>> >>> >>> On Tue, Nov 5, 2019 at 2:50 PM Chad Dombrova <chad...@gmail.com> wrote: >>>> >>>> Glad to hear we have such a forward-thinking community! >>>> >>>> >>>> On Tue, Nov 5, 2019 at 2:43 PM Robert Bradshaw <rober...@google.com> wrote: >>>>> >>>>> Sounds like we have consensus. Let's move forward. I'll follow up with >>>>> the discussions on the PRs themselves. >>>>> >>>>> On Wed, Oct 30, 2019 at 2:38 PM Robert Bradshaw <rober...@google.com> >>>>> wrote: >>>>> > >>>>> > On Wed, Oct 30, 2019 at 1:26 PM Chad Dombrova <chad...@gmail.com> wrote: >>>>> > > >>>>> > >> Do you believe that a future mypy plugin could replace pipeline type >>>>> > >> checks in Beam, or are there limits to what it can do? >>>>> > > >>>>> > > mypy will get us quite far on its own once we completely annotate the >>>>> > > beam code. That said, my PR does not include my efforts to turn >>>>> > > PTransforms into Generics, which will be required to properly analyze >>>>> > > pipelines, so there's still a lot more work to do. I've experimented >>>>> > > with a mypy plugin to smooth over some of the rough spots in that >>>>> > > workflow and I will just say that the mypy API has a very steep >>>>> > > learning curve. >>>>> > > >>>>> > > Another thing to note: mypy is very explicit about function >>>>> > > annotations. It does not do the "implicit" inference that Beam does, >>>>> > > such as automatically detecting function return types. I think it >>>>> > > should be possible to do a lot of that as a mypy plugin, and in fact, >>>>> > > since it has little to do with Beam it could grow into its own >>>>> > > project with outside contributors. >>>>> > >>>>> > Yeah, I don't think, as is, it can replace what we do, but with >>>>> > plugins I think it could possibly come closer. Certainly there is >>>>> > information that is only available at runtime (e.g. reading from a >>>>> > database or avro/parquet file could provide the schema which can be >>>>> > used for downstream checking) which may limit the ability to do >>>>> > everything statically (even Beam Java is moving this direction). Mypy >>>>> > clearly has an implementation of the "is compatible with" operator >>>>> > that I would love to borrow, but unfortunately it's not (easily?) >>>>> > exposed. >>>>> > >>>>> > That being said, we should leverage what we can for pipeline >>>>> > authoring, and it'll be a great development too regardless.
