Thanks Eugene. Will keep you cc'd. - Cham
On Tue, Mar 14, 2017 at 1:15 PM Eugene Kirpichov <[email protected]> wrote: > Thanks Cham! Agreed that this is a clean way to add SDF API to Python. > And agreed that it's a good idea to follow as similar path to Java - > direct runner first. The next easiest thing is going to be the various > streaming runners. > Please keep me cc'd on various PRs related to this. > > On Tue, Mar 14, 2017 at 12:46 PM Chamikara Jayalath <[email protected]> > wrote: > > Ccing Eugene and other authors of the original Splittable Dofn proposal. > > Thanks, > Cham > > On Mon, Mar 6, 2017 at 10:32 AM Chamikara Jayalath <[email protected]> > wrote: > > Thanks Ahmet. I created https://issues.apache.org/jira/browse/BEAM-1630 for > this. > > Any further comments are welcome :). > > Implementation wise, I think we should add Splittable DoFn to streaming > direct runner first, after [1] is finalized, and then follow it up with > support for bounded sources and other runners (currently Dataflow runner > and Dataflow/other runners through Fn API [2]). > > Thanks, > Cham > > [1] https://issues.apache.org/jira/browse/BEAM-1265 > [2] https://s.apache.org/beam-fn-api > > On Fri, Mar 3, 2017 at 5:34 PM Ahmet Altay <[email protected]> > wrote: > > +1 Thank you, this is a great and clean API proposal. > > Ahmet > > On Fri, Mar 3, 2017 at 5:16 PM, Chamikara Jayalath <[email protected]> > wrote: > > > Hi All, > > > > I've put together a document that proposes a Splittable DoFn API for > Python > > SDK. > > > > https://docs.google.com/document/d/1h_zprJrOilivK2xfvl4L42vaX4DMYGfH > > 1YDmi-s_ozM/edit?usp=sharing > > > > Splittable DoFn framework [1] is currently being implemented for Java SDK > > [2] and will unlock many use-cases that are not possible with the current > > BoundedSource framework [3] (see [1] for details). So, I believe, it will > > be good to add a similar framework to Python SDK as well. > > > > Please let me know what you think. > > > > Thanks, > > Cham > > > > [1] http://s.apache.org/splittable-do-fn > > [2] https://issues.apache.org/jira/browse/BEAM-65 > > [3] > > https://github.com/apache/beam/blob/master/sdks/python/ > > apache_beam/io/iobase.py > > > >
