Splittable DoFn for Python SDK

2017-03-03 Thread Chamikara Jayalath
Hi All, I've put together a document that proposes a Splittable DoFn API for Python SDK. https://docs.google.com/document/d/1h_zprJrOilivK2xfvl4L42vaX4DMYGfH1YDmi-s_ozM/edit?usp=sharing Splittable DoFn framework [1] is currently being implemented for Java SDK [2] and will unlock many use-cases

Re: Release 0.6.0

2017-03-03 Thread Kenneth Knowles
Hi all, BEAM-1619 is resolved. BEAM-1611 is in PR and I'll update again when it is confirmed and merged. Kenn On Fri, Mar 3, 2017 at 10:47 AM, Thomas Groh wrote: > Hey everyone; > > The submission of Surgery for the Dataflow Runner in the Java SDK has > broken all

Re: Introduction

2017-03-03 Thread Kenneth Knowles
Welcome Chinmay! Just in case you don't already know the whole layout, the JIRA component for the Apex runner is https://issues.apache.org/jira/browse/BEAM/component/12331007/ Kenn On Fri, Mar 3, 2017 at 6:53 AM, Chinmay Kolhatkar wrote: > Hi JB, > > Thanks for the

Re: Let's make Beam transforms comply with PTransform Style Guide

2017-03-03 Thread Eugene Kirpichov
It sounds like a great idea in general, though there are actually not that many issues remaining - I fixed a bunch, and have PRs out for two more (ParDo and BigQueryIO). The remaining ones are: [image: Bug - A problem which impairs or prevents the functions of the product.] BEAM-1355

Re: Let's make Beam transforms comply with PTransform Style Guide

2017-03-03 Thread Robert Bradshaw
Here's a crazy idea: what if we had a virtual fixit/hackathon to knock these out (similar to the virtual meet-up, but with an agenda)? I find communal hacking sessions towards a common goal are a good way to get to know each other and get a lot done. Would there be any interest in this? On Wed,