Re: [PROPOSAL]: a new feature branch for SQL DSL

Expose streaming snapshot via STATE is attractive in Beam model, but doubt it's the right way in SQL. IMO,there's 'INSERT INTO' to persistent streaming output. On Wed, Apr 12, 2017 at 8:37 PM, tarush grover wrote: > Hi Tyler, > > Transforming stream into a table will

Re: [PROPOSAL]: a new feature branch for SQL DSL

Hi Tyler, Transforming stream into a table will also depend on the time frame in the stream or what windows we choose for the stream. Regards, Tarush On Tue, 11 Apr 2017 at 11:29 PM, Tyler Akidau wrote: > Hi 陈竞, > > I'm doubtful there will be an explicit

Re: JStorm runner

Also, JIRA component created [1], with Pei as component lead. Davor [1] https://issues.apache.org/jira/browse/BEAM/component/12332477/?selectedTab=com.atlassian.jira.jira-projects-plugin:component-summary-panel On Thu, Apr 6, 2017 at 7:55 PM, Kenneth Knowles wrote: >

Re: Combine.Global

Ah, I found my mistake. You overrode the getAccumulator and getDefaultOutputCoders which my implementation did not. This approach is straight forward now. Thanks! On 2017-04-07 23:46 (-0500), Aviem Zur wrote: > I wasn't able to reproduce the issue you're experiencing. >

Re: Apache Storm/JStorm Runner(s) for Apache Beam

This is a great discussion; thanks everyone. >From my perspective, the functionality to execute pipelines on both Storm and JStorm is very welcome and a big step forward for Beam. I'm not an expert on the Storm/JStorm differences, but one vs. two runners discussion sounds more like a packaging /

Re: Renaming SideOutput

This question maybe obvious to others but why is there a distinction between main output and additional outputs? Why not just have a simple list of outputs where the first one is the Main one. -- AC Sent from my iPhone > On Apr 12, 2017, at 18:08, Melissa Pashniak

Re: Renaming SideOutput

I agree, I'll create a PR with the doc changes (the rename + text changes to make things more clear). I know of at least 2 places we refer to side outputs (programming guide and the "Design your pipeline" page). On Tue, Apr 11, 2017 at 5:34 PM, Thomas Groh wrote: > I

Re: Python build artifacts seem to be misconfigured

This is also root cause for the flakiness in test_using_slow_impl very flaky locally tests (https://issues.apache.org/jira/browse/BEAM-1910). Kenn, have you found anything that might explain why tox is not deleting them? Ahmet On Tue, Apr 11, 2017 at 11:50 AM, Robert Bradshaw <

Re: [DISCUSSION] Consistent use of loggers

For examples (which I think is auto-propagated to examples archetype), and I think also manually done for starter archetype: * Every runner, including DirectRunner, is in a profile: -Pdirect-runner: https://github.com/apache/beam/blob/master/examples/java/pom.xml#L43 * The slf4j-jdk14 is already

Re: Renaming SideOutput

Cool! I've filed https://issues.apache.org/jira/browse/BEAM-1949 and authored https://github.com/apache/beam/pull/2512 to make this change. On Tue, Apr 11, 2017 at 11:33 PM, Ted Yu wrote: > +1 > > > On Apr 11, 2017, at 5:34 PM, Thomas Groh >

Re: [DISCUSSION] PAssert success/failure count validation for all runners

That sounds very good! Now we only have to manage to get this in before the first stable release because I think this is a very important signal for ensuring Runner correctness. @Pablo Do you already have plans regarding 3., i.e. stable URNs for the assertions. And also for verifying them in a

Re: Low availability on my end in the coming 3 weeks

Thanks for the update and your trust Amit ! Safe travel and take your time to move to your new home ;) Regards JB On 04/12/2017 03:23 PM, Amit Sela wrote: Hi everyone, I will be traveling and moving in the next ~3 weeks so I will be less available than usual. I believe our dev community is

Low availability on my end in the coming 3 weeks

Hi everyone, I will be traveling and moving in the next ~3 weeks so I will be less available than usual. I believe our dev community is more mature now so my absence won't be noticed :-) but I still asked JB to formally take my place with anything concerning the Spark runner and he kindly agreed.

Re: Renaming SideOutput

+1 > On Apr 11, 2017, at 5:34 PM, Thomas Groh wrote: > > I think that's a good idea. I would call the outputs of a ParDo the "Main > Output" and "Additional Outputs" - it seems like an easy way to make it > clear that there's one output that is always expected, and