RE: how to work on gearpump-runner branch

Hi Manu I would suggest to prepare a PR based on master. I would be more than happy to help you on this. Regards JB  Original message From: Manu Zhang Date: 27/07/2016 04:32 (GMT+01:00) To: dev@beam.incubator.apache.org Subject: how to work

[PROPOSAL] A brand new DoFn

Hi all, I have a major new feature to propose: the next generation of DoFn. It sounds a bit grandiose, but I think it is the best way to understand the proposal. This is strongly motivated by the design for state and timers, aka "per-key workflows". Since the two features are separable and have

[PROPOSAL] State and Timers for DoFn (aka per-key workflows)

Hi everyone, I would like to offer a proposal for a much-requested feature in Beam: Stateful processing in a DoFn. Please check out and comment on the proposal at this URL: https://s.apache.org/beam-state This proposal includes user-facing APIs for persistent state and timers. Together,

how to work on gearpump-runner branch

Hi All, As gearpump-runner has been merged into https://github.com/apache/incubator-beam/tree/gearpump-runner branch, I'd like to add ask about our working model here. 1. how (often) should be the branch synced with master and who will commit it ? 2. Do I work on new PRs based on branch or

Build failed in Jenkins: beam_Release_NightlySnapshot #114

See Changes: [klk] Tidy WriteWithShardingFactory [klk] Introduce StateInternalsFactory [klk] Port runners to use GroupAlsoByWindows via StateInternalsFactory [lcwik] Remove overrides of isStreaming() and getAppName() in

Re: [PROPOSAL] CoGBK as primitive transform instead of GBK

I created https://issues.apache.org/jira/browse/BEAM-490 which will track the work for swapping the primitive from being GBK to CoGBK. On Thu, Jul 21, 2016 at 9:24 AM, Lukasz Cwik wrote: > As of today, Cloud Dataflow will also be executed as a GBK. > > On Thu, Jul 21, 2016 at

Re: Beam/Flink : State access

I think your interested in https://issues.apache.org/jira/browse/BEAM-25 which is about exposing access to state in a runner independent way in the Beam model. You can watch that issue and comment on design / development as it is currently being worked on. On Tue, Jul 26, 2016 at 8:53 AM, Aparup

Re: Beam/Flink : State access

Hi, the purpose of Beam is to abstract the user from the underlying execution engine. IMHO, allowing access to state of the underlying execution engine will never be a goal for the Beam project. If you want/need to access Flink state, I think this is a good indicator that you should use Flink