Re: [KUDOS] Contributed runner: Gearpump!

Awesome! On Wed, Jul 20, 2016 at 6:42 PM, Manu Zhang wrote: > Thanks Kenn and others for the review and help along the way. Feel free to > ping me on slack if you want to know more about gearpump-runner or > gearpump. > > Thanks everyone :) > Manu Zhang > > > > On Thu,

[KUDOS] Contributed runner: Gearpump!

Hi all, I would like to call attention to a huge contribution to Beam: a runner for Apache Gearpump (incubating). The runner landed on the gearpump-runner feature branch today. Check it out! And contribute towards getting it ready for the master branch :-) Please join me in special

[Proposal] Add waitToFinish(), cancel(), waitToRunning() to PipelineResult.

Hi everyone, Here is a proposal to address the following issue: JIRA issue https://issues.apache.org/jira/browse/BEAM-443 Currently, users doesn’t have a consistent way to wait for the pipeline to finish. Different runners have different implementations. For example: 1. DirectRunner have a

Re: [PROPOSAL] CoGBK as primitive transform instead of GBK

+1 I assume that the intent is for the semantics of both GBK and CoGBK to be unchanged, just swapping their status as primitives. This seems like a good change, with strictly positive impact on users and SDK authors, with only an extremely minor burden (doing an insertion of the provided

[PROPOSAL] CoGBK as primitive transform instead of GBK

I would like to propose a change to Beam to make CoGBK the basis for grouping instead of GBK. The idea behind this proposal is that CoGBK is a more powerful operator then GBK allowing for two key benefits: 1) SDKs are simplified: transforming a CoGBK into a GBK is trivial while the reverse is

Re: Clojure SDK

hey Ted, Awesome -- glad to hear it and welcome! Looking forward to working with you (and learning about Clojure in the process). Just to verify some definitions: are you planning on implementing an entirely new SDK from scratch, or are you planning on writing a Clojure wrapper for the existing

Re: Adding DoFn Setup and Teardown methods

+1 I think this will prove itself even more important with Per-Key workflows where DoFn's might communicate constantly with external services - DB, Elasticsearch, etc. On Mon, Jul 18, 2016 at 5:44 PM Maximilian Michels wrote: > Hoping it becomes usual as soon as we have this