Re: Aggregate-and-join in Beam

2020-02-14 Thread Paweł Kordek
ead around it. The important part is that the emitted value's aggregate takes into account current element's value itself. ____ From: Paweł Kordek Sent: Friday, February 14, 2020 10:53 To: user@beam.apache.org Subject: Aggregate-and-join in Beam Hello I am working on a relativ

Fwd: Running a Beam Pipeline on GCP Dataproc Flink Cluster

2020-02-08 Thread Paweł Kordek
From: Paweł Kordek Sent: Saturday, February 8, 2020, 08:48 To: Xander Song Subject: Re: Running a Beam Pipeline on GCP Dataproc Flink Cluster It's because 1.9.1 is no longer hosted under this particular link you can change to 1.9.2. BTW this link is just

Re: Running a Beam Pipeline on GCP Dataproc Flink Cluster

2020-02-07 Thread Paweł Kordek
Hi I had similar use-case recently, and adding a metadata key solved the issue https://github.com/GoogleCloudDataproc/initialization-actions/pull/334. You keep the original initialization action and add for example (using gcloud) '--metadata

Re: Python errors when using batch+windows+textio

2019-09-16 Thread Paweł Kordek
r the record, this looks like a known issue: > https://issues.apache.org/jira/browse/BEAM-6860 > > Kyle Weaver | Software Engineer | github.com/ibzib | kcwea...@google.com > > > On Wed, Sep 11, 2019 at 6:33 AM Paweł Kordek > wrote: > >> Hi >> >> I was develop

Python errors when using batch+windows+textio

2019-09-11 Thread Paweł Kordek
Hi I was developing a simple pipeline where I aggregate records by key and sum values for a predefined window. I was getting some errors, and after checking, I am getting exactly the same issues when running Wikipedia example from the Beam repo. The output is as follows: