Slack inivitation

2017-10-09 Thread Artur Mrozowski
Hej, could you please invite me to the slack channel? Thanks in advance /Artur

Re: Beam support on Spark 2.x

2017-11-12 Thread Artur Mrozowski
or just upgrade to Spark 2.x and drop support for Spark 1.x. > > You can take a look on the beam-samples: they all run using the Spark > runner. > > Regards > JB > > > On 11/10/2017 01:46 PM, Artur Mrozowski wrote: > >> Hi, >> I have seen the compatibil

Re: Beam support on Spark 2.x

2017-11-13 Thread Artur Mrozowski
o run on Spark 1.6 using Spark runner provided in > Beam 2.1.0. > > For Spark 2.0, you will have to wait Spark runner that will be provided in > Beam 2.3.0. > > Regards > JB > > On 11/13/2017 06:50 AM, Artur Mrozowski wrote: > >> Hi Jean-Baptiste, >> that

IllegalStateException when combining 3 streams?

2017-11-03 Thread Artur Mrozowski
Hi, I am on second week of our PoC with Beam and I am really amazed by the capabilities of the framework and how well engineered it is. Amazed does not mean experienced so please bear with me. What we try to achieve is to join several streams using windowing and triggers. And that is where I

Re: IllegalStateException when combining 3 streams?

2017-11-04 Thread Artur Mrozowski
lobal.main(CustomerStreamPipelineGlobal.java:220) On Sat, Nov 4, 2017 at 6:36 PM, Artur Mrozowski <art...@gmail.com> wrote: > Sure, here is the url to my repo https://github.com/afuyo/ > beamStuff/blob/master/src/main/java/com/tryg/beam/kafka/poc/impl/ > CustomerStreamPipelineGlobal.java > &g

Re: IllegalStateException when combining 3 streams?

2017-11-04 Thread Artur Mrozowski
bal windows and make sure the exception > is the same? Basically, this kind of problem has to do with whether you > have applied a grouping operation already. The triggering (and sometimes > windowing) before a grouping operation is different than after. So, if you > are having problem

Re: IllegalStateException when combining 3 streams?

2017-11-04 Thread Artur Mrozowski
wrote: > Hello, > You are using for policyInput Repeatedly.forever, for customerInput > Repeatedly.forever and for claimInput your trigger is without > Repeatedly.forever. > > > Best regards > Aleksandr Gortujev > > > сб, 4 нояб. 2017 г. в 20:39, Artur Mrozowski <art

Re: IllegalStateException when combining 3 streams?

2017-11-03 Thread Artur Mrozowski
going in to the join. > > Kenn > > On Fri, Nov 3, 2017 at 9:48 AM, Aleksandr <aleksandr...@gmail.com> wrote: > >> Hello, >> You can try put PCollection after flatten into same global window with >> triggers as it was before flattening. >> >> Best regards

Re: IllegalStateException when combining 3 streams?

2017-11-04 Thread Artur Mrozowski
t, because you are performing a > single, three-way join instead of two, two-way joins. > > https://beam.apache.org/documentation/sdks/javadoc/2. > 1.0/org/apache/beam/sdk/transforms/join/CoGroupByKey.html for more > information. > > -- Ben > > On Fri, Nov 3, 2017 at 2