Re: IllegalStateException when combining 3 streams?

Yes, indeed. It works just fine with equally define triggers in global window. Sorry about that and thank you so much. I can now leav the valley of despair and continue on my learning path;) Best Regards /Artur On Sat, Nov 4, 2017 at 10:40 PM, Aleksandr wrote: >

Re: IllegalStateException when combining 3 streams?

And the error message: Exception in thread "main" java.lang.IllegalStateException: Inputs to Flatten had incompatible triggers: Repeatedly.forever(AfterProcessingTime.pastFirstElementInPane().plusDelayOf(10 seconds)), AfterProcessingTime.pastFirstElementInPane().plusDelayOf(10 seconds) at

Re: IllegalStateException when combining 3 streams?

Sure, here is the url to my repo https://github.com/afuyo/beamStuff/blob/master/src/main/java/com/tryg/beam/kafka/poc/impl/CustomerStreamPipelineGlobal.java It is fairly simple and I do no grouping before. Just read from Kafka and then join. I followed your advice and the 3 way join works for

Re: IllegalStateException when combining 3 streams?

Can you share the program using global windows and make sure the exception is the same? Basically, this kind of problem has to do with whether you have applied a grouping operation already. The triggering (and sometimes windowing) before a grouping operation is different than after. So, if you are

Re: IllegalStateException when combining 3 streams?

Hej Ben, thank you for your answer. I think I forgot to mention that the join class already implements CoGroupByKey and comes from sdk extensions. I haven now modified it slightly to do 3 way join(see below). It works for 3 PCollections with fixed windows and provides the output this time, even

Re: IllegalStateException when combining 3 streams?

It looks like this is a problematic interaction between the 2-layer join and the (unfortunately complicated) continuation triggering. Specifically, the triggering of Join(A, B) has the continuation trigger, so it isn't possible to join that with C. Instead of trying to do Join(Join(A, B), C),

Re: IllegalStateException when combining 3 streams?

Hi Kenneth and Aleksandr and thank you for your prompt answer. So there are two scenarios that I've been trying out but operation is always the same, join setA+setB =>setAB, and then join setAB+setC=>setABC. In scenario 1 I define three PCollections with global widnows and exactly the same

Re: IllegalStateException when combining 3 streams?

Hi Artur, When you join the PCollections, they will be flattened and go through a GroupByKey together. Since the trigger governs when the GroupByKey can emit output, the triggers have to be equal or the GroupByKey doesn't have a clear guide as to when it should output. If you can make the

Re: IllegalStateException when combining 3 streams?

Hello, You can try put PCollection after flatten into same global window with triggers as it was before flattening. Best regards Aleksandr Gortujev 3. nov 2017 11:04 AM kirjutas kuupäeval "Artur Mrozowski" : Hi, I am on second week of our PoC with Beam and I am really amazed