Re: PreAggregate operator with timeout trigger

Yes you are right. Good to hear that you have solved your issue :) Piotrek > On 5 Nov 2019, at 09:56, Felipe Gutierrez > wrote: > > Thanks Piotr, > > the thing is that I am on Stream data and not on keyed stream data. So, I > cannot use the TimerService concept here. I am triggering a local

Re: PreAggregate operator with timeout trigger

Ah, yep. I do create a keyed stream which does not partition data. And I pre-aggregate key-values inside my operator. But I cannot rely on the number of keys to pre-aggregate because I can never receive some specific number of keys. So, the master concept to pre-aggregate is the time. After that, I

Re: PreAggregate operator with timeout trigger

@Gyula, I am afraid I haven't got your point. *--* *-- Felipe Gutierrez* *-- skype: felipe.o.gutierrez* *--* *https://felipeogutierrez.blogspot.com * On Tue, Nov 5, 2019 at 10:11 AM Gyula Fóra wrote: > You might have to introduce some dummy keys for a mor

Re: PreAggregate operator with timeout trigger

You might have to introduce some dummy keys for a more robust solution that integrates with the fault-tolerance mechanism. Gyula On Tue, Nov 5, 2019 at 9:57 AM Felipe Gutierrez < felipe.o.gutier...@gmail.com> wrote: > Thanks Piotr, > > the thing is that I am on Stream data and not on keyed strea

Re: PreAggregate operator with timeout trigger

Thanks Piotr, the thing is that I am on Stream data and not on keyed stream data. So, I cannot use the TimerService concept here. I am triggering a local window. I ended up using java.util.Timer [1] and it seems to suffice my requirements. [1] https://docs.oracle.com/javase/7/docs/api/java/util/T

Re: PreAggregate operator with timeout trigger

Hi, If you want to register a processing/event time trigger in your custom operator, you can take a look how other operators are doing it, by calling AbstractStreamOperator#getInternalTimerService [1]. You can look around the Flink’s code base for usages of this method, there are at least couple

PreAggregate operator with timeout trigger

Hi all, I have my own stream operator which trigger an aggregation based on the number of items received (OneInputStreamOperator#processElement(StreamRecord)). However, it is possible to not trigger my aggregation if my operator does not receive the max items that have been set. So, I need a timeo