Re: continous time triger

[Brian Chhun]

Hi Aljoscha,

No problem with the change. I think it's more what a user would expect as
well.

On Wed, Jan 27, 2016 at 3:27 AM, Aljoscha Krettek <aljos...@apache.org>
wrote:

> Hi Brian,
> you are right about changing the behavior of windows when closing. Would
> this be a problem for you?
>
> Cheers,
> Aljoscha
> > On 26 Jan 2016, at 17:53, Radu Tudoran <radu.tudo...@huawei.com> wrote:
> >
> > Hi,
> >
> > Thank you for sharing your experience and also to Till for the advice.
> > What I would like to do is to be able to fire the window potentially
> multiple times, even if an event did not arrive. I will look more about how
> dealing with the processing time could help in this
> >
> > Dr. Radu Tudoran
> > Research Engineer - Big Data Expert
> > IT R Division
> >
> > 
> > HUAWEI TECHNOLOGIES Duesseldorf GmbH
> > European Research Center
> > Riesstrasse 25, 80992 München
> >
> > E-mail: radu.tudo...@huawei.com
> > Mobile: +49 15209084330
> > Telephone: +49 891588344173
> >
> > HUAWEI TECHNOLOGIES Duesseldorf GmbH
> > Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com
> > Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
> > Managing Director: Bo PENG, Wanzhou MENG, Lifang CHEN
> > Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
> > Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN
> > This e-mail and its attachments contain confidential information from
> HUAWEI, which is intended only for the person or entity whose address is
> listed above. Any use of the information contained herein in any way
> (including, but not limited to, total or partial disclosure, reproduction,
> or dissemination) by persons other than the intended recipient(s) is
> prohibited. If you receive this e-mail in error, please notify the sender
> by phone or email immediately and delete it!
> >
> > From: Brian Chhun [mailto:brian.ch...@getbraintree.com]
> > Sent: Tuesday, January 26, 2016 5:28 PM
> > To: user@flink.apache.org
> > Subject: Re: continous time triger
> >
> > For what it's worth, we have a trigger that fires once a day for a
> recurring calculation. When an element comes in, we set the trigger
> context's processing time timer to the exact millisecond of the desired
> time. The predefined triggers were useful to look at to achieve this:
> https://github.com/apache/flink/tree/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/triggers
> >
> > Some things I discovered along the way, particularly using processing
> time, which may be useful:
> > - registering a time that's already passed will cause the timer callback
> to be called
> > - when the system shuts down, the window is fired even though the
> trigger has not gone off (this sounds subject to change though)
> >
> > On Tue, Jan 26, 2016 at 3:47 AM, Till Rohrmann <trohrm...@apache.org>
> wrote:
> > Hi Radu,
> >
> > you can register processing and event time time triggers using the
> TriggerContext which is given to the onElement, onProcessingTime and
> onEventTime methods of Trigger. In case you register a processing time
> timer, the onProcessingTime method will be called once the system clock has
> passed the timer time. In case of an event time timer, the
> onEventTimemethod is called once a watermark has been received which has a
> higher watermark than the timer.
> >
> > I hope this helps you to solve your problem.
> >
> > Cheers,
> > Till
> >
> > ​
> >
> > On Mon, Jan 25, 2016 at 9:25 PM, Radu Tudoran <radu.tudo...@huawei.com>
> wrote:
> > Re-Hi,
> >
> > I have another question regarding the triggering of the processing of a
> window. Can this be done in some way at specific time intervals,
> independent of whether  an event has been received or not, via a trigger?
> >
> > The reason why I am considering a trigger rather than timeWindow(All) is
> that timeWindow will end up generating multiple windows and duplicating
> data, while having the option from the trigger to actually fire the
> processing at certain times, independent of when the events arrived) would
> enable to operate with a single window.
> >
> > Regards,
> >
> > Dr. Radu Tudoran
> > Research Engineer - Big Data Expert
> > IT R Division
> >
> > 
> > HUAWEI TECHNOLOGIES Duesseldorf GmbH
> > European Research Center
> > Riesstrasse 25, 80992 München
> >
> > E-mail: radu.tudo...@huawei.com
> > Mobile: +49 15209084330
> > Telephone: +49 891588344173
> >
> > HU

Re: continous time triger

[Aljoscha Krettek]

Hi Brian,
you are right about changing the behavior of windows when closing. Would this 
be a problem for you?

Cheers,
Aljoscha
> On 26 Jan 2016, at 17:53, Radu Tudoran <radu.tudo...@huawei.com> wrote:
> 
> Hi,
>  
> Thank you for sharing your experience and also to Till for the advice.
> What I would like to do is to be able to fire the window potentially multiple 
> times, even if an event did not arrive. I will look more about how dealing 
> with the processing time could help in this
>  
> Dr. Radu Tudoran
> Research Engineer - Big Data Expert
> IT R Division
>  
> 
> HUAWEI TECHNOLOGIES Duesseldorf GmbH
> European Research Center
> Riesstrasse 25, 80992 München
>  
> E-mail: radu.tudo...@huawei.com
> Mobile: +49 15209084330
> Telephone: +49 891588344173
>  
> HUAWEI TECHNOLOGIES Duesseldorf GmbH
> Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com
> Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
> Managing Director: Bo PENG, Wanzhou MENG, Lifang CHEN
> Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
> Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN
> This e-mail and its attachments contain confidential information from HUAWEI, 
> which is intended only for the person or entity whose address is listed 
> above. Any use of the information contained herein in any way (including, but 
> not limited to, total or partial disclosure, reproduction, or dissemination) 
> by persons other than the intended recipient(s) is prohibited. If you receive 
> this e-mail in error, please notify the sender by phone or email immediately 
> and delete it!
>  
> From: Brian Chhun [mailto:brian.ch...@getbraintree.com] 
> Sent: Tuesday, January 26, 2016 5:28 PM
> To: user@flink.apache.org
> Subject: Re: continous time triger
>  
> For what it's worth, we have a trigger that fires once a day for a recurring 
> calculation. When an element comes in, we set the trigger context's 
> processing time timer to the exact millisecond of the desired time. The 
> predefined triggers were useful to look at to achieve this: 
> https://github.com/apache/flink/tree/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/triggers
>  
> Some things I discovered along the way, particularly using processing time, 
> which may be useful:
> - registering a time that's already passed will cause the timer callback to 
> be called
> - when the system shuts down, the window is fired even though the trigger has 
> not gone off (this sounds subject to change though)
>  
> On Tue, Jan 26, 2016 at 3:47 AM, Till Rohrmann <trohrm...@apache.org> wrote:
> Hi Radu,
> 
> you can register processing and event time time triggers using the 
> TriggerContext which is given to the onElement, onProcessingTime and 
> onEventTime methods of Trigger. In case you register a processing time timer, 
> the onProcessingTime method will be called once the system clock has passed 
> the timer time. In case of an event time timer, the onEventTimemethod is 
> called once a watermark has been received which has a higher watermark than 
> the timer.
> 
> I hope this helps you to solve your problem.
> 
> Cheers,
> Till
> 
> ​
>  
> On Mon, Jan 25, 2016 at 9:25 PM, Radu Tudoran <radu.tudo...@huawei.com> wrote:
> Re-Hi,
>  
> I have another question regarding the triggering of the processing of a 
> window. Can this be done in some way at specific time intervals, independent 
> of whether  an event has been received or not, via a trigger?
>  
> The reason why I am considering a trigger rather than timeWindow(All) is that 
> timeWindow will end up generating multiple windows and duplicating data, 
> while having the option from the trigger to actually fire the processing at 
> certain times, independent of when the events arrived) would enable to 
> operate with a single window.
>  
> Regards,
>  
> Dr. Radu Tudoran
> Research Engineer - Big Data Expert
> IT R Division
>  
> 
> HUAWEI TECHNOLOGIES Duesseldorf GmbH
> European Research Center
> Riesstrasse 25, 80992 München
>  
> E-mail: radu.tudo...@huawei.com
> Mobile: +49 15209084330
> Telephone: +49 891588344173
>  
> HUAWEI TECHNOLOGIES Duesseldorf GmbH
> Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com
> Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
> Managing Director: Bo PENG, Wanzhou MENG, Lifang CHEN
> Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
> Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN
> This e-mail and its attachments contain confidential information from HUAWEI, 
> which is intended only for the person or entity whose address is listed 
> above. Any use of the information contained herein in any way (including, but 
> not limited to, total or partial disclosure, reproduction, or dissemination) 
> by persons other than the intended recipient(s) is prohibited. If you receive 
> this e-mail in error, please notify the sender by phone or email immediately 
> and delete it!

Re: continous time triger

[Till Rohrmann]

Hi Radu,

you can register processing and event time time triggers using the
TriggerContext which is given to the onElement, onProcessingTime and
onEventTime methods of Trigger. In case you register a processing time
timer, the onProcessingTime method will be called once the system clock has
passed the timer time. In case of an event time timer, the onEventTime
method is called once a watermark has been received which has a higher
watermark than the timer.

I hope this helps you to solve your problem.

Cheers,
Till
​

On Mon, Jan 25, 2016 at 9:25 PM, Radu Tudoran 
wrote:

> Re-Hi,
>
>
>
> I have another question regarding the triggering of the processing of a
> window. Can this be done in some way at specific time intervals,
> independent of whether  an event has been received or not, via a trigger?
>
>
>
> The reason why I am considering a trigger rather than timeWindow(All) is
> that timeWindow will end up generating multiple windows and duplicating
> data, while having the option from the trigger to actually fire the
> processing at certain times, independent of when the events arrived) would
> enable to operate with a single window.
>
>
>
> Regards,
>
>
>
> Dr. Radu Tudoran
>
> Research Engineer - Big Data Expert
>
> IT R Division
>
>
>
> [image: cid:image007.jpg@01CD52EB.AD060EE0]
>
> HUAWEI TECHNOLOGIES Duesseldorf GmbH
>
> European Research Center
>
> Riesstrasse 25, 80992 München
>
>
>
> E-mail: *radu.tudo...@huawei.com *
>
> Mobile: +49 15209084330
>
> Telephone: +49 891588344173
>
>
>
> HUAWEI TECHNOLOGIES Duesseldorf GmbH
> Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com
> Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
> Managing Director: Bo PENG, Wanzhou MENG, Lifang CHEN
> Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
> Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN
>
> This e-mail and its attachments contain confidential information from
> HUAWEI, which is intended only for the person or entity whose address is
> listed above. Any use of the information contained herein in any way
> (including, but not limited to, total or partial disclosure, reproduction,
> or dissemination) by persons other than the intended recipient(s) is
> prohibited. If you receive this e-mail in error, please notify the sender
> by phone or email immediately and delete it!
>
>
>

Re: continous time triger

[Brian Chhun]

For what it's worth, we have a trigger that fires once a day for a
recurring calculation. When an element comes in, we set the trigger
context's processing time timer to the exact millisecond of the desired
time. The predefined triggers were useful to look at to achieve this:
https://github.com/apache/flink/tree/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/windowing/triggers

Some things I discovered along the way, particularly using processing time,
which may be useful:
- registering a time that's already passed will cause the timer callback to
be called
- when the system shuts down, the window is fired even though the trigger
has not gone off (this sounds subject to change though)

On Tue, Jan 26, 2016 at 3:47 AM, Till Rohrmann  wrote:

> Hi Radu,
>
> you can register processing and event time time triggers using the
> TriggerContext which is given to the onElement, onProcessingTime and
> onEventTime methods of Trigger. In case you register a processing time
> timer, the onProcessingTime method will be called once the system clock
> has passed the timer time. In case of an event time timer, the onEventTime
> method is called once a watermark has been received which has a higher
> watermark than the timer.
>
> I hope this helps you to solve your problem.
>
> Cheers,
> Till
> ​
>
> On Mon, Jan 25, 2016 at 9:25 PM, Radu Tudoran 
> wrote:
>
>> Re-Hi,
>>
>>
>>
>> I have another question regarding the triggering of the processing of a
>> window. Can this be done in some way at specific time intervals,
>> independent of whether  an event has been received or not, via a trigger?
>>
>>
>>
>> The reason why I am considering a trigger rather than timeWindow(All) is
>> that timeWindow will end up generating multiple windows and duplicating
>> data, while having the option from the trigger to actually fire the
>> processing at certain times, independent of when the events arrived) would
>> enable to operate with a single window.
>>
>>
>>
>> Regards,
>>
>>
>>
>> Dr. Radu Tudoran
>>
>> Research Engineer - Big Data Expert
>>
>> IT R Division
>>
>>
>>
>> [image: cid:image007.jpg@01CD52EB.AD060EE0]
>>
>> HUAWEI TECHNOLOGIES Duesseldorf GmbH
>>
>> European Research Center
>>
>> Riesstrasse 25, 80992 München
>>
>>
>>
>> E-mail: *radu.tudo...@huawei.com *
>>
>> Mobile: +49 15209084330
>>
>> Telephone: +49 891588344173
>>
>>
>>
>> HUAWEI TECHNOLOGIES Duesseldorf GmbH
>> Hansaallee 205, 40549 Düsseldorf, Germany, www.huawei.com
>> Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
>> Managing Director: Bo PENG, Wanzhou MENG, Lifang CHEN
>> Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
>> Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN
>>
>> This e-mail and its attachments contain confidential information from
>> HUAWEI, which is intended only for the person or entity whose address is
>> listed above. Any use of the information contained herein in any way
>> (including, but not limited to, total or partial disclosure, reproduction,
>> or dissemination) by persons other than the intended recipient(s) is
>> prohibited. If you receive this e-mail in error, please notify the sender
>> by phone or email immediately and delete it!
>>
>>
>>
>
>

continous time triger

[Radu Tudoran]

Re-Hi,

I have another question regarding the triggering of the processing of a window. 
Can this be done in some way at specific time intervals, independent of whether 
 an event has been received or not, via a trigger?

The reason why I am considering a trigger rather than timeWindow(All) is that 
timeWindow will end up generating multiple windows and duplicating data, while 
having the option from the trigger to actually fire the processing at certain 
times, independent of when the events arrived) would enable to operate with a 
single window.

Regards,

Dr. Radu Tudoran
Research Engineer - Big Data Expert
IT R Division

[cid:image007.jpg@01CD52EB.AD060EE0]
HUAWEI TECHNOLOGIES Duesseldorf GmbH
European Research Center
Riesstrasse 25, 80992 München

E-mail: radu.tudo...@huawei.com
Mobile: +49 15209084330
Telephone: +49 891588344173

HUAWEI TECHNOLOGIES Duesseldorf GmbH
Hansaallee 205, 40549 Düsseldorf, Germany, 
www.huawei.com
Registered Office: Düsseldorf, Register Court Düsseldorf, HRB 56063,
Managing Director: Bo PENG, Wanzhou MENG, Lifang CHEN
Sitz der Gesellschaft: Düsseldorf, Amtsgericht Düsseldorf, HRB 56063,
Geschäftsführer: Bo PENG, Wanzhou MENG, Lifang CHEN
This e-mail and its attachments contain confidential information from HUAWEI, 
which is intended only for the person or entity whose address is listed above. 
Any use of the information contained herein in any way (including, but not 
limited to, total or partial disclosure, reproduction, or dissemination) by 
persons other than the intended recipient(s) is prohibited. If you receive this 
e-mail in error, please notify the sender by phone or email immediately and 
delete it!