Re: watermark trigger doesn't check whether element's timestamp is passed

Thanks, that will be great. I'd like to test against my particular use cases once your PR is available. Manu On Wed, Nov 2, 2016 at 11:09 PM Ventura Del Monte wrote: > Hello, > > I have just opened the JIRA issue >

Re: watermark trigger doesn't check whether element's timestamp is passed

Hello, I have just opened the JIRA issue and I have almost completed the implementation of this feature. I will keep you posted :) Cheers, Ventura This message, for the D. Lgs n. 196/2003 (Privacy Code), may contain confidential and/or

Re: watermark trigger doesn't check whether element's timestamp is passed

Hi, a contributor (Bonaventure Del Monte) has started working on this. He should open a Jira this week. Cheer, Aljoscha On Tue, 1 Nov 2016 at 23:57 aj heller wrote: Hi Manu, Aljoscha, I had been interested in implementing FLIP-2, but I haven't been able to make time for it.

Re: watermark trigger doesn't check whether element's timestamp is passed

Hi Manu, Aljoscha, I had been interested in implementing FLIP-2, but I haven't been able to make time for it. There is no implementation yet that I'm aware of, and I'll gladly step aside (or help out how I can) if you or anyone is interested to take charge of it. That said, I'm also not sure if

Re: watermark trigger doesn't check whether element's timestamp is passed

Thanks. The ideal case is to fire after watermark past each element from the window but that requires a custom trigger and FLIP-2 as well. The enhanced window evictor will help to avoid the last firing. Are the discussions on FLIP-2 still going on ? Are there any opening JIRAs or PRs ? (The

Re: watermark trigger doesn't check whether element's timestamp is passed

Ah, I finally understand it. You would a way to query the current watermark in the window function to only emit those elements where the timestamp is lower than the watermark. When the window fires again, do you want to emit elements that you emitted during the last firing again? If not, I think

Re: watermark trigger doesn't check whether element's timestamp is passed

Yes, here's the example https://github.com/manuzhang/flink/blob/pv/flink-examples/flink-examples-streaming/src/main/scala/org/apache/flink/streaming/scala/examples/session/PageViewSessionWindowing.scala If you print and compare the timestamp of timer with that of "PageView" in the outputs, you

Re: watermark trigger doesn't check whether element's timestamp is passed

Hmm, I don't completely understand what's going on. Could you maybe post an example, with the trigger code that shows this behaviour? Cheers, Aljoscha On Thu, 27 Oct 2016 at 17:12 Manu Zhang wrote: > Hi, > > It's what I'm seeing. If timers are not fired at the end of

Re: watermark trigger doesn't check whether element's timestamp is passed

Hi, It's what I'm seeing. If timers are not fired at the end of window, a state (in the window) whose timestamp is *after *the timer will also be emitted. That's a problem for event-time trigger. Thanks, Manu On Thu, Oct 27, 2016 at 10:29 PM Aljoscha Krettek wrote: > Hi,

Re: watermark trigger doesn't check whether element's timestamp is passed

Hi, is that example input/output what you would like to achieve or what you are currently seeing with Flink? I think for your use case a custom Trigger would be required that works like the event-time trigger but additionally registers timers for each element where you want to emit. Cheers,

Re: watermark trigger doesn't check whether element's timestamp is passed

Hi, with some additional information we might be able to figure this out together. What specific combination of WindowAssigner/Trigger are you using for your example and what is the input stream (including watermarks)? Cheers, Aljoscha On Mon, 24 Oct 2016 at 06:30 Manu Zhang