Re: Event-time based in-window trigger

@Kenn, 1. when the watermark jumps from 0 to 7, http://foo -> http://foo/bar -> http://foo will be emitted We can emit events with timestamps before watermark in the pane 2. http://foo -> http://foo/bizzle -> http://foo/bar -> http://foo will be emitted if it's within the allowed lateness which

Re: Event-time based in-window trigger

And one more question while we're at it: what if you have events happening every second within the window? Do you really want to emit a new pane every second as the watermark progresses (assuming it progresses relatively smoothly)? What if we're talking differences of event times of milliseconds?

Re: Event-time based in-window trigger

Thanks for laying out some details. On Thu, Dec 1, 2016 at 7:09 PM, Manu Zhang wrote: > > Yes, the difficulty is to define that trigger. The existing triggers fire > at the end of window. (I could be mistaken, which will be good news) > You are not mistaken that the

Re: Event-time based in-window trigger

@Tyler, Yes, the difficulty is to define that trigger. The existing triggers fire at the end of window. (I could be mistaken, which will be good news) @Ben, B and C which are not mutually exclusive More on my use case. Say a user visits http://foo at 1, http://foo/bar at 4 and back to http://foo

Re: Event-time based in-window trigger

So what you want is essentially a trigger that fires when the watermark has passed the event time of the oldest un-emitted element in the current pane? You could them presumably wrap this in a repeat to get the overall desired semantics, right? -Tyler On Fri, Dec 2, 2016 at 7:32 AM Manu Zhang

Re: Support for reading avro files from HDFS?

Hi Rico, As a short-term workaround, you should also be able to use the HadoopFileSource with AvroInputFormat. https://avro.apache.org/docs/1.7.6/api/java/org/apache/avro/mapred/AvroInputFormat.html On Mon, Nov 21, 2016 at 3:34 AM, Amit Sela wrote: > Hi Rico, > > We're

Re: Event-time based in-window trigger

My use case is to track user trajectory based on page view event when they visit a website. The input would be like a list of PageView(userId, url, eventTimestamp) with watermarks (= eventTimestamp - duration). I'm trying Sessions with event time trigger. Note we can't wait for the end of session

Re: Event-time based in-window trigger

Can you provide more details about the problem your trying to solve with some examples showing input and the expected output? On Wed, Nov 30, 2016 at 11:08 PM, Manu Zhang wrote: > Hi, > > Recently I’m addressing a problem where users want to trigger after > watermark

Re: java.lang.IllegalAccessError with KafkaIO

Hi Max, Here is the result from the "flink run" launcher node (devbox): --- root@devbox:~# echo ${HADOOP_CLASSPATH}:${HADOOP_CONF_DIR}:${YARN_CONF_DIR}:${HBASE_CONF_DIR} :/etc/hadoop-conf:/etc/yarn-conf: --- Here is the result from one of the Cloudera