Hi Max,
Here is the result from the "flink run" launcher node (devbox):
---
root@devbox:~# echo
${HADOOP_CLASSPATH}:${HADOOP_CONF_DIR}:${YARN_CONF_DIR}:${HBASE_CONF_DIR}
:/etc/hadoop-conf:/etc/yarn-conf:
---
Here is the result from one of the Cloudera
Can you provide more details about the problem your trying to solve with
some examples showing input and the expected output?
On Wed, Nov 30, 2016 at 11:08 PM, Manu Zhang
wrote:
> Hi,
>
> Recently I’m addressing a problem where users want to trigger after
> watermark
My use case is to track user trajectory based on page view event when they
visit a website. The input would be like a list of PageView(userId, url,
eventTimestamp) with watermarks (= eventTimestamp - duration). I'm trying
Sessions with event time trigger. Note we can't wait for the end of session
Thanks for laying out some details.
On Thu, Dec 1, 2016 at 7:09 PM, Manu Zhang wrote:
>
> Yes, the difficulty is to define that trigger. The existing triggers fire
> at the end of window. (I could be mistaken, which will be good news)
>
You are not mistaken that the
And one more question while we're at it: what if you have events happening
every second within the window? Do you really want to emit a new pane every
second as the watermark progresses (assuming it progresses relatively
smoothly)? What if we're talking differences of event times of
milliseconds?
@Tyler,
Yes, the difficulty is to define that trigger. The existing triggers fire
at the end of window. (I could be mistaken, which will be good news)
@Ben,
B and C which are not mutually exclusive
More on my use case. Say a user visits http://foo at 1, http://foo/bar at 4
and back to http://foo
@Kenn,
1. when the watermark jumps from 0 to 7, http://foo -> http://foo/bar ->
http://foo will be emitted
We can emit events with timestamps before watermark in the pane
2. http://foo -> http://foo/bizzle -> http://foo/bar -> http://foo will be
emitted if it's within the allowed lateness
which
So what you want is essentially a trigger that fires when the watermark has
passed the event time of the oldest un-emitted element in the current pane?
You could them presumably wrap this in a repeat to get the overall desired
semantics, right?
-Tyler
On Fri, Dec 2, 2016 at 7:32 AM Manu Zhang
Hi Rico,
As a short-term workaround, you should also be able to use the
HadoopFileSource with AvroInputFormat.
https://avro.apache.org/docs/1.7.6/api/java/org/apache/avro/mapred/AvroInputFormat.html
On Mon, Nov 21, 2016 at 3:34 AM, Amit Sela wrote:
> Hi Rico,
>
> We're