If you are using tumbling time-windows, then the timestamp of the
aggregated records emitted from the window are all the maximum timestamp
that would have been accepted for the window.
For example, if you have an hourly tumbling window, the window from 2 to 3
o'clock would include all timestamps be
“Consistent” means that in the same time window, the timestamps of the
three streams should be kept the same.
In my application, I am trying to build an online learning system. I need
to join the streams from 1 and 2 on the SAME timestamp to form training
samples which will be fed to some online l
What do you mean by "consistent"?
Of course you can do this only at the time the timpstamp is defined (e.g. Using
NTP). However, this is never perfect .
Then it is unrealistic that they always end up in the same window because of
network delays etc. you will need here a global state that is defi
Thanks for your reply. I have another question:
In my situation, each of the three streams contains a local timestamp
segment. How can I ensure that their timestamps are consistent in each time
window before the merging operation? And how to ensure the arrival of all
the streams with consistent tim
Hi,
To expand on Fabian's answer, there's a few API for join.
* connect - you have to provide a CoprocessFunction.
* window join/cogroup - you provide key selector functions, a time window and
a join/cogroup function.
With the first method, you have to write more code, in exchange for much mo
Hi,
there are basically two operations to merge streams.
1. Union simply merges the input streams such that the resulting stream has
the records of all input streams. Union is a built-in operator in the
DataStream API. For that all streams must have the same data type.
2. Join connects records of
I have three data streams
1. app exposed and click
2. app download
3. app install
How can i merge the streams to create a unified stream,then compute it on
time-based windows
Thanks
