Hi everyone,
Let's imagine I have a stream of events coming a bit like this:
{ id: "1", value: 1, timestamp: 1 }
{ id: "2", value: 2, timestamp: 1 }
{ id: "1", value: 4, timestamp: 3 }
{ id: "1", value: 5, timestamp: 2 }
{ id: "2", value: 5, timestamp: 3 }
...
As you can see with the non monotonically increasing timestamps, for
various reasons, events can be slightly "un-ordered"
Now I want to use Flink to process this stream, to compute by id (my key)
the latest value and update it in a DB. But obviously that latest value
must reflect the original time stamp and not the processing time stamp.
I've seen that Flink can deal with event-time processing, in the sense that
if I need to do a windowed operation I can ensure an event will be assign
to the "correct" window.
But here the use-case seems slightly different. How would you proceed to do
that in Flink?
Thanks!
--
Christophe
