Thanks a lot, David and Fabian.
I will give this a try.
Cheers,Chirag
On Monday, 1 October, 2018, 3:48:42 PM IST, David Anderson
wrote:
Hi Chirag,
The community is also looking at an approach that involves using Bravo[1][2] to
bootstrap state by loading the initial version of the state
Hi Chirag,
The community is also looking at an approach that involves using
Bravo[1][2] to bootstrap state by loading the initial version of the state
into a savepoint.
[1] https://github.com/king/bravo
[2]
Hi Chirag,
Flink 1.5.0 added support for BroadcastState which should address your
requirement of replicating the data. [1]
The replicated data is stored in the configured state backend which can
also be RocksDB.
Regarding the reload, I would recommend Lasse's approach of having a custom
source
Thanks Lasse, that is rightly put. That's the only solution I can think of too.
Only thing which I can't get my head around is using the coMap and coFlatMap
functions with such a stream. Since they dont support side outputs, is there a
way my lookup map/flatmap function simply consume a stream?
Hi Lasse,
One approach I’ve used in a similar situation is to have a “UnionedSource”
wrapper that first emits the (bounded) data that will be loaded in-memory, and
then starts running the source that emits the continuous stream of data.
This outputs an Either, which I then split, and broadcast
Hi.
We have created our own database source that pools the data with a configured
interval. We then use a co processed function. It takes to input one from our
database and one from our data input. I require that you keyby with the
attributes you use lookup in your map function.
To delay
