Hi Zhen, This might be a rather inefficient solution. We have encountered situations that we need to have some daily config update pushed to our flink streaming application, where the state is very large (but keyed). We end-up having a service to push that data into a separated kafka stream (which basically has one burst of messages each day), they used a stream-stream join to consolidate the configuration with the actual real data stream.
This would only work if your real-stream and your configuration are basically sharing the same keyed dimension though. -- Rong On Sat, Jun 30, 2018 at 2:37 AM zhen li <lizhenm...@hotmail.com> wrote: > Hi，Fabian： > I use connected stream to solve this problem，one is config stream that > load some config data from redis.another is real data stream. Due to the > config data is vary big, then the config stream is slowly than the real > data stream. When use some config to deal data in flatmap2, it arise the > NullPointerException. > Is there any config to make one stream finished and then start the second? > or any other ways to solve this problem？ > . > > 在 2018年6月21日，下午6:31，Fabian Hueske <fhue...@gmail.com> 写道： > > That's correct. > > Broadcast state was added with Flink 1.5. You can use > DataStream.broadcast() in Flink 1.3 but it has a few limitations. > For example, you cannot connect a keyed and a broadcasted stream. > > 2018-06-21 11:58 GMT+02:00 zhen li <lizhenm...@hotmail.com>: > >> Thanks for your reply. >> But broadcast state seems not supported in Flink-1.3 . >> I found this in Flink-1.3: >> Broadcasting >> DataStream → DataStream >> >> Broadcasts elements to every partition. >> >> dataStream.broadcast(); >> >> But I don’t know how to convert it to list and get it in stream context . >> >> 在 2018年6月21日，下午5:20，Fabian Hueske <fhue...@gmail.com> 写道： >> >> >> https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/stream/state/broadcast_state.html >> >> >> > >
