Stephan Ewen created FLINK-3514:
-----------------------------------
Summary: Add support for slowly changing streaming broadcast
variables
Key: FLINK-3514
URL: https://issues.apache.org/jira/browse/FLINK-3514
Project: Flink
Issue Type: New Feature
Components: Streaming
Reporter: Stephan Ewen
With "slow changing inputs", I mean auxiliary data sets that change over time.
Think about dictionaries, set of rules, etc that are updates at slow rates and
then applied onto another stream.
They are a bit like broadcast variables in the DataSet API, but not constant
over the entire lifetime of the job.
One can currently approximate this via connected streams:
{code}
DataStream<Rule> rules = ...
DataStream<Event> events = ...
rules.broadcast().connect(events).flatMap(...);
{code}
The disadvantage is that one has no control that the broadcasted "rules" input
is materialized in the function before the "events" are delivered.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
