Zhipeng Zhang created FLINK-31901: ------------------------------------- Summary: AbstractBroadcastWrapperOperator should not block checkpoint barriers when processing cached records Key: FLINK-31901 URL: https://issues.apache.org/jira/browse/FLINK-31901 Project: Flink Issue Type: Improvement Components: Library / Machine Learning Reporter: Zhipeng Zhang Fix For: ml-2.3.0
Currently `BroadcastUtils#withBroadcast` tries to caches the non-broadcast input until the broadcast inputs are all processed. After the broadcast variables are ready, we first process the cached records and then continue to process the newly arrived records. Processing cached elements is invoked via `Input#processElement` and `Input#processWatermark`. However, processing cached element may take a long time since there may be many cached records, which could potentially block the checkpoint barrier. -- This message was sent by Atlassian Jira (v8.20.10#820010)