[
https://issues.apache.org/jira/browse/FLINK-33753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prabhu Joseph updated FLINK-33753:
----------------------------------
Affects Version/s: 1.16.0
(was: 1.18.0)
> ContinuousFileReaderOperator consume records as mini batch
> ----------------------------------------------------------
>
> Key: FLINK-33753
> URL: https://issues.apache.org/jira/browse/FLINK-33753
> Project: Flink
> Issue Type: Improvement
> Affects Versions: 1.16.0
> Reporter: Prabhu Joseph
> Priority: Major
>
> The ContinuousFileReaderOperator reads and collects the records from a split
> in a loop. If the split size is large, then the loop will take more time, and
> then the mailbox executor won't have a chance to process the checkpoint
> barrier. This leads to checkpoint timing out. ContinuousFileReaderOperator
> could be improved to consume the records in a mini batch, similar to Hudi's
> StreamReadOperator (https://issues.apache.org/jira/browse/HUDI-2485).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
