wombatu-kun opened a new issue, #19302: URL: https://github.com/apache/hudi/issues/19302
### Background PR #19202 reworked the Flink Source V2 read path so that each `BatchRecords` holds a materialized, bounded minibatch of records instead of a live `ClosableIterator`. As a result, `BatchRecords#recycle()` is now a no-op: the batch owns no live I/O, and the reader function (`AbstractSplitReaderFunction`) owns and closes the iterator, `CdcImageManager` and file-group readers on the split-fetcher thread. ### Follow-up Raised by @cshuo during review of #19202: explore reusing consumed batches (and their row objects) through an object pool, so `recycle()` can return the buffer and its `RowData` copies to a pool instead of discarding them. This could reduce allocation pressure on the source side when a read produces many small minibatches. ### Scope and constraints - Source-side only (`hudi-flink` reader path: `AbstractSplitReaderFunction`, `BatchRecords`, `HoodieSourceSplitReader`). - Must preserve the current copy-on-materialize semantics: each buffered record is a distinct copy (not the reader's reused `RowData`) and its `RowKind` is retained. - Must preserve the existing offset continuity, checkpoint-resume skipping, global limit push-down, and two-phase split-finish contract. - Pooled objects must be recycled only after the task thread has finished draining a batch, so a pooled row is never handed back while still referenced downstream. Link back: #19202 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
