beryllw commented on code in PR #3430:
URL: https://github.com/apache/fluss/pull/3430#discussion_r3354288738
##########
fluss-flink/fluss-flink-common/src/main/java/org/apache/fluss/flink/tiering/source/TieringSplitReader.java:
##########
@@ -171,8 +187,22 @@ public
RecordsWithSplitIds<TableBucketWriteResult<WriteResult>> fetch() throws I
if (reachTieringMaxDurationTables.contains(currentTableId)) {
return forceCompleteTieringLogRecords();
}
- ScanRecords scanRecords = currentLogScanner.poll(pollTimeout);
- return forLogRecords(scanRecords);
+ if (useRecordBatchPath()) {
+ ArrowScanRecords arrowScanRecords =
+ ((LogScannerImpl)
currentLogScanner).pollRecordBatch(pollTimeout);
+ return processLogRecords(
+ arrowScanRecords.buckets(),
+ arrowScanRecords::records,
+ this::handleArrowBatchRecords,
Review Comment:
If processLogRecords throws partway through iterating buckets, the remaining
buckets' Arrow batches won't be closed. Since ArrowScanRecords is
AutoCloseable, wrapping it might be the cleanest way to ensure all off-heap
memory is released on error paths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
