the-other-tim-brown commented on code in PR #9819:
URL: https://github.com/apache/hudi/pull/9819#discussion_r1353734650
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/BaseHoodieLogRecordReader.java:
##########
@@ -410,6 +418,34 @@ private void scanInternalV1(Option<KeySpec> keySpecOpt) {
}
}
+ private static void prepareRecordPositions(String blockPositionStr,
+ List<Integer> blockPositions,
+ Set<Integer> globalPositions,
+ boolean positionBasedMergeEnabled,
+ boolean useEventOrdering) throws
IOException {
+ if (!positionBasedMergeEnabled || blockPositionStr == null ||
blockPositionStr.isEmpty()) {
+ return;
+ }
+
+ Roaring64NavigableMap posBitmap =
PositionUtils.deserializeCodecStringWithRoaring64NavigableMap(blockPositionStr);
+ Iterator<Long> iterator = posBitmap.iterator();
+ while (iterator.hasNext()) {
+ long pos = iterator.next();
Review Comment:
2 billion is probably larger than we will see. Are we worried about casting
and then boxing the int to integer for each row in the file from a performance
point of view?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
