linliu-code commented on code in PR #9819:
URL: https://github.com/apache/hudi/pull/9819#discussion_r1353478334
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/BaseHoodieLogRecordReader.java:
##########
@@ -410,6 +418,34 @@ private void scanInternalV1(Option<KeySpec> keySpecOpt) {
}
}
+ private static void prepareRecordPositions(String blockPositionStr,
+ List<Integer> blockPositions,
+ Set<Integer> globalPositions,
+ boolean positionBasedMergeEnabled,
+ boolean useEventOrdering) throws
IOException {
+ if (!positionBasedMergeEnabled || blockPositionStr == null ||
blockPositionStr.isEmpty()) {
+ return;
+ }
+
+ Roaring64NavigableMap posBitmap =
PositionUtils.deserializeCodecStringWithRoaring64NavigableMap(blockPositionStr);
+ Iterator<Long> iterator = posBitmap.iterator();
+ while (iterator.hasNext()) {
+ long pos = iterator.next();
Review Comment:
@the-other-tim-brown , probably not. Max value of integer is more than 2
billion, which is the number of the rows in a base file. Given the default
setting for a base file size (128MB), it is impossible to have row number > 2
billion since 2 billion rows * row_size (> 1Byte) > 2 ^ 30 = 2GB (without meta
columns). But not sure if a user wants to have a really huge base file or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
