linliu-code commented on code in PR #9819:
URL: https://github.com/apache/hudi/pull/9819#discussion_r1353655893
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/BaseHoodieLogRecordReader.java:
##########
@@ -756,6 +792,50 @@ private void processDataBlock(HoodieDataBlock dataBlock,
Option<KeySpec> keySpec
}
}
+ private void processDataBlockBasedOnPosition(HoodieDataBlock dataBlock,
Option<KeySpec> keySpecOpt) throws Exception {
+ checkState(partitionNameOverrideOpt.isPresent() ||
partitionPathFieldOpt.isPresent(),
+ "Either partition-name override or partition-path field had to be
present");
+
+ Option<Pair<String, String>> recordKeyPartitionPathFieldPair =
populateMetaFields
+ ? Option.empty()
+ : Option.of(Pair.of(recordKeyField,
partitionPathFieldOpt.orElse(null)));
+
+ // Prepare position information for the records in the block.
+ String blockPositionStr =
dataBlock.getLogBlockHeader().get(HoodieLogBlock.HeaderMetadataType.RECORD_POSITIONS);
+ // TODO: Use keys to filter this list.
+ List<Integer> blockPositions = new ArrayList<>();
+ prepareRecordPositions(
Review Comment:
Add a check to throw when the blockPositions is empty; I assume that when we
use record position, the position information should exist.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
