cshuo commented on code in PR #19804:
URL: https://github.com/apache/hudi/pull/19804#discussion_r3912729383
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/AbstractHoodieLogRecordScanner.java:
##########
@@ -220,7 +229,40 @@ protected AbstractHoodieLogRecordScanner(HoodieStorage
storage, String basePath,
}
this.partitionNameOverrideOpt = partitionNameOverride;
+ this.deletePartitionPathOpt = resolveDeletePartitionPath(basePath,
logFilePaths, partitionNameOverride);
this.recordType = recordMerger.getRecordType();
+ this.readerContext = readerContext;
+ }
+
+ private static Option<String> resolveDeletePartitionPath(String basePath,
List<String> logFilePaths,
+ Option<String>
partitionNameOverride) {
+ if (partitionNameOverride.isPresent()) {
+ return partitionNameOverride;
+ }
+ if (logFilePaths.isEmpty()) {
+ return Option.empty();
+ }
+ return Option.of(getRelativePartitionPath(
+ new StoragePath(basePath), new
StoragePath(logFilePaths.get(0)).getParent()));
+ }
+
+ private <T> void initializeReaderContext(HoodieReaderContext<T>
readerContext) {
+ if (!isReaderContextInitialized) {
+ TypedProperties readerProps =
TypedProperties.copy(readerContext.getHoodieReaderConfig().getProps());
+ TypedProperties mergeProps = readerContext.getMergeProps(readerProps);
+ readerContext.setHasLogFiles(true);
+ readerContext.setHasBootstrapBaseFile(false);
+ readerContext.setShouldMergeUseRecordPosition(false);
+
readerContext.setTablePath(hoodieTableMetaClient.getBasePath().toString());
+ readerContext.setLatestCommitTime(latestInstantTime);
+
readerContext.getRecordContext().setPartitionPath(deletePartitionPathOpt.orElse(null));
+ readerContext.initRecordMerger(mergeProps);
Review Comment:
This reader context is only used to deserialize native delete records; the
scanner uses its own `recordMerger` for merging. Initializing the
reader-context merger can fail for `CUSTOM` merge mode when
`hoodie.write.record.merge.custom.implementation.classes` is unavailable, even
though it will never be used here.
Please remove `initRecordMerger` and `setSchemaHandler`. Instead, update
`HoodieAvroReaderContext#getFileRecordIterator` so the `isLogFile` path uses
`requiredSchema` directly without accessing the schema handler.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
