rahil-c commented on code in PR #13591:
URL: https://github.com/apache/hudi/pull/13591#discussion_r2230246531
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/io/FileGroupReaderBasedAppendHandle.java:
##########
@@ -85,8 +85,10 @@ public void doAppend() {
// Initializes the record iterator, log compaction requires writing the
deletes into the delete block of the resulting log file.
try (HoodieFileGroupReader<T> fileGroupReader =
HoodieFileGroupReader.<T>newBuilder().withReaderContext(readerContext).withHoodieTableMetaClient(hoodieTable.getMetaClient())
.withLatestCommitTime(instantTime).withPartitionPath(partitionPath).withLogFiles(logFiles).withBaseFileOption(Option.empty()).withDataSchema(writeSchemaWithMetaFields)
-
.withRequestedSchema(writeSchemaWithMetaFields).withEnableOptimizedLogBlockScan(true).withInternalSchema(internalSchemaOption).withProps(props).withEmitDelete(true)
-
.withShouldUseRecordPosition(usePosition).withSortOutput(hoodieTable.requireSortedRecords()).build())
{
+
.withRequestedSchema(writeSchemaWithMetaFields).withInternalSchema(internalSchemaOption).withProps(props).withEmitDelete(true)
+
.withShouldUseRecordPosition(usePosition).withSortOutput(hoodieTable.requireSortedRecords())
+ // instead of using config.enableOptimizedLogBlocksScan(), we set to
true as log compaction blocks only supported in scanV2
+ .withEnableOptimizedLogBlockScan(true).build()) {
Review Comment:
Thanks @danny0405 for sharing context.
@lokeshj1703 this seemed to have been the only area in code base where this
was set to `true` explicitly. When reading the java doc of the class it
mentioned this
```
* This append-handle is used for log-compaction, which passes a file slice
from the
* compaction operation of a single file group to a file group reader, get
an iterator of
* the records, and writes the records to a new log file.
*/
```
So this value will need to be true always, as this will invoke the scanV2
path which has support of the log compaction.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
