nanjeshramesh commented on code in PR #20353:
URL: https://github.com/apache/hudi/pull/20353#discussion_r4299955353
##########
hudi-common/src/main/java/org/apache/hudi/common/table/log/HoodieLogBlockMetadataScanner.java:
##########
@@ -36,12 +37,20 @@ public class HoodieLogBlockMetadataScanner extends
BaseHoodieLogRecordReader<Ind
public HoodieLogBlockMetadataScanner(HoodieTableMetaClient metaClient,
List<HoodieLogFile> logFiles,
int bufferSize, String maxInstantTime,
Option<InstantRange> instantRange) {
- super(getReaderContext(metaClient, maxInstantTime),
TableState.fromMetaClient(metaClient), metaClient.getStorage(), logFiles,
+ super(getReaderContext(metaClient, maxInstantTime),
TableState.fromMetaClient(metaClient), metaClient.getStorage(),
withoutCdcLogFiles(logFiles),
Review Comment:
Thanks @shangxinli, good catch. The file count shortcut ran before the
scanner and counted CDC log files, so a slice with only CDC logs could still be
scheduled. I pushed a follow-up commit that leaves CDC log files out of that
count too, with a test: a slice of CDC logs alone is not eligible, and enough
regular log files still are. I also updated the PR description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
