trushev commented on code in PR #7113:
URL: https://github.com/apache/hudi/pull/7113#discussion_r1011312520
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/source/FileIndex.java:
##########
@@ -80,6 +80,12 @@ public static FileIndex instance(Path path, Configuration
conf, RowType rowType)
return new FileIndex(path, conf, rowType);
}
+ public static FileIndex instance(Path path, Configuration conf, RowType
rowType, List<ResolvedExpression> filters) {
+ FileIndex fileIndex = instance(path, conf, rowType);
Review Comment:
It is a really good question
Quick answer -- yes we should keep because the method is used in
`IncrementalInputSplits` without filter at all.
But why there is no a filter in `IncrementalInputSplits`. Looks like
`FileIndex` useless in this way. I mean data skipping feature skips nothing
Mb we should pass filters from `HoodieTableSource` here:
```java
case FlinkOptions.QUERY_TYPE_INCREMENTAL:
IncrementalInputSplits incrementalInputSplits =
IncrementalInputSplits.builder()
.conf(conf)
.path(FilePathUtils.toFlinkPath(path))
.rowType(this.tableRowType)
.maxCompactionMemoryInBytes(maxCompactionMemoryInBytes)
.requiredPartitions(getRequiredPartitionPaths()).build();
```
I think it is not part of this fix related to `copy()` by `HepPlanner`
WDYT about the separate ticket/PR for this problem if the one exists?:)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
