shangxinli commented on code in PR #20481:
URL: https://github.com/apache/hudi/pull/20481#discussion_r4443913710
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/HoodieTableSource.java:
##########
@@ -399,7 +399,10 @@ public Result applyFilters(List<ResolvedExpression>
filters) {
Tuple2<List<ResolvedExpression>, List<ResolvedExpression>> splitFilters =
splitExprByPartitionCall(simpleFilters, this.partitionKeys, this.tableRowType);
this.predicates = ExpressionPredicates.fromExpression(splitFilters.f0);
this.columnStatsProbe = ColumnStatsProbe.newInstance(splitFilters.f0);
- this.partitionPruner = createPartitionPruner(splitFilters.f1,
columnStatsProbe);
+ // Partition stats only reflect the latest table state and can exclude
values in historical incremental reads.
+ boolean incrementalRead = conf.get(FlinkOptions.READ_AS_STREAMING)
Review Comment:
You're right, setupReadOptions normalizes QUERY_TYPE to incremental before
this runs, so the case I raised can't actually reach here through the table
source.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
