YuweiXiao commented on code in PR #6680:
URL: https://github.com/apache/hudi/pull/6680#discussion_r1000217067
##########
hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java:
##########
@@ -347,6 +417,17 @@ private void validate(HoodieTimeline activeTimeline,
Option<String> queryInstant
}
}
+ protected boolean isAllInputFileSlicesCached() {
+ if (cachedAllPartitionPaths == null) {
+ return false;
+ }
+ return cachedAllPartitionPaths.stream().allMatch(p ->
cachedAllInputFileSlices.containsKey(p));
+ }
+
+ protected boolean isPartitionedTable() {
+ return !queryAsNonePartitionedTable && (partitionColumns.length > 0 ||
HoodieTableMetadata.isMetadataTable(basePath.toString()));
Review Comment:
Yeah, I was trying to remove it but some tests will fail. The thing is if we
fail to parse partition values from the partition path (e.g., non-encoded
partition value containing `/`), the original implementation will query the
table as non-partitioned-table to obtain correct results.
Check out `SparkHoodieTableFileIndex::parsePartitionColumnValues`. And in
the failure case we will flag the query:
```
this.queryAsNonePartitionedTable =
getAllQueryPartitionPaths().stream().anyMatch(p -> p.values.length == 0);
```
Can not come up a good way to get rid of the variable completely :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
