jonvex commented on code in PR #10098:
URL: https://github.com/apache/hudi/pull/10098#discussion_r1395910938
##########
hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java:
##########
@@ -300,6 +311,19 @@ protected List<PartitionPath>
listPartitionPaths(List<String> relativePartitionP
protected List<PartitionPath> listPartitionPaths(List<String>
relativePartitionPaths) {
List<String> matchedPartitionPaths;
try {
+ if (isPartitionedTable()) {
+ if (queryType == HoodieTableQueryType.INCREMENTAL &&
beginInstantTime.isPresent()) {
+ HoodieTimeline timelineAfterBeginInstant =
TimelineUtils.getCommitsTimelineAfter(metaClient, beginInstantTime.get(),
Option.empty());
Review Comment:
This is finding the range (startTs, endTs] for queries.
MergeOnReadIncrementalRelation seems to be doing the same thing. This seems
like an easy thing to mess up with incremental so just want to highlight it
##########
hudi-common/src/main/java/org/apache/hudi/BaseHoodieTableFileIndex.java:
##########
@@ -300,6 +311,19 @@ protected List<PartitionPath>
listPartitionPaths(List<String> relativePartitionP
protected List<PartitionPath> listPartitionPaths(List<String>
relativePartitionPaths) {
List<String> matchedPartitionPaths;
try {
+ if (isPartitionedTable()) {
+ if (queryType == HoodieTableQueryType.INCREMENTAL &&
beginInstantTime.isPresent()) {
+ HoodieTimeline timelineAfterBeginInstant =
TimelineUtils.getCommitsTimelineAfter(metaClient, beginInstantTime.get(),
Option.empty());
+ HoodieTimeline timelineToQuery =
endInstantTime.map(timelineAfterBeginInstant::findInstantsBeforeOrEquals).orElse(timelineAfterBeginInstant);
+ matchedPartitionPaths =
TimelineUtils.getWrittenPartitions(timelineToQuery);
+ } else {
+ matchedPartitionPaths =
tableMetadata.getPartitionPathWithPathPrefixes(relativePartitionPaths);
+ }
+ } else {
+ matchedPartitionPaths = Collections.singletonList("");
+ }
+
+
matchedPartitionPaths =
tableMetadata.getPartitionPathWithPathPrefixes(relativePartitionPaths);
Review Comment:
I think this line needs to be deleted
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
