satishkotha commented on a change in pull request #1859:
URL: https://github.com/apache/hudi/pull/1859#discussion_r458513411
##########
File path:
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java
##########
@@ -103,14 +105,19 @@ protected void init(HoodieTableMetaClient metaClient,
HoodieTimeline visibleActi
* @param visibleActiveTimeline Visible Active Timeline
*/
protected void refreshTimeline(HoodieTimeline visibleActiveTimeline) {
- this.visibleCommitsAndCompactionTimeline =
visibleActiveTimeline.getCommitsAndCompactionTimeline();
+ this.visibleCommitsAndCompactionTimeline =
visibleActiveTimeline.getCommitsReplaceAndCompactionTimeline();
}
/**
* Adds the provided statuses into the file system view, and also caches it
inside this object.
*/
protected List<HoodieFileGroup> addFilesToView(FileStatus[] statuses) {
HoodieTimer timer = new HoodieTimer().startTimer();
+ final Map<String, Set<String>> partitionFileIdsToExclude =
getFileIdsToExclude(visibleCommitsAndCompactionTimeline);
Review comment:
Yes, for time travel, consider this scenario:
t0 -> insert
t1 -> insert overwrite1
t2 -> insert overwrite2
If we set high watermark to t1 for time travel, visibleCommitTimeline would
not have t2.commit, t2.replace. So file groups in t1 would still show as active
file groups.
When we move to t2, visibleCommitTimeline will have t2 commit/replace. So
file groups in t1 will not show as active
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
