codope commented on code in PR #5941:
URL: https://github.com/apache/hudi/pull/5941#discussion_r904895593
##########
hudi-common/src/main/java/org/apache/hudi/common/table/view/AbstractTableFileSystemView.java:
##########
@@ -973,6 +974,7 @@ Stream<FileSlice> fetchAllFileSlices(String partitionPath) {
*/
public Stream<HoodieBaseFile> fetchLatestBaseFiles(final String
partitionPath) {
return fetchAllStoredFileGroups(partitionPath)
+ .filter(fg -> !isFileGroupReplaced(fg))
Review Comment:
This is needed when metadata is not enabled in presto-hive connector then we
let presto do the listing and just fetch base files using this API after adding
to the view. `getLatestBaseFiles(String partitionStr)` does additional work,
which we did not want for perf reasons.
> should we audit all similar APIs regarding compaction and clustering?
I did check all other public APIs in this view class. This is the only one.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
