lamber-ken commented on a change in pull request #2581:
URL: https://github.com/apache/hudi/pull/2581#discussion_r580038941
##########
File path:
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/index/HoodieIndexUtils.java
##########
@@ -37,6 +38,29 @@
*/
public class HoodieIndexUtils {
+ /**
+ * Fetches Pair of partition path and {@link HoodieBaseFile}s for interested
partitions.
+ *
+ * @param partition Partition of interest
+ * @param context Instance of {@link HoodieEngineContext} to use
+ * @param hoodieTable Instance of {@link HoodieTable} of interest
+ * @return the list of {@link HoodieBaseFile}
+ */
+ public static List<HoodieBaseFile> getLatestBaseFilesForPartition(
Review comment:
It's good that `getLatestBaseFilesForPartition` was extracted from
`getLatestBaseFilesForAllPartitions`.
Current codebase:
```
public static List<HoodieBaseFile> getLatestBaseFilesForPartition(
final String partition,
final HoodieTable hoodieTable) {
Option<HoodieInstant> latestCommitTime =
hoodieTable.getMetaClient().getCommitsTimeline()
.filterCompletedInstants().lastInstant();
if (latestCommitTime.isPresent()) {
return hoodieTable.getBaseFileOnlyView()
.getLatestBaseFilesBeforeOrOn(partition,
latestCommitTime.get().getTimestamp())
.collect(toList());
}
return Collections.emptyList();
}
```
Maybe the following implementation is more efficient
```
public static List<HoodieBaseFile> getLatestBaseFilesForPartition(
final String partition,
final HoodieTable hoodieTable) {
return hoodieTable.getFileSystemView()
.getAllFileGroups(partition)
.map(HoodieFileGroup::getLatestDataFile)
.filter(Option::isPresent)
.map(Option::get)
.collect(toList());
}
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
