[
https://issues.apache.org/jira/browse/HUDI-8372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Y Ethan Guo updated HUDI-8372:
------------------------------
Description:
There are methods of the same functionality in DataSourceTestUtils, using
FileSystem or HoodieStorage instance. Once we get rid of relevant FileSystem
usage, we should keep one of them only.
{code:java}
public static String latestCommitCompletionTime(FileSystem fs, String basePath)
{
HoodieTimeline timeline =
HoodieDataSourceHelpers.allCompletedCommitsCompactions(fs, basePath);
return timeline.getInstantsAsStream()
.map(HoodieInstant::getCompletionTime)
.filter(Objects::nonNull)
.max(String::compareTo)
.orElse(null);
}
public static String latestCommitCompletionTime(HoodieStorage storage, String
basePath) {
HoodieTimeline timeline =
HoodieDataSourceHelpers.allCompletedCommitsCompactions(storage, basePath);
return timeline.getInstantsAsStream()
.map(HoodieInstant::getCompletionTime)
.filter(Objects::nonNull)
.max(String::compareTo)
.orElse(null);
} {code}
was:
There are methods of the same functionality, using FileSystem or HoodieStorage
instance. Once we get rid of relevant FileSystem usage, we should keep one of
them only.
{code:java}
public static String latestCommitCompletionTime(FileSystem fs, String basePath)
{
HoodieTimeline timeline =
HoodieDataSourceHelpers.allCompletedCommitsCompactions(fs, basePath);
return timeline.getInstantsAsStream()
.map(HoodieInstant::getCompletionTime)
.filter(Objects::nonNull)
.max(String::compareTo)
.orElse(null);
}
public static String latestCommitCompletionTime(HoodieStorage storage, String
basePath) {
HoodieTimeline timeline =
HoodieDataSourceHelpers.allCompletedCommitsCompactions(storage, basePath);
return timeline.getInstantsAsStream()
.map(HoodieInstant::getCompletionTime)
.filter(Objects::nonNull)
.max(String::compareTo)
.orElse(null);
} {code}
> Keep one latestCommitCompletionTime method in DataSourceTestUtils
> -----------------------------------------------------------------
>
> Key: HUDI-8372
> URL: https://issues.apache.org/jira/browse/HUDI-8372
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: Y Ethan Guo
> Priority: Major
> Fix For: 1.1.0
>
>
> There are methods of the same functionality in DataSourceTestUtils, using
> FileSystem or HoodieStorage instance. Once we get rid of relevant FileSystem
> usage, we should keep one of them only.
> {code:java}
> public static String latestCommitCompletionTime(FileSystem fs, String
> basePath) {
> HoodieTimeline timeline =
> HoodieDataSourceHelpers.allCompletedCommitsCompactions(fs, basePath);
> return timeline.getInstantsAsStream()
> .map(HoodieInstant::getCompletionTime)
> .filter(Objects::nonNull)
> .max(String::compareTo)
> .orElse(null);
> }
> public static String latestCommitCompletionTime(HoodieStorage storage, String
> basePath) {
> HoodieTimeline timeline =
> HoodieDataSourceHelpers.allCompletedCommitsCompactions(storage, basePath);
> return timeline.getInstantsAsStream()
> .map(HoodieInstant::getCompletionTime)
> .filter(Objects::nonNull)
> .max(String::compareTo)
> .orElse(null);
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
