yihua commented on code in PR #11131:
URL: https://github.com/apache/hudi/pull/11131#discussion_r1593573466
##########
hudi-hadoop-common/src/test/java/org/apache/hudi/common/testutils/FileSystemTestUtils.java:
##########
@@ -79,34 +68,4 @@ public static void deleteFile(File fileToDelete) throws
IOException {
throw new IOException(message);
}
}
-
- public static List<FileStatus> listRecursive(FileSystem fs, Path path)
throws IOException {
- return listFiles(fs, path, true);
- }
-
- public static List<FileStatus> listFiles(FileSystem fs, Path path, boolean
recursive) throws IOException {
- RemoteIterator<LocatedFileStatus> itr = fs.listFiles(path, recursive);
- List<FileStatus> statuses = new ArrayList<>();
- while (itr.hasNext()) {
- statuses.add(itr.next());
- }
- return statuses;
- }
-
- public static List<StoragePathInfo> listRecursive(HoodieStorage storage,
StoragePath path)
- throws IOException {
- return listFiles(storage, path);
- }
-
- public static List<StoragePathInfo> listFiles(HoodieStorage storage,
StoragePath path)
- throws IOException {
- return storage.listFiles(path);
- }
-
- public static String readLastLineFromResourceFile(String resourceName)
throws IOException {
- try (InputStream inputStream =
TestLogReaderUtils.class.getResourceAsStream(resourceName)) {
- List<String> lines = FileIOUtils.readAsUTFStringLines(inputStream);
- return lines.get(lines.size() - 1);
- }
- }
Review Comment:
Are these only used by tests? Should the methods stay here if there is no
compelling reason to move them?
##########
hudi-hadoop-mr/pom.xml:
##########
@@ -108,6 +114,22 @@
<type>test-jar</type>
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>org.apache.hudi</groupId>
+ <artifactId>hudi-hadoop-common</artifactId>
+ <version>${project.version}</version>
+ <classifier>tests</classifier>
+ <type>test-jar</type>
Review Comment:
Is `test-jar` type required?
##########
hudi-hadoop-common/src/test/java/org/apache/hudi/common/table/timeline/TestWaitBasedTimeGenerator.java:
##########
@@ -75,7 +75,7 @@ public boolean tryLock(long time, TimeUnit unit) {
// Clock skew time
private final long clockSkewTime = 20L;
- private final StorageConfiguration<?> storageConf =
HadoopFSUtils.getStorageConf(new Configuration());
+ private final StorageConfiguration<?> storageConf =
HoodieStorageUtils.getNewStorageConf();
Review Comment:
use `getDefaultStorageConf()`
##########
hudi-hadoop-common/src/test/resources/props/testdfs.properties:
##########
@@ -0,0 +1,18 @@
+
Review Comment:
nit: remove empty line
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
