the-other-tim-brown commented on code in PR #11152:
URL: https://github.com/apache/hudi/pull/11152#discussion_r1625155568
##########
hudi-client/hudi-client-common/src/test/java/org/apache/hudi/testutils/Assertions.java:
##########
@@ -51,4 +67,88 @@ public static void assertFileSizesEqual(List<WriteStatus>
statuses, CheckedFunct
assertEquals(fileSizeGetter.apply(status),
status.getStat().getFileSizeInBytes())));
}
+ public static void assertPartitionMetadataForRecords(String basePath,
List<HoodieRecord> inputRecords,
+ HoodieStorage storage) throws
IOException {
+ Set<String> partitionPathSet = inputRecords.stream()
+ .map(HoodieRecord::getPartitionPath)
+ .collect(Collectors.toSet());
+ assertPartitionMetadata(basePath,
partitionPathSet.stream().toArray(String[]::new), storage);
Review Comment:
For this line and 83, you can simplify this by not collecting to a set and
just use `distinct()` on the stream
##########
hudi-client/hudi-client-common/src/test/java/org/apache/hudi/utils/HoodieWriterClientTestHarness.java:
##########
@@ -165,71 +247,1183 @@ public HoodieWriteConfig.Builder
getConfigBuilder(String schemaStr, HoodieIndex.
return builder;
}
- public void assertPartitionMetadataForRecords(String basePath,
List<HoodieRecord> inputRecords,
- HoodieStorage storage) throws
IOException {
- Set<String> partitionPathSet = inputRecords.stream()
- .map(HoodieRecord::getPartitionPath)
- .collect(Collectors.toSet());
- assertPartitionMetadata(basePath,
partitionPathSet.stream().toArray(String[]::new), storage);
+ // Functional Interfaces for passing lambda and Hoodie Write API contexts
+
+ @FunctionalInterface
+ public interface Function2<R, T1, T2> {
+
+ R apply(T1 v1, T2 v2) throws IOException;
+ }
+
+ @FunctionalInterface
+ public interface Function3<R, T1, T2, T3> {
+
+ R apply(T1 v1, T2 v2, T3 v3) throws IOException;
+ }
+
+ /* Auxiliary methods for testing CopyOnWriteStorage with Spark and Java
clients
+ to avoid code duplication in TestHoodieClientOnCopyOnWriteStorage and
TestHoodieJavaClientOnCopyOnWriteStorage */
+
+ protected List<WriteStatus> writeAndVerifyBatch(BaseHoodieWriteClient
client, List<HoodieRecord> inserts, String commitTime, boolean
populateMetaFields, boolean autoCommitOff) throws IOException {
+ // override in subclasses if needed
+ return Collections.emptyList();
Review Comment:
Should this just be abstract? Returning empty list by default may be
misleading to other developers in the future that extend this class
##########
hudi-client/hudi-client-common/src/test/java/org/apache/hudi/utils/HoodieWriterClientTestHarness.java:
##########
@@ -165,71 +247,1183 @@ public HoodieWriteConfig.Builder
getConfigBuilder(String schemaStr, HoodieIndex.
return builder;
}
- public void assertPartitionMetadataForRecords(String basePath,
List<HoodieRecord> inputRecords,
- HoodieStorage storage) throws
IOException {
- Set<String> partitionPathSet = inputRecords.stream()
- .map(HoodieRecord::getPartitionPath)
- .collect(Collectors.toSet());
- assertPartitionMetadata(basePath,
partitionPathSet.stream().toArray(String[]::new), storage);
+ // Functional Interfaces for passing lambda and Hoodie Write API contexts
+
+ @FunctionalInterface
+ public interface Function2<R, T1, T2> {
Review Comment:
There is already a BiFunction in java that does the same thing, can we just
use that?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
