geserdugarov commented on code in PR #12796:
URL: https://github.com/apache/hudi/pull/12796#discussion_r1962879159
##########
hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/bucket/ITTestBucketStreamWrite.java:
##########
@@ -91,7 +97,20 @@ public void
testBucketStreamWriteAfterRollbackFirstFileGroupCreation(boolean isC
}
}
- private static void doDeleteCommit(String tablePath, boolean isCow) throws
Exception {
+ @ParameterizedTest
+ @ValueSource(booleans = {true, false})
+ public void testBucketWriteIntoMultipleBuckets(boolean isCow) throws
Exception {
+ String tablePath = tempFile.getAbsolutePath();
+ doWrite(tablePath, isCow, 2, "upsert");
+ if (isCow) {
+ TestData.checkWrittenDataCOW(tempFile, EXPECTED_LIST);
Review Comment:
`TestData::checkWrittenData` doesn't expect data with buckets, when we have
multiple base files in one partition. So
`ITTestBucketStreamWrite::testBucketStreamWriteAfterRollbackFirstFileGroupCreation`
works only because `bucketNum` is set to 1. If we will change it to 2, then
this test will fail:
https://github.com/apache/hudi/blob/ec044790fe71f04710472ad63e7859f1b29e3985/hudi-flink-datasource/hudi-flink/src/test/java/org/apache/hudi/sink/bucket/ITTestBucketStreamWrite.java#L77-L92
I suppose we need to fix it with revisiting of `TestData::checkWrittenData`,
`TestData::checkWrittenDataCOW`, `TestData::checkWrittenDataMOR` to prevent
such mistakes in future. I could work on it later. I've created a corresponding
task [HUDI-9056](https://issues.apache.org/jira/browse/HUDI-9056).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
