stevenzwu commented on code in PR #4943:
URL: https://github.com/apache/iceberg/pull/4943#discussion_r889715728
##########
flink/v1.15/flink/src/test/java/org/apache/iceberg/flink/source/TestStreamingMonitorFunction.java:
##########
@@ -208,6 +210,67 @@ public void testCheckpointRestore() throws Exception {
}
}
+ @Test
+ public void testConsumeWithMaxSnapshotsPerMonitorInterval() throws Exception
{
+ List<List<Record>> recordsList = generateRecordsAndCommitTxn(10);
+
+ final ScanContext scanContext1 = ScanContext.builder()
+ .monitorInterval(Duration.ofMillis(100))
+ .maxSnapshotsPerMonitorInterval(0)
+ .build();
+
+ AssertHelpers.assertThrows("Should throw exception because of invalid
config",
+ IllegalArgumentException.class, "must be greater than zero",
+ () -> {
+ createFunction(scanContext1);
+ return null;
+ }
+ );
+
+ final ScanContext scanContext2 = ScanContext.builder()
+ .monitorInterval(Duration.ofMillis(100))
+ .maxSnapshotsPerMonitorInterval(-10)
+ .build();
+
+ AssertHelpers.assertThrows("Should throw exception because of invalid
config",
+ IllegalArgumentException.class, "must be greater than zero",
+ () -> {
+ createFunction(scanContext2);
+ return null;
+ }
+ );
+
+ for (int maxSnapshotsNum = 1; maxSnapshotsNum < 15; maxSnapshotsNum =
maxSnapshotsNum + 1) {
+ ScanContext scanContext = ScanContext.builder()
+ .monitorInterval(Duration.ofMillis(100))
+ .splitSize(1000L)
+ .maxSnapshotsPerMonitorInterval(maxSnapshotsNum)
+ .build();
+
+ FlinkInputSplit[] expectedSplits = FlinkSplitPlanner
Review Comment:
I think we should directly define the number of expected splits and avoid
using `FlinkSplitPlanner`. if `FlinkSplitPlanner` is not honoring
`maxSnapshotsPerMonitorInterval`, the assertion later will still be correct
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
