EdwinIngJ opened a new pull request, #18627: URL: https://github.com/apache/druid/pull/18627
I also found some tests in `KafkaSupervisorTest.java` that nondeterministically fail similar to [PR](https://github.com/apache/druid/pull/18617). Specifically, the following tests: - org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorTest.testMultiTask - org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorTest.testReportWhenMultipleActiveTasks ### Description Here is a sample of an error from running **org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorTest.testMultiTask**: ``` [ERROR] org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorTest.testMultiTask[numThreads = 1] -- Time elapsed: 2.178 s <<< FAILURE! java.lang.AssertionError: expected:<2> but was:<1> at org.junit.Assert.fail(Assert.java:89) at org.junit.Assert.failNotEquals(Assert.java:835) at org.junit.Assert.assertEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:633) at org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorTest.testMultiTask(KafkaSupervisorTest.java:636) ``` **Problem**: Similar to the `testMultiTask` in this [PR](https://github.com/apache/druid/pull/18617), this test assumes a specific ordering of the `KafkaIndexTask` returned by the capture. The ordering of the elements is not guaranteed to have a defined order but the tests assume a specific order. **Proposed Changes**: Before the assertions, the list is sorted according to `KafkaIndexTask::getId`, ensuring that the ordering is always the same. Here is a sample of an error from running **org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorTest.testReportWhenMultipleActiveTasks**: ``` [ERROR] org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorTest.testReportWhenMultipleActiveTasks[numThreads = 1] -- Time elapsed: 2.723 s <<< FAILURE! org.junit.ComparisonFailure: expected:<id[2]> but was:<id[1]> at org.junit.Assert.assertEquals(Assert.java:117) at org.junit.Assert.assertEquals(Assert.java:146) at org.apache.druid.indexing.kafka.supervisor.KafkaSupervisorTest.testReportWhenMultipleActiveTasks(KafkaSupervisorTest.java:2466) ``` **Problem**: This test occasionally fails due to nondeterministic behavior in how the Kafka supervisor creates and reports active tasks. The ordering depends on the ordering of `activelyReadingTaskGroups` ConcurrentHashMap in `SeekableStreamSupervisor.java`. Unordered iteration of `activelyReadingTaskGroups` in `generateReport` function on line [1396](https://github.com/apache/druid/blob/42c7b062d8aec9f732a2a0b1d48da9a047774469/indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java#L1396) contributes to these failures as the test assumes the `TaskReportData` appear in a specific order. **Proposed Changes**: The `TaskReportData` is sorted according to `TaskReportData::getId`, ensuring that the ordering is always the same. --- This PR has: - [x] been self-reviewed. - [x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
