[jira] [Commented] (KAFKA-8011) Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated
[ https://issues.apache.org/jira/browse/KAFKA-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118262#comment-17118262 ] Matthias J. Sax commented on KAFKA-8011: We can also create a new ticket. I don't care. > Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated > > > Key: KAFKA-8011 > URL: https://issues.apache.org/jira/browse/KAFKA-8011 > Project: Kafka > Issue Type: Bug > Components: streams >Reporter: Bill Bejeck >Assignee: Bill Bejeck >Priority: Blocker > Labels: flaky-test, newbie > Fix For: 1.0.3, 1.1.2, 2.2.0, 2.0.2, 2.1.2, 2.6.0 > > Attachments: > org.apache.kafka.streams.integration.RegexSourceIntegrationTest.html, > streams_1_0_test_results.png, streams_1_1_tests.png > > > The RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated > and RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenDeleted tests use > an ArrayList to assert the topics assigned to the Streams application. > The ConsumerRebalanceListener used in the test operates on this list as does > the TestUtils.waitForCondition() to verify the expected topic assignments. > Using the same list in both places can cause a ConcurrentModficationException > if the rebalance listener modifies the assignment at the same time > TestUtils.waitForCondition() is using the list to verify the expected topics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-8011) Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated
[ https://issues.apache.org/jira/browse/KAFKA-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118245#comment-17118245 ] Sophie Blee-Goldman commented on KAFKA-8011: Attaching the output from local testing: [^org.apache.kafka.streams.integration.RegexSourceIntegrationTest.html] > Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated > > > Key: KAFKA-8011 > URL: https://issues.apache.org/jira/browse/KAFKA-8011 > Project: Kafka > Issue Type: Bug > Components: streams >Reporter: Bill Bejeck >Assignee: Bill Bejeck >Priority: Blocker > Labels: flaky-test, newbie > Fix For: 1.0.3, 1.1.2, 2.2.0, 2.0.2, 2.1.2, 2.6.0 > > Attachments: > org.apache.kafka.streams.integration.RegexSourceIntegrationTest.html, > streams_1_0_test_results.png, streams_1_1_tests.png > > > The RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated > and RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenDeleted tests use > an ArrayList to assert the topics assigned to the Streams application. > The ConsumerRebalanceListener used in the test operates on this list as does > the TestUtils.waitForCondition() to verify the expected topic assignments. > Using the same list in both places can cause a ConcurrentModficationException > if the rebalance listener modifies the assignment at the same time > TestUtils.waitForCondition() is using the list to verify the expected topics. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (KAFKA-8011) Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated
[
https://issues.apache.org/jira/browse/KAFKA-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118222#comment-17118222
]
Sophie Blee-Goldman commented on KAFKA-8011:
[~mjsax] should we create a separate ticket for that test? AFAICT the test this
ticket is for is only failing because that test is failing and the cleanup
isn't done properly. But the test you pointed out seems to be a real bug:
{code:java}
[2020-05-27 17:56:00,413] ERROR Exception caught during Deserialization,
taskId: 0_0, topic: TEST-TOPIC-1, partition: 0, offset: 0
(org.apache.kafka.streams.errors.LogAndFailExceptionHandler:39)
java.lang.NullPointerException
at
org.apache.kafka.streams.processor.internals.SourceNode.deserializeKey(SourceNode.java:58)
at
org.apache.kafka.streams.processor.internals.RecordDeserializer.deserialize(RecordDeserializer.java:65)
at
org.apache.kafka.streams.processor.internals.RecordQueue.updateHead(RecordQueue.java:176)
at
org.apache.kafka.streams.processor.internals.RecordQueue.addRawRecords(RecordQueue.java:112)
at
org.apache.kafka.streams.processor.internals.PartitionGroup.addRawRecords(PartitionGroup.java:185)
at
org.apache.kafka.streams.processor.internals.StreamTask.addRecords(StreamTask.java:844)
at
org.apache.kafka.streams.processor.internals.StreamThread.addRecordsToTasks(StreamThread.java:836)
at
org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:646)
at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:550)
at
org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:509)
{code}
I also later see
{code:java}
[2020-05-27 17:57:23,145] ERROR stream-thread
[regex-source-integration-test-9e3e325f-3fe3-4741-9862-eff246a8fb2a-StreamThread-1]
Encountered the following exception during processing and the thread is going
to shut down: (org.apache.kafka.streams.processor.internals.StreamThread:529)
org.apache.kafka.streams.errors.TopologyException: Invalid topology: Topic foo
is already matched for another regex pattern foo.* and hence cannot be matched
to this regex pattern f.* any more. at
org.apache.kafka.streams.processor.internals.InternalTopologyBuilder$SourceNodeFactory.getTopics(InternalTopologyBuilder.java:261)
at
org.apache.kafka.streams.processor.internals.InternalTopologyBuilder.setRegexMatchedTopicsToSourceNodes(InternalTopologyBuilder.java:1122)
at
org.apache.kafka.streams.processor.internals.InternalTopologyBuilder.updateSubscribedTopics(InternalTopologyBuilder.java:1983)
at
org.apache.kafka.streams.processor.internals.InternalTopologyBuilder.addSubscribedTopicsFromMetadata(InternalTopologyBuilder.java:1973)
at
org.apache.kafka.streams.processor.internals.TaskManager.handleRebalanceStart(TaskManager.java:136)
at
org.apache.kafka.streams.processor.internals.StreamsPartitionAssignor.handleRebalanceStart(StreamsPartitionAssignor.java:1585)
at
org.apache.kafka.streams.processor.internals.StreamsPartitionAssignor.subscriptionUserData(StreamsPartitionAssignor.java:232)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.metadata(ConsumerCoordinator.java:222)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.sendJoinGroupRequest(AbstractCoordinator.java:561)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.initiateJoinGroup(AbstractCoordinator.java:496)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:418)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:359)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:506)
at
org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1265)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1231)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1206)
at
org.apache.kafka.streams.processor.internals.StreamThread.pollRequests(StreamThread.java:770)
at
org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:630)
at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:550)
at
org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:509)
{code}
> Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated
>
>
> Key: KAFKA-8011
> URL: https://issues.apache.org/jira/browse/KAFKA-8011
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bill Bejeck
>Assignee: Bill Bejeck
>Priority:
[jira] [Commented] (KAFKA-8011) Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated
[
https://issues.apache.org/jira/browse/KAFKA-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17118098#comment-17118098
]
Matthias J. Sax commented on KAFKA-8011:
Different test method:
[https://builds.apache.org/job/kafka-pr-jdk8-scala2.12/2533/testReport/junit/org.apache.kafka.streams.integration/RegexSourceIntegrationTest/testRegexRecordsAreProcessedAfterReassignment/]
{quote}java.lang.AssertionError: Condition not met within timeout 6. Did
not receive all [KeyValue(1, 1), KeyValue(2, 2)] records from topic
outputTopic_1 at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:26)
at
org.apache.kafka.test.TestUtils.lambda$waitForCondition$5(TestUtils.java:381)
at
org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:429)
at
org.apache.kafka.test.TestUtils.retryOnExceptionWithTimeout(TestUtils.java:397)
at org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:378) at
org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:368) at
org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilFinalKeyValueRecordsReceived(IntegrationTestUtils.java:704)
at
org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilFinalKeyValueRecordsReceived(IntegrationTestUtils.java:658)
at
org.apache.kafka.streams.integration.utils.IntegrationTestUtils.waitUntilFinalKeyValueRecordsReceived(IntegrationTestUtils.java:619)
at
org.apache.kafka.streams.integration.RegexSourceIntegrationTest.testRegexRecordsAreProcessedAfterReassignment(RegexSourceIntegrationTest.java:214){quote}
> Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated
>
>
> Key: KAFKA-8011
> URL: https://issues.apache.org/jira/browse/KAFKA-8011
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bill Bejeck
>Assignee: Bill Bejeck
>Priority: Critical
> Labels: flaky-test, newbie
> Fix For: 1.0.3, 1.1.2, 2.2.0, 2.0.2, 2.1.2
>
> Attachments: streams_1_0_test_results.png, streams_1_1_tests.png
>
>
> The RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated
> and RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenDeleted tests use
> an ArrayList to assert the topics assigned to the Streams application.
> The ConsumerRebalanceListener used in the test operates on this list as does
> the TestUtils.waitForCondition() to verify the expected topic assignments.
> Using the same list in both places can cause a ConcurrentModficationException
> if the rebalance listener modifies the assignment at the same time
> TestUtils.waitForCondition() is using the list to verify the expected topics.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-8011) Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated
[ https://issues.apache.org/jira/browse/KAFKA-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781883#comment-16781883 ] Matthias J. Sax commented on KAFKA-8011: Done: https://issues.apache.org/jira/browse/KAFKA-8026 > Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated > > > Key: KAFKA-8011 > URL: https://issues.apache.org/jira/browse/KAFKA-8011 > Project: Kafka > Issue Type: Bug > Components: streams >Reporter: Bill Bejeck >Assignee: Bill Bejeck >Priority: Critical > Labels: flaky-test, newbie > Attachments: streams_1_0_test_results.png, streams_1_1_tests.png > > > The RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated > and RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenDeleted tests use > an ArrayList to assert the topics assigned to the Streams application. > The ConsumerRebalanceListener used in the test operates on this list as does > the TestUtils.waitForCondition() to verify the expected topic assignments. > Using the same list in both places can cause a ConcurrentModficationException > if the rebalance listener modifies the assignment at the same time > TestUtils.waitForCondition() is using the list to verify the expected topics. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-8011) Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated
[ https://issues.apache.org/jira/browse/KAFKA-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781880#comment-16781880 ] Matthias J. Sax commented on KAFKA-8011: Your commit might be unrelated. It's older branches that don't run regularly and longer. I'll just create a new ticket for it. Thanks! > Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated > > > Key: KAFKA-8011 > URL: https://issues.apache.org/jira/browse/KAFKA-8011 > Project: Kafka > Issue Type: Bug > Components: streams >Reporter: Bill Bejeck >Assignee: Bill Bejeck >Priority: Critical > Labels: flaky-test, newbie > Attachments: streams_1_0_test_results.png, streams_1_1_tests.png > > > The RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated > and RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenDeleted tests use > an ArrayList to assert the topics assigned to the Streams application. > The ConsumerRebalanceListener used in the test operates on this list as does > the TestUtils.waitForCondition() to verify the expected topic assignments. > Using the same list in both places can cause a ConcurrentModficationException > if the rebalance listener modifies the assignment at the same time > TestUtils.waitForCondition() is using the list to verify the expected topics. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-8011) Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated
[ https://issues.apache.org/jira/browse/KAFKA-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781755#comment-16781755 ] Bill Bejeck commented on KAFKA-8011: I've been able to verify that both 1.0 and 1.1 tests pass locally. If there is a failure in a subsequent build should I revert the cherry-pick? > Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated > > > Key: KAFKA-8011 > URL: https://issues.apache.org/jira/browse/KAFKA-8011 > Project: Kafka > Issue Type: Bug > Components: streams >Reporter: Bill Bejeck >Assignee: Bill Bejeck >Priority: Critical > Labels: flaky-test, newbie > Attachments: streams_1_0_test_results.png, streams_1_1_tests.png > > > The RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated > and RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenDeleted tests use > an ArrayList to assert the topics assigned to the Streams application. > The ConsumerRebalanceListener used in the test operates on this list as does > the TestUtils.waitForCondition() to verify the expected topic assignments. > Using the same list in both places can cause a ConcurrentModficationException > if the rebalance listener modifies the assignment at the same time > TestUtils.waitForCondition() is using the list to verify the expected topics. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-8011) Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated
[
https://issues.apache.org/jira/browse/KAFKA-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781387#comment-16781387
]
Matthias J. Sax commented on KAFKA-8011:
It seems the cherry-pick broke older branches. The following happened in 1.0
and 1.1
[https://builds.apache.org/blue/organizations/jenkins/kafka-1.0-jdk7/detail/kafka-1.0-jdk7/263/tests]
[https://builds.apache.org/blue/organizations/jenkins/kafka-1.1-jdk7/detail/kafka-1.1-jdk7/249/tests]
STDOUT:
{quote}Exception in thread
"regex-source-integration-test-f12281d4-a87a-4612-b2d0-34200123c1b7-StreamThread-128"
org.apache.kafka.streams.errors.TopologyException: Invalid topology: Topic foo
is already matched for another regex pattern foo.* and hence cannot be matched
to this regex pattern f.* any more.
at
org.apache.kafka.streams.processor.internals.InternalTopologyBuilder$SourceNodeFactory.getTopics(InternalTopologyBuilder.java:310)
at
org.apache.kafka.streams.processor.internals.InternalTopologyBuilder.setRegexMatchedTopicsToSourceNodes(InternalTopologyBuilder.java:1087)
at
org.apache.kafka.streams.processor.internals.InternalTopologyBuilder.updateSubscriptions(InternalTopologyBuilder.java:1263)
at
org.apache.kafka.streams.processor.internals.StreamPartitionAssignor.updateSubscribedTopics(StreamPartitionAssignor.java:292)
at
org.apache.kafka.streams.processor.internals.StreamPartitionAssignor.subscription(StreamPartitionAssignor.java:281)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.metadata(ConsumerCoordinator.java:152)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.sendJoinGroupRequest(AbstractCoordinator.java:443)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.initiateJoinGroup(AbstractCoordinator.java:402)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:363)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:316)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:295)
at
org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1146)
at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:)
at
org.apache.kafka.streams.processor.internals.StreamThread.pollRequests(StreamThread.java:848)
at
org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:805)
at
org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:771)
at
org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:741){quote}
Can you check [~bbejeck]? Did you run the tests locally on both branches before
pushing the cherry-pick?
> Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated
>
>
> Key: KAFKA-8011
> URL: https://issues.apache.org/jira/browse/KAFKA-8011
> Project: Kafka
> Issue Type: Bug
> Components: streams
>Reporter: Bill Bejeck
>Assignee: Bill Bejeck
>Priority: Critical
> Labels: flaky-test, newbie
>
> The RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated
> and RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenDeleted tests use
> an ArrayList to assert the topics assigned to the Streams application.
> The ConsumerRebalanceListener used in the test operates on this list as does
> the TestUtils.waitForCondition() to verify the expected topic assignments.
> Using the same list in both places can cause a ConcurrentModficationException
> if the rebalance listener modifies the assignment at the same time
> TestUtils.waitForCondition() is using the list to verify the expected topics.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KAFKA-8011) Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated
[ https://issues.apache.org/jira/browse/KAFKA-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781105#comment-16781105 ] ASF GitHub Bot commented on KAFKA-8011: --- bbejeck commented on pull request #6338: KAFKA-8011: Fix for race condition causing ConcurrentModificationException URL: https://github.com/apache/kafka/pull/6338 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated > > > Key: KAFKA-8011 > URL: https://issues.apache.org/jira/browse/KAFKA-8011 > Project: Kafka > Issue Type: Bug > Components: streams >Reporter: Bill Bejeck >Assignee: Bill Bejeck >Priority: Critical > Labels: flaky-test, newbie > > The RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated > and RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenDeleted tests use > an ArrayList to assert the topics assigned to the Streams application. > The ConsumerRebalanceListener used in the test operates on this list as does > the TestUtils.waitForCondition() to verify the expected topic assignments. > Using the same list in both places can cause a ConcurrentModficationException > if the rebalance listener modifies the assignment at the same time > TestUtils.waitForCondition() is using the list to verify the expected topics. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-8011) Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated
[ https://issues.apache.org/jira/browse/KAFKA-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16779605#comment-16779605 ] ASF GitHub Bot commented on KAFKA-8011: --- bbejeck commented on pull request #6338: KAFKA-8011: Fix for race condition causing ConcurrentModificationException URL: https://github.com/apache/kafka/pull/6338 In the `RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated()` and `RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenDeleted()` a race condition exists where the `ConsumerRebalanceListener` in the test modifies the list of subscribed topics when the condition for the test success is comparing the same array instance against expected values. This PR should fix this race condition by using a `CopyOnWriteArrayList` which guarantees safe traversal of the list even when a concurrent modification is happening. Using the `CopyOnWriteArrayList` should not impact performance negatively as the number of traversals, a result of using `ArrayList.equals()`, far outnumber (`TestUtils.waitForCondition()` checks for a successful result every`100ms`) the possible modifications as there will be at most one topic name added/removed during the test. For testing, I updated the `RegexSourceIntegrationTest`integration test and ran the suite of streams tests. ### Committer Checklist (excluded from commit message) - [ ] Verify design and implementation - [ ] Verify test coverage and CI build status - [ ] Verify documentation (including upgrade notes) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated > > > Key: KAFKA-8011 > URL: https://issues.apache.org/jira/browse/KAFKA-8011 > Project: Kafka > Issue Type: Bug > Components: streams >Reporter: Bill Bejeck >Priority: Critical > Labels: flaky-test, newbie > > The RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated > and RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenDeleted tests use > an ArrayList to assert the topics assigned to the Streams application. > The ConsumerRebalanceListener used in the test operates on this list as does > the TestUtils.waitForCondition() to verify the expected topic assignments. > Using the same list in both places can cause a ConcurrentModficationException > if the rebalance listener modifies the assignment at the same time > TestUtils.waitForCondition() is using the list to verify the expected topics. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (KAFKA-8011) Flaky Test RegexSourceIntegrationTest#testRegexMatchesTopicsAWhenCreated
[
https://issues.apache.org/jira/browse/KAFKA-8011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16779472#comment-16779472
]
Bill Bejeck commented on KAFKA-8011:
Here's the link to the build
[https://builds.apache.org/job/kafka-pr-jdk11-scala2.12/2730/]
Stacktrace
{noformat}
java.util.ConcurrentModificationException
at
java.base/java.util.ArrayList.checkForComodification(ArrayList.java:604)
at java.base/java.util.ArrayList.equals(ArrayList.java:563)
at
org.apache.kafka.streams.integration.RegexSourceIntegrationTest.lambda$testRegexMatchesTopicsAWhenDeleted$3(RegexSourceIntegrationTest.java:213)
at org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:355)
at org.apache.kafka.test.TestUtils.waitForCondition(TestUtils.java:325)
at
org.apache.kafka.streams.integration.RegexSourceIntegrationTest.testRegexMatchesTopicsAWhenDeleted(RegexSourceIntegrationTest.java:213)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305)
at
org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:365)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
at org.junit.runners.ParentRunner$4.run(ParentRunner.java:330)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:78)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:328)
at org.junit.runners.ParentRunner.access$100(ParentRunner.java:65)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:292)
at
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:305)
at org.junit.runners.ParentRunner.run(ParentRunner.java:412)
at
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
at
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
at
org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
at
org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
at
org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at
org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at
org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
at
org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
at com.sun.proxy.$Proxy2.processTestClass(Unknown Source)
at
org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:118)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
