[ https://issues.apache.org/jira/browse/KAFKA-6824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16785938#comment-16785938 ]
Matthias J. Sax commented on KAFKA-6824: ---------------------------------------- One more on `2.2`: [https://jenkins.confluent.io/job/apache-kafka-test/job/2.2/49/testReport/junit/kafka.server/DynamicBrokerReconfigurationTest/testAddRemoveSaslListeners/] {quote}java.util.concurrent.ExecutionException: java.lang.AssertionError: expected:<0> but was:<1> at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:206) at kafka.server.DynamicBrokerReconfigurationTest.verifyTimeout(DynamicBrokerReconfigurationTest.scala:1317) at kafka.server.DynamicBrokerReconfigurationTest.verifyConnectionFailure(DynamicBrokerReconfigurationTest.scala:1311) at kafka.server.DynamicBrokerReconfigurationTest.verifyRemoveListener(DynamicBrokerReconfigurationTest.scala:960) at kafka.server.DynamicBrokerReconfigurationTest.testAddRemoveSaslListeners(DynamicBrokerReconfigurationTest.scala:841){quote} STDOUT {quote}Completed Updating config for entity: brokers '0'. Completed Updating config for entity: brokers '1'. Completed Updating config for entity: brokers '2'. [2019-03-06 14:37:15,003] ERROR [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=1] Error for partition testtopic-2 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:37:15,003] ERROR [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=1] Error for partition testtopic-8 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:37:15,041] ERROR [ReplicaFetcher replicaId=1, leaderId=2, fetcherId=1] Error for partition testtopic-2 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:37:15,048] ERROR [ReplicaFetcher replicaId=1, leaderId=2, fetcherId=1] Error for partition testtopic-8 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:37:15,074] ERROR [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=0] Error for partition testtopic-5 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:37:15,074] ERROR [ReplicaFetcher replicaId=1, leaderId=2, fetcherId=0] Error for partition testtopic-5 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. Completed Updating config for entity: brokers '0'. Completed Updating config for entity: brokers '1'. Completed Updating config for entity: brokers '2'. [2019-03-06 14:37:28,893] ERROR [ReplicaFetcher replicaId=2, leaderId=0, fetcherId=0] Error for partition __consumer_offsets-3 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:37:28,899] ERROR [ReplicaFetcher replicaId=2, leaderId=1, fetcherId=0] Error for partition __consumer_offsets-1 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:37:28,901] ERROR [ReplicaFetcher replicaId=0, leaderId=1, fetcherId=0] Error for partition __consumer_offsets-1 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:37:28,901] ERROR [ReplicaFetcher replicaId=0, leaderId=1, fetcherId=1] Error for partition __consumer_offsets-4 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:37:33,312] WARN Unable to read additional data from client sessionid 0x10069a6e5f20004, likely client has closed socket (org.apache.zookeeper.server.NIOServerCnxn:376) Completed Updating config for entity: brokers '0'. Completed Updating config for entity: brokers '1'. Completed Updating config for entity: brokers '2'. [2019-03-06 14:37:39,387] ERROR [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=0] Error for partition testtopic-7 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:37:39,388] ERROR [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=0] Error for partition testtopic-1 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:37:43,536] WARN Unable to reconnect to ZooKeeper service, session 0x10069a712880002 has expired (org.apache.zookeeper.ClientCnxn:1289) [2019-03-06 14:37:45,287] WARN Unable to reconnect to ZooKeeper service, session 0x10069a579540003 has expired (org.apache.zookeeper.ClientCnxn:1289) [2019-03-06 14:37:45,289] ERROR [ZooKeeperClient] Uncaught error in handler kafka.zookeeper.ZooKeeperClientTest$$anon$7 (kafka.zookeeper.ZooKeeperClient:76) java.lang.RuntimeException at kafka.zookeeper.ZooKeeperClientTest$$anon$7.beforeInitializingSession(ZooKeeperClientTest.scala:366) at kafka.zookeeper.ZooKeeperClient.callBeforeInitializingSession(ZooKeeperClient.scala:382) at kafka.zookeeper.ZooKeeperClient.$anonfun$reinitialize$1(ZooKeeperClient.scala:348) at kafka.zookeeper.ZooKeeperClient.$anonfun$reinitialize$1$adapted(ZooKeeperClient.scala:348) at scala.collection.Iterator.foreach(Iterator.scala:941) at scala.collection.Iterator.foreach$(Iterator.scala:941) at scala.collection.AbstractIterator.foreach(Iterator.scala:1429) at scala.collection.MapLike$DefaultValuesIterable.foreach(MapLike.scala:213) at kafka.zookeeper.ZooKeeperClient.reinitialize(ZooKeeperClient.scala:348) at kafka.zookeeper.ZooKeeperClient.$anonfun$scheduleSessionExpiryHandler$1(ZooKeeperClient.scala:402) at kafka.utils.KafkaScheduler.$anonfun$schedule$2(KafkaScheduler.scala:114) at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:63) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) [2019-03-06 14:37:45,296] ERROR SASL authentication failed using login context 'Client' with exception: {} (org.apache.zookeeper.client.ZooKeeperSaslClient:247) javax.security.sasl.SaslException: Error in authenticating with a Zookeeper Quorum member: the quorum member's saslToken is null. at org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslToken(ZooKeeperSaslClient.java:279) at org.apache.zookeeper.client.ZooKeeperSaslClient.respondToServer(ZooKeeperSaslClient.java:242) at org.apache.zookeeper.ClientCnxn$SendThread.readResponse(ClientCnxn.java:805) at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:94) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1145) [2019-03-06 14:37:45,297] ERROR [ZooKeeperClient] Auth failed. (kafka.zookeeper.ZooKeeperClient:74) Completed Updating config for entity: brokers '0'. Completed Updating config for entity: brokers '1'. Completed Updating config for entity: brokers '2'. [2019-03-06 14:37:53,110] ERROR [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=1] Error for partition testtopic-2 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:37:53,111] ERROR [ReplicaFetcher replicaId=1, leaderId=0, fetcherId=1] Error for partition testtopic-8 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. Completed Updating config for entity: brokers '0'. Completed Updating config for entity: brokers '1'. Completed Updating config for entity: brokers '2'. [2019-03-06 14:38:10,946] ERROR [ReplicaFetcher replicaId=0, leaderId=1, fetcherId=0] Error for partition testtopic-3 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:10,946] ERROR [ReplicaFetcher replicaId=0, leaderId=1, fetcherId=0] Error for partition testtopic-9 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:10,965] ERROR [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=1] Error for partition testtopic-4 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:10,979] ERROR [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=0] Error for partition testtopic-7 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:10,979] ERROR [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=0] Error for partition testtopic-1 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:11,002] ERROR [ReplicaFetcher replicaId=0, leaderId=1, fetcherId=1] Error for partition testtopic-6 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:11,003] ERROR [ReplicaFetcher replicaId=0, leaderId=1, fetcherId=1] Error for partition testtopic-0 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. Completed Updating config for entity: user-principal 'scram-user'. Completed Updating config for entity: user-principal 'scram-admin'. Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt false ticketCache is null isInitiator true KeyTab is /tmp/kafka8342100398049352126.tmp refreshKrb5Config is false principal is kafka/localh...@example.com tryFirstPass is false useFirstPass is false storePass is false clearPass is false principal is kafka/localh...@example.com Will use keytab Commit Succeeded Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt false ticketCache is null isInitiator true KeyTab is /tmp/kafka8342100398049352126.tmp refreshKrb5Config is false principal is kafka/localh...@example.com tryFirstPass is false useFirstPass is false storePass is false clearPass is false principal is kafka/localh...@example.com Will use keytab Commit Succeeded [2019-03-06 14:38:20,143] ERROR [MetadataCache brokerId=1] Listeners are not identical across brokers: Map(0 -> Map(ListenerName(EXTERNAL) -> localhost:44557 (id: 0 rack: null), ListenerName(INTERNAL) -> localhost:38174 (id: 0 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:41395 (id: 0 rack: null)), 2 -> Map(ListenerName(EXTERNAL) -> localhost:37911 (id: 2 rack: null), ListenerName(INTERNAL) -> localhost:43657 (id: 2 rack: null)), 1 -> Map(ListenerName(EXTERNAL) -> localhost:33125 (id: 1 rack: null), ListenerName(INTERNAL) -> localhost:35100 (id: 1 rack: null))) (kafka.server.MetadataCache:74) [2019-03-06 14:38:20,143] ERROR [MetadataCache brokerId=2] Listeners are not identical across brokers: Map(0 -> Map(ListenerName(EXTERNAL) -> localhost:44557 (id: 0 rack: null), ListenerName(INTERNAL) -> localhost:38174 (id: 0 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:41395 (id: 0 rack: null)), 2 -> Map(ListenerName(EXTERNAL) -> localhost:37911 (id: 2 rack: null), ListenerName(INTERNAL) -> localhost:43657 (id: 2 rack: null)), 1 -> Map(ListenerName(EXTERNAL) -> localhost:33125 (id: 1 rack: null), ListenerName(INTERNAL) -> localhost:35100 (id: 1 rack: null))) (kafka.server.MetadataCache:74) [2019-03-06 14:38:20,143] ERROR [MetadataCache brokerId=0] Listeners are not identical across brokers: Map(0 -> Map(ListenerName(EXTERNAL) -> localhost:44557 (id: 0 rack: null), ListenerName(INTERNAL) -> localhost:38174 (id: 0 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:41395 (id: 0 rack: null)), 2 -> Map(ListenerName(EXTERNAL) -> localhost:37911 (id: 2 rack: null), ListenerName(INTERNAL) -> localhost:43657 (id: 2 rack: null)), 1 -> Map(ListenerName(EXTERNAL) -> localhost:33125 (id: 1 rack: null), ListenerName(INTERNAL) -> localhost:35100 (id: 1 rack: null))) (kafka.server.MetadataCache:74) [2019-03-06 14:38:20,158] ERROR [MetadataCache brokerId=2] Listeners are not identical across brokers: Map(0 -> Map(ListenerName(EXTERNAL) -> localhost:44557 (id: 0 rack: null), ListenerName(INTERNAL) -> localhost:38174 (id: 0 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:41395 (id: 0 rack: null)), 2 -> Map(ListenerName(EXTERNAL) -> localhost:37911 (id: 2 rack: null), ListenerName(INTERNAL) -> localhost:43657 (id: 2 rack: null)), 1 -> Map(ListenerName(EXTERNAL) -> localhost:33125 (id: 1 rack: null), ListenerName(INTERNAL) -> localhost:35100 (id: 1 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:38205 (id: 1 rack: null))) (kafka.server.MetadataCache:74) [2019-03-06 14:38:20,160] ERROR [MetadataCache brokerId=1] Listeners are not identical across brokers: Map(0 -> Map(ListenerName(EXTERNAL) -> localhost:44557 (id: 0 rack: null), ListenerName(INTERNAL) -> localhost:38174 (id: 0 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:41395 (id: 0 rack: null)), 2 -> Map(ListenerName(EXTERNAL) -> localhost:37911 (id: 2 rack: null), ListenerName(INTERNAL) -> localhost:43657 (id: 2 rack: null)), 1 -> Map(ListenerName(EXTERNAL) -> localhost:33125 (id: 1 rack: null), ListenerName(INTERNAL) -> localhost:35100 (id: 1 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:38205 (id: 1 rack: null))) (kafka.server.MetadataCache:74) [2019-03-06 14:38:20,159] ERROR [MetadataCache brokerId=0] Listeners are not identical across brokers: Map(0 -> Map(ListenerName(EXTERNAL) -> localhost:44557 (id: 0 rack: null), ListenerName(INTERNAL) -> localhost:38174 (id: 0 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:41395 (id: 0 rack: null)), 2 -> Map(ListenerName(EXTERNAL) -> localhost:37911 (id: 2 rack: null), ListenerName(INTERNAL) -> localhost:43657 (id: 2 rack: null)), 1 -> Map(ListenerName(EXTERNAL) -> localhost:33125 (id: 1 rack: null), ListenerName(INTERNAL) -> localhost:35100 (id: 1 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:38205 (id: 1 rack: null))) (kafka.server.MetadataCache:74) Debug is true storeKey true useTicketCache false useKeyTab true doNotPrompt false ticketCache is null isInitiator true KeyTab is /tmp/kafka4671719597430065611.tmp refreshKrb5Config is false principal is cli...@example.com tryFirstPass is false useFirstPass is false storePass is false clearPass is false principal is cli...@example.com Will use keytab Commit Succeeded [2019-03-06 14:38:21,869] ERROR [MetadataCache brokerId=2] Listeners are not identical across brokers: Map(0 -> Map(ListenerName(EXTERNAL) -> localhost:44557 (id: 0 rack: null), ListenerName(INTERNAL) -> localhost:38174 (id: 0 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:41395 (id: 0 rack: null)), 2 -> Map(ListenerName(EXTERNAL) -> localhost:37911 (id: 2 rack: null), ListenerName(INTERNAL) -> localhost:43657 (id: 2 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:36552 (id: 2 rack: null)), 1 -> Map(ListenerName(EXTERNAL) -> localhost:33125 (id: 1 rack: null), ListenerName(INTERNAL) -> localhost:35100 (id: 1 rack: null))) (kafka.server.MetadataCache:74) [2019-03-06 14:38:21,938] ERROR [MetadataCache brokerId=1] Listeners are not identical across brokers: Map(0 -> Map(ListenerName(EXTERNAL) -> localhost:44557 (id: 0 rack: null), ListenerName(INTERNAL) -> localhost:38174 (id: 0 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:41395 (id: 0 rack: null)), 2 -> Map(ListenerName(EXTERNAL) -> localhost:37911 (id: 2 rack: null), ListenerName(INTERNAL) -> localhost:43657 (id: 2 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:36552 (id: 2 rack: null)), 1 -> Map(ListenerName(EXTERNAL) -> localhost:33125 (id: 1 rack: null), ListenerName(INTERNAL) -> localhost:35100 (id: 1 rack: null))) (kafka.server.MetadataCache:74) [2019-03-06 14:38:21,942] ERROR [MetadataCache brokerId=0] Listeners are not identical across brokers: Map(0 -> Map(ListenerName(EXTERNAL) -> localhost:44557 (id: 0 rack: null), ListenerName(INTERNAL) -> localhost:38174 (id: 0 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:41395 (id: 0 rack: null)), 2 -> Map(ListenerName(EXTERNAL) -> localhost:37911 (id: 2 rack: null), ListenerName(INTERNAL) -> localhost:43657 (id: 2 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:36552 (id: 2 rack: null)), 1 -> Map(ListenerName(EXTERNAL) -> localhost:33125 (id: 1 rack: null), ListenerName(INTERNAL) -> localhost:35100 (id: 1 rack: null))) (kafka.server.MetadataCache:74) [2019-03-06 14:38:21,953] ERROR [MetadataCache brokerId=1] Listeners are not identical across brokers: Map(0 -> Map(ListenerName(EXTERNAL) -> localhost:44557 (id: 0 rack: null), ListenerName(INTERNAL) -> localhost:38174 (id: 0 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:41395 (id: 0 rack: null)), 2 -> Map(ListenerName(EXTERNAL) -> localhost:37911 (id: 2 rack: null), ListenerName(INTERNAL) -> localhost:43657 (id: 2 rack: null)), 1 -> Map(ListenerName(EXTERNAL) -> localhost:33125 (id: 1 rack: null), ListenerName(INTERNAL) -> localhost:35100 (id: 1 rack: null))) (kafka.server.MetadataCache:74) [2019-03-06 14:38:21,953] ERROR [MetadataCache brokerId=2] Listeners are not identical across brokers: Map(0 -> Map(ListenerName(EXTERNAL) -> localhost:44557 (id: 0 rack: null), ListenerName(INTERNAL) -> localhost:38174 (id: 0 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:41395 (id: 0 rack: null)), 2 -> Map(ListenerName(EXTERNAL) -> localhost:37911 (id: 2 rack: null), ListenerName(INTERNAL) -> localhost:43657 (id: 2 rack: null)), 1 -> Map(ListenerName(EXTERNAL) -> localhost:33125 (id: 1 rack: null), ListenerName(INTERNAL) -> localhost:35100 (id: 1 rack: null))) (kafka.server.MetadataCache:74) [2019-03-06 14:38:21,967] ERROR [MetadataCache brokerId=0] Listeners are not identical across brokers: Map(0 -> Map(ListenerName(EXTERNAL) -> localhost:44557 (id: 0 rack: null), ListenerName(INTERNAL) -> localhost:38174 (id: 0 rack: null), ListenerName(SASL_PLAINTEXT) -> localhost:41395 (id: 0 rack: null)), 2 -> Map(ListenerName(EXTERNAL) -> localhost:37911 (id: 2 rack: null), ListenerName(INTERNAL) -> localhost:43657 (id: 2 rack: null)), 1 -> Map(ListenerName(EXTERNAL) -> localhost:33125 (id: 1 rack: null), ListenerName(INTERNAL) -> localhost:35100 (id: 1 rack: null))) (kafka.server.MetadataCache:74) Completed Updating config for entity: brokers '0'. Completed Updating config for entity: brokers '1'. Completed Updating config for entity: brokers '2'. [2019-03-06 14:38:29,714] ERROR [ReplicaFetcher replicaId=0, leaderId=1, fetcherId=0] Error for partition testtopic-7 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:29,743] ERROR [ReplicaFetcher replicaId=0, leaderId=1, fetcherId=0] Error for partition testtopic-1 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:29,758] ERROR [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=0] Error for partition testtopic-5 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:29,806] ERROR [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=1] Error for partition testtopic-2 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:29,806] ERROR [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=1] Error for partition testtopic-8 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:29,850] ERROR [ReplicaFetcher replicaId=2, leaderId=1, fetcherId=1] Error for partition testtopic-4 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:29,855] ERROR [ReplicaFetcher replicaId=0, leaderId=1, fetcherId=1] Error for partition testtopic-4 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:31,676] ERROR [Producer clientId=test-client] Connection to node -3 (localhost/127.0.0.1:40987) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:38:31,677] ERROR [Producer clientId=test-client] Connection to node -1 (localhost/127.0.0.1:39689) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:38:31,677] ERROR [Producer clientId=test-client] Connection to node -2 (localhost/127.0.0.1:39458) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:38:32,232] ERROR [Producer clientId=test-client] Connection to node -3 (localhost/127.0.0.1:40987) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:38:37,137] ERROR [Producer clientId=test-client] Connection to node -3 (localhost/127.0.0.1:40987) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:38:37,169] ERROR [Producer clientId=test-client] Connection to node -1 (localhost/127.0.0.1:39689) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:38:37,171] ERROR [Producer clientId=test-client] Connection to node -2 (localhost/127.0.0.1:39458) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:38:37,316] ERROR [Producer clientId=test-client] Connection to node -1 (localhost/127.0.0.1:39689) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:38:37,318] ERROR [Producer clientId=test-client] Connection to node -2 (localhost/127.0.0.1:39458) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:38:37,318] ERROR [Producer clientId=test-client] Connection to node -3 (localhost/127.0.0.1:40987) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:38:37,684] ERROR [Producer clientId=test-client] Connection to node -3 (localhost/127.0.0.1:40987) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:38:37,818] ERROR [Producer clientId=test-client] Connection to node -1 (localhost/127.0.0.1:39689) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) Completed Updating config for entity: brokers '0'. Completed Updating config for entity: brokers '1'. Completed Updating config for entity: brokers '2'. [2019-03-06 14:38:45,395] WARN Unable to read additional data from client sessionid 0x10069a813070005, likely client has closed socket (org.apache.zookeeper.server.NIOServerCnxn:376) [2019-03-06 14:38:45,844] ERROR [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=1] Error for partition testtopic-4 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:45,862] ERROR [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=0] Error for partition testtopic-7 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:45,862] ERROR [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=0] Error for partition testtopic-1 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:45,894] ERROR [ReplicaFetcher replicaId=0, leaderId=1, fetcherId=0] Error for partition testtopic-3 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:45,895] ERROR [ReplicaFetcher replicaId=0, leaderId=1, fetcherId=0] Error for partition testtopic-9 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:45,896] ERROR [ReplicaFetcher replicaId=0, leaderId=1, fetcherId=1] Error for partition testtopic-6 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:45,896] ERROR [ReplicaFetcher replicaId=0, leaderId=1, fetcherId=1] Error for partition testtopic-0 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:45,896] ERROR [ReplicaFetcher replicaId=2, leaderId=1, fetcherId=1] Error for partition testtopic-6 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:45,897] ERROR [ReplicaFetcher replicaId=2, leaderId=1, fetcherId=1] Error for partition testtopic-0 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:45,946] ERROR [ReplicaFetcher replicaId=2, leaderId=1, fetcherId=0] Error for partition testtopic-3 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:45,947] ERROR [ReplicaFetcher replicaId=2, leaderId=1, fetcherId=0] Error for partition testtopic-9 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:38:48,911] ERROR [ReplicaFetcher replicaId=2, leaderId=1, fetcherId=1] Error for partition testtopic2-0 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. Completed Updating config for entity: brokers '0'. Completed Updating config for entity: brokers '1'. Completed Updating config for entity: brokers '2'. [2019-03-06 14:39:06,870] ERROR [ReplicaFetcher replicaId=2, leaderId=1, fetcherId=0] Error for partition testtopic-3 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:39:06,870] ERROR [ReplicaFetcher replicaId=2, leaderId=1, fetcherId=0] Error for partition testtopic-9 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:39:06,911] ERROR [ReplicaFetcher replicaId=2, leaderId=0, fetcherId=0] Error for partition testtopic-5 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:39:06,925] ERROR [ReplicaFetcher replicaId=2, leaderId=1, fetcherId=1] Error for partition testtopic-6 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:39:06,926] ERROR [ReplicaFetcher replicaId=2, leaderId=1, fetcherId=1] Error for partition testtopic-0 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:39:06,928] ERROR [ReplicaFetcher replicaId=2, leaderId=0, fetcherId=1] Error for partition testtopic-2 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:39:06,928] ERROR [ReplicaFetcher replicaId=2, leaderId=0, fetcherId=1] Error for partition testtopic-8 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:39:10,680] ERROR [Producer clientId=test-client] Connection to node -3 (localhost/127.0.0.1:40048) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:10,680] ERROR [Producer clientId=test-client] Connection to node -1 (localhost/127.0.0.1:33230) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:10,681] ERROR [Producer clientId=test-client] Connection to node -2 (localhost/127.0.0.1:40390) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:10,795] ERROR [Producer clientId=test-client] Connection to node -3 (localhost/127.0.0.1:40048) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:10,799] ERROR [Producer clientId=test-client] Connection to node -2 (localhost/127.0.0.1:40390) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:10,810] ERROR [Producer clientId=test-client] Connection to node -1 (localhost/127.0.0.1:33230) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) Completed updating config for broker: 0. [2019-03-06 14:39:11,177] ERROR [Producer clientId=test-client] Connection to node -2 (localhost/127.0.0.1:40390) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:11,294] ERROR [Producer clientId=test-client] Connection to node -3 (localhost/127.0.0.1:40048) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) Completed updating config for broker: 1. Completed updating config for broker: 2. [2019-03-06 14:39:11,831] ERROR [Producer clientId=test-client] Connection to node -3 (localhost/127.0.0.1:40048) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:11,981] ERROR [Producer clientId=test-client] Connection to node -3 (localhost/127.0.0.1:40048) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:11,991] ERROR [Producer clientId=test-client] Connection to node -1 (localhost/127.0.0.1:33230) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:12,104] ERROR [Producer clientId=test-client] Connection to node -2 (localhost/127.0.0.1:40390) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:12,216] ERROR [Producer clientId=test-client] Connection to node -1 (localhost/127.0.0.1:33230) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:12,226] ERROR [Producer clientId=test-client] Connection to node -2 (localhost/127.0.0.1:40390) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:12,492] ERROR [Producer clientId=test-client] Connection to node -3 (localhost/127.0.0.1:40048) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:12,605] ERROR [Producer clientId=test-client] Connection to node -2 (localhost/127.0.0.1:40390) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:12,732] ERROR [Producer clientId=test-client] Connection to node -1 (localhost/127.0.0.1:33230) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:13,354] ERROR [Producer clientId=test-client] Connection to node -3 (localhost/127.0.0.1:40048) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:13,686] ERROR [Producer clientId=test-client] Connection to node -2 (localhost/127.0.0.1:40390) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:13,972] ERROR [Producer clientId=test-client] Connection to node -1 (localhost/127.0.0.1:33230) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:14,283] ERROR [Producer clientId=test-client] Connection to node -3 (localhost/127.0.0.1:40048) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:14,838] ERROR [Producer clientId=test-client] Connection to node -2 (localhost/127.0.0.1:40390) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) [2019-03-06 14:39:15,167] ERROR [Producer clientId=test-client] Connection to node -1 (localhost/127.0.0.1:33230) failed authentication due to: SSL handshake failed (org.apache.kafka.clients.NetworkClient:714) Completed Updating config for entity: brokers '0'. Completed Updating config for entity: brokers '1'. Completed Updating config for entity: brokers '2'. Completed updating config for broker: 0. Completed updating config for broker: 1. Completed updating config for broker: 2. Completed Updating config for entity: brokers '0'. Completed Updating config for entity: brokers '1'. Completed Updating config for entity: brokers '2'. [2019-03-06 14:39:30,175] ERROR [ReplicaFetcher replicaId=0, leaderId=1, fetcherId=1] Error for partition testtopic-4 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:39:30,214] ERROR [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=1] Error for partition testtopic-2 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:39:30,214] ERROR [ReplicaFetcher replicaId=0, leaderId=2, fetcherId=1] Error for partition testtopic-8 at offset 0 (kafka.server.ReplicaFetcherThread:76) org.apache.kafka.common.errors.UnknownTopicOrPartitionException: This server does not host this topic-partition. [2019-03-06 14:39:33,036] ERROR [kafka-log-cleaner-thread-1]: Error due to (kafka.log.LogCleaner:76) java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at kafka.utils.ShutdownableThread.pause(ShutdownableThread.scala:69) at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:302) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82) [2019-03-06 14:39:33,047] ERROR [kafka-log-cleaner-thread-0]: Error due to (kafka.log.LogCleaner:76) java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at kafka.utils.ShutdownableThread.pause(ShutdownableThread.scala:69) at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:302) at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:82){quote} > Flaky Test DynamicBrokerReconfigurationTest#testAddRemoveSslListener > -------------------------------------------------------------------- > > Key: KAFKA-6824 > URL: https://issues.apache.org/jira/browse/KAFKA-6824 > Project: Kafka > Issue Type: Bug > Components: core, unit tests > Affects Versions: 2.2.0 > Reporter: Anna Povzner > Assignee: Rajini Sivaram > Priority: Critical > Labels: flaky-test > Fix For: 2.3.0, 2.2.1 > > > Observed two failures of this test (both in PR builds) :( > > *Failure #1: (JDK 7 and Scala 2.11 )* > *17:20:49* kafka.server.DynamicBrokerReconfigurationTest > > testAddRemoveSslListener FAILED > *17:20:49* java.lang.AssertionError: expected:<10> but was:<12> > *17:20:49* at org.junit.Assert.fail(Assert.java:88) > *17:20:49* at org.junit.Assert.failNotEquals(Assert.java:834) > *17:20:49* at org.junit.Assert.assertEquals(Assert.java:645) > *17:20:49* at org.junit.Assert.assertEquals(Assert.java:631) > *17:20:49* at > kafka.server.DynamicBrokerReconfigurationTest.verifyProduceConsume(DynamicBrokerReconfigurationTest.scala:959) > *17:20:49* at > kafka.server.DynamicBrokerReconfigurationTest.verifyRemoveListener(DynamicBrokerReconfigurationTest.scala:784) > *17:20:49* at > kafka.server.DynamicBrokerReconfigurationTest.testAddRemoveSslListener(DynamicBrokerReconfigurationTest.scala:705) > > *Failure #2: (JDK 8)* > *18:46:23* kafka.server.DynamicBrokerReconfigurationTest > > testAddRemoveSslListener FAILED > *18:46:23* java.util.concurrent.ExecutionException: > org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is > not the leader for that topic-partition. > *18:46:23* at > org.apache.kafka.clients.producer.internals.FutureRecordMetadata.valueOrError(FutureRecordMetadata.java:94) > *18:46:23* at > org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:77) > *18:46:23* at > org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:29) > *18:46:23* at > kafka.server.DynamicBrokerReconfigurationTest.$anonfun$verifyProduceConsume$3(DynamicBrokerReconfigurationTest.scala:953) > *18:46:23* at > scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:234) > *18:46:23* at scala.collection.Iterator.foreach(Iterator.scala:929) > *18:46:23* at scala.collection.Iterator.foreach$(Iterator.scala:929) > *18:46:23* at > scala.collection.AbstractIterator.foreach(Iterator.scala:1417) > *18:46:23* at > scala.collection.IterableLike.foreach(IterableLike.scala:71) > *18:46:23* at > scala.collection.IterableLike.foreach$(IterableLike.scala:70) > *18:46:23* at > scala.collection.AbstractIterable.foreach(Iterable.scala:54) > *18:46:23* at > scala.collection.TraversableLike.map(TraversableLike.scala:234) > *18:46:23* at > scala.collection.TraversableLike.map$(TraversableLike.scala:227) > *18:46:23* at > scala.collection.AbstractTraversable.map(Traversable.scala:104) > *18:46:23* at > kafka.server.DynamicBrokerReconfigurationTest.verifyProduceConsume(DynamicBrokerReconfigurationTest.scala:953) > *18:46:23* at > kafka.server.DynamicBrokerReconfigurationTest.verifyRemoveListener(DynamicBrokerReconfigurationTest.scala:816) > *18:46:23* at > kafka.server.DynamicBrokerReconfigurationTest.testAddRemoveSslListener(DynamicBrokerReconfigurationTest.scala:705) > *18:46:23* > *18:46:23* Caused by: > *18:46:23* > org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is > not the leader for that topic-partition. -- This message was sent by Atlassian JIRA (v7.6.3#76005)