racorn opened a new issue #8128: URL: https://github.com/apache/pulsar/issues/8128
**Describe the bug** As title says. I think I had to ask `pulsarbot` to `run-failure-checks` four times before cpp-tests succeeded. I see that the checks for many other unrelated PRs fail in the same manner: **Sample log** ``` 2020-09-24 09:36:35.063 INFO [139999495969408] ConnectionPool:85 | Created connection for pulsar://localhost:6650 2020-09-24 09:36:35.073 INFO [139999344776960] ClientConnection:344 | [127.0.0.1:36914 -> 127.0.0.1:6650] Connected to broker 2020-09-24 09:36:35.075 INFO [139999344776960] HandlerBase:53 | [persistent://public/default/1600940195testIndividualAck, ] Getting connection from pool 2020-09-24 09:36:35.096 INFO [139999344776960] ProducerImpl:164 | [persistent://public/default/1600940195testIndividualAck, ] Created producer on broker [127.0.0.1:36914 -> 127.0.0.1:6650] 2020-09-24 09:36:35.096 INFO [139999495969408] Client:88 | Subscribing on Topic :persistent://public/default/1600940195testIndividualAck 2020-09-24 09:36:35.097 INFO [139999344776960] HandlerBase:53 | [persistent://public/default/1600940195testIndividualAck, subscription-name, 0] Getting connection from pool 2020-09-24 09:36:35.105 INFO [139999344776960] ConsumerImpl:212 | [persistent://public/default/1600940195testIndividualAck, subscription-name, 0] Created consumer on broker [127.0.0.1:36914 -> 127.0.0.1:6650] 2020-09-24 09:36:35.106 INFO [139999495969408] ConsumerImpl:879 | [persistent://public/default/1600940195testIndividualAck, subscription-name, 0] Closing consumer for topic persistent://public/default/1600940195testIndividualAck 2020-09-24 09:36:35.107 INFO [139999344776960] ConsumerImpl:933 | [persistent://public/default/1600940195testIndividualAck, subscription-name, 0] Closed consumer 0 2020-09-24 09:36:35.107 INFO [139999495969408] Client:88 | Subscribing on Topic :persistent://public/default/1600940195testIndividualAck 2020-09-24 09:36:35.111 INFO [139999344776960] HandlerBase:53 | [persistent://public/default/1600940195testIndividualAck, subscription-name, 1] Getting connection from pool 2020-09-24 09:36:35.120 INFO [139999344776960] ConsumerImpl:212 | [persistent://public/default/1600940195testIndividualAck, subscription-name, 1] Created consumer on broker [127.0.0.1:36914 -> 127.0.0.1:6650] 2020-09-24 09:36:36.075 INFO [139999344776960] ProducerStatsImpl:92 | Producer [persistent://public/default/1600940195testIndividualAck, ] , ProducerStatsImpl (numMsgsSent_ = 10, numBytesSent_ = 280, sendMap_ = {[Key: Ok, Value: 10], }, latencyAccumulator_ = Latencies [ 50pct: 4.56ms, 90pct: 4.542ms, 99pct: 2.787ms, 99.9pct: 2.729ms], totalMsgsSent_ = 10, totalBytesSent_ = 280, totalAcksReceived_ = , totalSendMap_ = {[Key: Ok, Value: 10], }, totalLatencyAccumulator_ = Latencies [ 50pct: 4.56ms, 90pct: 4.542ms, 99pct: 2.787ms, 99.9pct: 2.729ms]) 2020-09-24 09:36:36.097 INFO [139999344776960] ConsumerStatsImpl:65 | Consumer [persistent://public/default/1600940195testIndividualAck, subscription-name, 0] , ConsumerStatsImpl (numBytesRecieved_ = 0, totalNumBytesRecieved_ = 0, receivedMsgMap_ = {}, ackedMsgMap_ = {}, totalReceivedMsgMap_ = {}, totalAckedMsgMap_ = {}) 2020-09-24 09:36:36.111 INFO [139999344776960] ConsumerStatsImpl:65 | Consumer [persistent://public/default/1600940195testIndividualAck, subscription-name, 1] , ConsumerStatsImpl (numBytesRecieved_ = 280, totalNumBytesRecieved_ = 280, receivedMsgMap_ = {[Key: Ok, Value: 10], }, ackedMsgMap_ = {[Key: {Result: Ok, ackType: 0}, Value: 5], }, totalReceivedMsgMap_ = {[Key: Ok, Value: 10], }, totalAckedMsgMap_ = {[Key: {Result: Ok, ackType: 0}, Value: 5], }) 2020-09-24 09:36:37.075 INFO [139999344776960] ProducerStatsImpl:92 | Producer [persistent://public/default/1600940195testIndividualAck, ] , ProducerStatsImpl (numMsgsSent_ = 0, numBytesSent_ = 0, sendMap_ = {}, latencyAccumulator_ = Latencies [ 50pct: 0ms, 90pct: 0ms, 99pct: 0ms, 99.9pct: 0ms], totalMsgsSent_ = 10, totalBytesSent_ = 280, totalAcksReceived_ = , totalSendMap_ = {[Key: Ok, Value: 10], }, totalLatencyAccumulator_ = Latencies [ 50pct: 4.56ms, 90pct: 4.542ms, 99pct: 2.787ms, 99.9pct: 2.729ms]) 2020-09-24 09:36:37.097 INFO [139999344776960] ConsumerStatsImpl:65 | Consumer [persistent://public/default/1600940195testIndividualAck, subscription-name, 0] , ConsumerStatsImpl (numBytesRecieved_ = 0, totalNumBytesRecieved_ = 0, receivedMsgMap_ = {}, ackedMsgMap_ = {}, totalReceivedMsgMap_ = {}, totalAckedMsgMap_ = {}) 2020-09-24 09:36:37.111 INFO [139999344776960] ConsumerStatsImpl:65 | Consumer [persistent://public/default/1600940195testIndividualAck, subscription-name, 1] , ConsumerStatsImpl (numBytesRecieved_ = 0, totalNumBytesRecieved_ = 280, receivedMsgMap_ = {}, ackedMsgMap_ = {}, totalReceivedMsgMap_ = {[Key: Ok, Value: 10], }, totalAckedMsgMap_ = {[Key: {Result: Ok, ackType: 0}, Value: 5], }) 2020-09-24 09:36:38.075 INFO [139999344776960] ProducerStatsImpl:92 | Producer [persistent://public/default/1600940195testIndividualAck, ] , ProducerStatsImpl (numMsgsSent_ = 0, numBytesSent_ = 0, sendMap_ = {}, latencyAccumulator_ = Latencies [ 50pct: 0ms, 90pct: 0ms, 99pct: 0ms, 99.9pct: 0ms], totalMsgsSent_ = 10, totalBytesSent_ = 280, totalAcksReceived_ = , totalSendMap_ = {[Key: Ok, Value: 10], }, totalLatencyAccumulator_ = Latencies [ 50pct: 4.56ms, 90pct: 4.542ms, 99pct: 2.787ms, 99.9pct: 2.729ms]) 2020-09-24 09:36:38.097 INFO [139999344776960] ConsumerStatsImpl:65 | Consumer [persistent://public/default/1600940195testIndividualAck, subscription-name, 0] , ConsumerStatsImpl (numBytesRecieved_ = 0, totalNumBytesRecieved_ = 0, receivedMsgMap_ = {}, ackedMsgMap_ = {}, totalReceivedMsgMap_ = {}, totalAckedMsgMap_ = {}) 2020-09-24 09:36:38.111 INFO [139999344776960] ConsumerStatsImpl:65 | Consumer [persistent://public/default/1600940195testIndividualAck, subscription-name, 1] , ConsumerStatsImpl (numBytesRecieved_ = 0, totalNumBytesRecieved_ = 280, receivedMsgMap_ = {}, ackedMsgMap_ = {}, totalReceivedMsgMap_ = {[Key: Ok, Value: 10], }, totalAckedMsgMap_ = {[Key: {Result: Ok, ackType: 0}, Value: 5], }) 2020-09-24 09:36:39.075 INFO [139999344776960] ProducerStatsImpl:92 | Producer [persistent://public/default/1600940195testIndividualAck, ] , ProducerStatsImpl (numMsgsSent_ = 0, numBytesSent_ = 0, sendMap_ = {}, latencyAccumulator_ = Latencies [ 50pct: 0ms, 90pct: 0ms, 99pct: 0ms, 99.9pct: 0ms], totalMsgsSent_ = 10, totalBytesSent_ = 280, totalAcksReceived_ = , totalSendMap_ = {[Key: Ok, Value: 10], }, totalLatencyAccumulator_ = Latencies [ 50pct: 4.56ms, 90pct: 4.542ms, 99pct: 2.787ms, 99.9pct: 2.729ms]) 2020-09-24 09:36:39.097 INFO [139999344776960] ConsumerStatsImpl:65 | Consumer [persistent://public/default/1600940195testIndividualAck, subscription-name, 0] , ConsumerStatsImpl (numBytesRecieved_ = 0, totalNumBytesRecieved_ = 0, receivedMsgMap_ = {}, ackedMsgMap_ = {}, totalReceivedMsgMap_ = {}, totalAckedMsgMap_ = {}) 2020-09-24 09:36:39.111 INFO [139999344776960] ConsumerStatsImpl:65 | Consumer [persistent://public/default/1600940195testIndividualAck, subscription-name, 1] , ConsumerStatsImpl (numBytesRecieved_ = 0, totalNumBytesRecieved_ = 280, receivedMsgMap_ = {}, ackedMsgMap_ = {}, totalReceivedMsgMap_ = {[Key: Ok, Value: 10], }, totalAckedMsgMap_ = {[Key: {Result: Ok, ackType: 0}, Value: 5], }) 2020-09-24 09:36:40.075 INFO [139999344776960] ProducerStatsImpl:92 | Producer [persistent://public/default/1600940195testIndividualAck, ] , ProducerStatsImpl (numMsgsSent_ = 0, numBytesSent_ = 0, sendMap_ = {}, latencyAccumulator_ = Latencies [ 50pct: 0ms, 90pct: 0ms, 99pct: 0ms, 99.9pct: 0ms], totalMsgsSent_ = 10, totalBytesSent_ = 280, totalAcksReceived_ = , totalSendMap_ = {[Key: Ok, Value: 10], }, totalLatencyAccumulator_ = Latencies [ 50pct: 4.56ms, 90pct: 4.542ms, 99pct: 2.787ms, 99.9pct: 2.729ms]) 2020-09-24 09:36:40.097 INFO [139999344776960] ConsumerStatsImpl:65 | Consumer [persistent://public/default/1600940195testIndividualAck, subscription-name, 0] , ConsumerStatsImpl (numBytesRecieved_ = 0, totalNumBytesRecieved_ = 0, receivedMsgMap_ = {}, ackedMsgMap_ = {}, totalReceivedMsgMap_ = {}, totalAckedMsgMap_ = {}) 2020-09-24 09:36:40.111 INFO [139999344776960] ConsumerStatsImpl:65 | Consumer [persistent://public/default/1600940195testIndividualAck, subscription-name, 1] , ConsumerStatsImpl (numBytesRecieved_ = 0, totalNumBytesRecieved_ = 280, receivedMsgMap_ = {}, ackedMsgMap_ = {}, totalReceivedMsgMap_ = {[Key: Ok, Value: 10], }, totalAckedMsgMap_ = {[Key: {Result: Ok, ackType: 0}, Value: 5], }) /pulsar/pulsar-client-cpp/tests/BatchMessageTest.cc:421: Failure Value of: numOfMessages Actual: 10 Expected: globalTestBatchMessagesCounter Which is: 5 2020-09-24 09:36:40.143 WARN [139999495969408] ConsumerImpl:132 | [persistent://public/default/1600940195testIndividualAck, subscription-name, 1] Destroyed consumer which was not properly closed 2020-09-24 09:36:40.143 INFO [139999495969408] ConsumerImpl:140 | [persistent://public/default/1600940195testIndividualAck, subscription-name, 1] Closed consumer for race condition: 1 2020-09-24 09:36:40.143 INFO [139999495969408] ClientConnection:1388 | [127.0.0.1:36914 -> 127.0.0.1:6650] Connection closed [ FAILED ] BatchMessageTest.testIndividualAck (5083 ms) ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org