lhotari commented on issue #9408: URL: https://github.com/apache/pulsar/issues/9408#issuecomment-771550559
I was able to reproduce the flakiness and I made some changes in https://github.com/apache/pulsar/compare/master...lhotari:lh-fix-flaky-load-balancer-test . However the flakiness doesn't go away. The logs are in https://gist.github.com/lhotari/0653dc5d9ca418a1bca2d01a632fd655 . The problem is that the leader doesn't get assigned to one of the follower brokers. The value of the current leaders stays Optional.empty. This can be found in the logs around these lines: ``` 10:38:34.605 [TestNG-method=testLeaderElection-1] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - New leader is : http://localhost4:32957 10:38:34.605 [TestNG-method=testLeaderElection-1] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Old leader is : http://localhost4:32957 10:38:34.705 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost4:32957 is : Optional[LeaderBroker(serviceUrl=http://localhost4:32957)] 10:38:34.806 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost2:35915 is : Optional[LeaderBroker(serviceUrl=http://localhost4:32957)] 10:38:34.906 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:35.008 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:35.108 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:35.208 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:35.308 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:35.408 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:35.509 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:35.609 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:35.709 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:35.809 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:35.910 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:36.010 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:36.110 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:36.210 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:36.310 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:36.411 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:36.511 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:36.611 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:36.711 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:36.811 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty 10:38:36.912 [awaitility-thread] INFO org.apache.pulsar.broker.loadbalance.LoadBalancerTest - Current leader for http://localhost1:43075 is : Optional.empty ... (continue until it times out) ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
