Joforde commented on issue #23889: URL: https://github.com/apache/pulsar/issues/23889#issuecomment-2614472357
> > In this case, I am using Pulsar version 4.0.2, with ZooKeeper as the metadata storage service. > > I have identified the critical issue: when a broker registers information with the metadata storage service, it sets the `expectedVersion` field to `Optional.empty()`. For implementations using ZooKeeper, this creates a PERSISTENT node, which means that the node cannot be destroyed when the broker crashes. > > Good observations, [@Joforde](https://github.com/Joforde). Thanks for sharing. Are you using the extensible load manager? It looks like changes [#23298](https://github.com/apache/pulsar/pull/23298) and [#23359](https://github.com/apache/pulsar/pull/23359) by [@BewareMyPower](https://github.com/BewareMyPower) are in this area. > > It seems that the Pulsar default load manager implementation (`loadManagerClassName=org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl`) would use this code to acquire a lock for the broker metadata: > > [pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java](https://github.com/apache/pulsar/blob/325c6a58d53b9e7b4fe31883ec47ae12c5abc71f/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java#L978) > > Line 978 in [325c6a5](/apache/pulsar/commit/325c6a58d53b9e7b4fe31883ec47ae12c5abc71f) > > brokerDataLock = brokersData.acquireLock(brokerZnodePath, localData).join(); > What `loadManagerClassName` setting do you have? org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImpl > > In this case, I am using Pulsar version 4.0.2, with ZooKeeper as the metadata storage service. > > I have identified the critical issue: when a broker registers information with the metadata storage service, it sets the `expectedVersion` field to `Optional.empty()`. For implementations using ZooKeeper, this creates a PERSISTENT node, which means that the node cannot be destroyed when the broker crashes. > > Good observations, [@Joforde](https://github.com/Joforde). Thanks for sharing. Are you using the extensible load manager? It looks like changes [#23298](https://github.com/apache/pulsar/pull/23298) and [#23359](https://github.com/apache/pulsar/pull/23359) by [@BewareMyPower](https://github.com/BewareMyPower) are in this area. > > It seems that the Pulsar default load manager implementation (`loadManagerClassName=org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl`) would use this code to acquire a lock for the broker metadata: > > [pulsar/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java](https://github.com/apache/pulsar/blob/325c6a58d53b9e7b4fe31883ec47ae12c5abc71f/pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java#L978) > > Line 978 in [325c6a5](/apache/pulsar/commit/325c6a58d53b9e7b4fe31883ec47ae12c5abc71f) > > brokerDataLock = brokersData.acquireLock(brokerZnodePath, localData).join(); > What `loadManagerClassName` setting do you have? The loadManagerClassName I have set is org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImpl. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
