bpereto edited a comment on issue #5510: URL: https://github.com/apache/pulsar/issues/5510#issuecomment-1024246874
we experience this also in version 2.8.1 - on high load (publish rate at 220k msg/s) bundle splitting seems not to be useful at this stage, because we have 1 topic (2 partitions). setup: * 1 proxy * 2 brokers * 3 bookies * 3 zookeepers **proxy** ``` 13:58:58.588 [pulsar-proxy-io-2-1] ERROR org.apache.pulsar.client.impl.ClientCnx - [id: 0xcab5b0cc .. Close connection because received internal-server error java.lang.IllegalStateException: Namespace bundle public/default/0x00e60000_0x00e68000 is being unloaded 13:58:58.590 [pulsar-proxy-io-2-1] WARN org.apache.pulsar.proxy.server.LookupProxyHandler - Failed to lookup topic netflow.goflow-proxy.pb-partition-1: java.lang.IllegalStateException: Namespace bundle public/default/0x00e60000_0x00e68000 is being unloaded ``` **broker** scraped logs: ``` 13:57:59.179 [BookKeeperClientWorker-OrderedExecutor-3-0] WARN org.apache.bookkeeper.client.BookieWatcherImpl - New ensemble: [lake03:3181, lake01:3181] is not adhering to Placement Policy. quarantinedBookies: [] 13:58:03.348 [pulsar-modular-load-manager-33-1] INFO org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Load-manager splitting bundle public/default/0x00e60000_0x00e80000 and unloading true 13:58:03.353 [pulsar-web-41-2] INFO org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Split namespace bundle public/default/0x00e60000_0x00e80000 13:58:03.358 [pulsar-web-41-2] ERROR org.apache.pulsar.broker.web.PulsarWebResource - [null] Invalid bundle range public/default/0x00e60000_0x00e80000, Invalid upper boundary for bundle 13:58:03.361 [pulsar-modular-load-manager-33-1] ERROR org.apache.pulsar.broker.loadbalance.impl.ModularLoadManagerImpl - Failed to split namespace bundle public/default/0x00e60000_0x00e80000 org.apache.pulsar.client.admin.PulsarAdminException$PreconditionFailedException: Invalid upper boundary for bundle 13:58:58.476 [pulsar-ordered-OrderedExecutor-5-0] INFO org.apache.pulsar.broker.namespace.OwnedBundle - Disabling ownership: public/default/0x00e68000_0x00e70000 13:58:58.478 [BookKeeperClientWorker-OrderedExecutor-0-0] ERROR org.apache.bookkeeper.client.PendingAddOp - Write of ledger entry to quorum failed: L396 E6613 13:58:58.587 [pulsar-io-4-8] INFO org.apache.pulsar.broker.lookup.TopicLookupBase - Failed to lookup null for topic persistent://public/default/netflow.goflow-proxy.pb-partition-1 with error Namespace bundle public/default/0x00e60000_0x00e68000 is being unloaded ``` with the workaround to disable bundle splitting `loadBalancerAutoBundleSplitEnabled=false` it seems stable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
