SchneiderV opened a new issue, #373: URL: https://github.com/apache/pulsar-helm-chart/issues/373
**Describe the bug** The broker and proxy pods does not start, because their wait-zookeeper-ready init container OOM kills the zookeeper client. **To Reproduce** Steps to reproduce the behavior: 1. A working Kubernetes Cluster with a pulsar 2.10.x 2. Configure a few topics and subscriptions 3. Send messages through pulsar 4. Then upgrade to 2.11.x or 3.0.0 5. Everything starts as expected, beside the broker and proxies 6. The log of the broker/proxy states repeatedly ``` pulsar cluster <cluster-name> isn't initialized yet ... check in 3 seconds ... Killed ``` However, the zookeepers are up and running **Expected behavior** The zookeeper client in the wait-zookeeper-ready init container is not OOM killed but works correctly **Additional context** The problem is, that for the zookeeper client, the args `-Xms2g -Xmx2g -XX:MaxDirectMemorySize=2g` are set. However, the init container does only have 512mb memory. In detail: - The wait-zookeeper-ready init container calls `bin/pulsar zookeeper-shell -server <cluster-name>-zookeeper get /admin/clusters/<cluster-name>` (proxy) or `bin/bookkeeper org.apache.zookeeper.ZooKeeperMain -server <cluster-name>-zookeeper:2181 get /admin/clusters/<cluster-name>` (broker) - The bin/pulsar or bin/bookkeeper script calls bkenv.sh (https://github.com/apache/pulsar/blob/3eadbc3a437b84a0a04fc926c268c4003feda81e/conf/bkenv.sh#L38) to prepare the JVM arguments - The bkenv.sh (line 38) sets the arguments for 2g of memory: `BOOKIE_MEM=${BOOKIE_MEM:-${PULSAR_MEM:-"-Xms2g -Xmx2g -XX:MaxDirectMemorySize=2g"}}` - At startup, the JVM of the zookeeper client tries to allocate around 1.7g of memory and is OOM Killed by the init container operating system. **Workaround** Use a helm post-renderer and add `export BOOKIE_MEM="-Xms256m -Xmx512m -XX:MaxDirectMemorySize=512m";` to the args of the wait-zookeeper-ready container **Intention** Please fix the helm-chart in some way, that the zookeeper client is not called with the args `-Xms2g -Xmx2g -XX:MaxDirectMemorySize=2g` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
