labuladong opened a new pull request, #18337:
URL: https://github.com/apache/pulsar/pull/18337
<!--
### Contribution Checklist
- PR title format should be *[type][component] summary*. For details, see
*[Guideline - Pulsar PR Naming
Convention](https://docs.google.com/document/d/1d8Pw6ZbWk-_pCKdOmdvx9rnhPiyuxwq60_TrD68d7BA/edit#heading=h.trs9rsex3xom)*.
- Fill out the template below to describe the changes contributed by the
pull request. That will give reviewers the context they need to do the review.
- Each pull request should address only one issue, not mix up code from
multiple issues.
- Each commit in the pull request has a meaningful commit message
- Once all items of the checklist are addressed, remove the above text and
this checklist, leaving only the filled out template below.
-->
### Motivation
https://github.com/apache/pulsar/pull/18217 fixed the
`OutOfDirectMemoryError` of `startBookieWithMetadataStore`. But if you use
zookeeper as metadata store `OutOfDirectMemoryError` will occur again:
```shell
> PULSAR_STANDALONE_USE_ZOOKEEPER=1 ./bin/pulsar standalone --num-bookies 3
2022-11-04T16:45:04,059+0800 [main] INFO
org.apache.bookkeeper.bookie.storage.ldb.SingleDirectoryDbLedgerStorage -
Creating single directory db ledger storage on
data/standalone/bookkeeper1/current
2022-11-04T16:45:04,060+0800 [main] ERROR
org.apache.pulsar.PulsarStandaloneStarter - Failed to start pulsar service.
io.netty.util.internal.OutOfDirectMemoryError: failed to allocate 536870912
byte(s) of direct memory (used: 3758161920, max: 4294967296)
at
io.netty.util.internal.PlatformDependent.incrementMemoryCounter(PlatformDependent.java:806)
~[io.netty-netty-common-4.1.77.Final.jar:4.1.77.Final]
at
io.netty.util.internal.PlatformDependent.allocateDirectNoCleaner(PlatformDependent.java:735)
~[io.netty-netty-common-4.1.77.Final.jar:4.1.77.Final]
at
io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledUnsafeNoCleanerDirectByteBuf.java:30)
~[io.netty-netty-buffer-4.1.77.Final.jar:4.1.77.Final]
at
io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.allocateDirect(UnpooledByteBufAllocator.java:186)
~[io.netty-netty-buffer-4.1.77.Final.jar:4.1.77.Final]
at
io.netty.buffer.UnpooledDirectByteBuf.<init>(UnpooledDirectByteBuf.java:64)
~[io.netty-netty-buffer-4.1.77.Final.jar:4.1.77.Final]
at
io.netty.buffer.UnpooledUnsafeDirectByteBuf.<init>(UnpooledUnsafeDirectByteBuf.java:41)
~[io.netty-netty-buffer-4.1.77.Final.jar:4.1.77.Final]
at
io.netty.buffer.UnpooledUnsafeNoCleanerDirectByteBuf.<init>(UnpooledUnsafeNoCleanerDirectByteBuf.java:25)
~[io.netty-netty-buffer-4.1.77.Final.jar:4.1.77.Final]
at
io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeNoCleanerDirectByteBuf.<init>(UnpooledByteBufAllocator.java:181)
~[io.netty-netty-buffer-4.1.77.Final.jar:4.1.77.Final]
at
io.netty.buffer.UnpooledByteBufAllocator.newDirectBuffer(UnpooledByteBufAllocator.java:91)
~[io.netty-netty-buffer-4.1.77.Final.jar:4.1.77.Final]
at
io.netty.buffer.AbstractByteBufAllocator.directBuffer(AbstractByteBufAllocator.java:188)
~[io.netty-netty-buffer-4.1.77.Final.jar:4.1.77.Final]
at io.netty.buffer.Unpooled.directBuffer(Unpooled.java:148)
~[io.netty-netty-buffer-4.1.77.Final.jar:4.1.77.Final]
at
org.apache.bookkeeper.bookie.storage.ldb.ReadCache.<init>(ReadCache.java:79)
~[org.apache.bookkeeper-bookkeeper-server-4.15.1.jar:4.15.1]
at
org.apache.bookkeeper.bookie.storage.ldb.ReadCache.<init>(ReadCache.java:67)
~[org.apache.bookkeeper-bookkeeper-server-4.15.1.jar:4.15.1]
at
org.apache.bookkeeper.bookie.storage.ldb.SingleDirectoryDbLedgerStorage.<init>(SingleDirectoryDbLedgerStorage.java:174)
~[org.apache.bookkeeper-bookkeeper-server-4.15.1.jar:4.15.1]
at
org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage.newSingleDirectoryDbLedgerStorage(DbLedgerStorage.java:196)
~[org.apache.bookkeeper-bookkeeper-server-4.15.1.jar:4.15.1]
at
org.apache.bookkeeper.bookie.storage.ldb.DbLedgerStorage.initialize(DbLedgerStorage.java:156)
~[org.apache.bookkeeper-bookkeeper-server-4.15.1.jar:4.15.1]
at
org.apache.bookkeeper.bookie.BookieResources.createLedgerStorage(BookieResources.java:110)
~[org.apache.bookkeeper-bookkeeper-server-4.15.1.jar:4.15.1]
at org.apache.bookkeeper.server.Main.buildBookieServer(Main.java:423)
~[org.apache.bookkeeper-bookkeeper-server-4.15.1.jar:4.15.1]
at
org.apache.pulsar.zookeeper.LocalBookkeeperEnsemble.startBK(LocalBookkeeperEnsemble.java:458)
~[org.apache.pulsar-pulsar-broker-2.11.0.jar:2.11.0]
at
org.apache.pulsar.zookeeper.LocalBookkeeperEnsemble.runBookies(LocalBookkeeperEnsemble.java:325)
~[org.apache.pulsar-pulsar-broker-2.11.0.jar:2.11.0]
at
org.apache.pulsar.zookeeper.LocalBookkeeperEnsemble.startStandalone(LocalBookkeeperEnsemble.java:436)
~[org.apache.pulsar-pulsar-broker-2.11.0.jar:2.11.0]
at
org.apache.pulsar.PulsarStandalone.startBookieWithZookeeper(PulsarStandalone.java:472)
~[org.apache.pulsar-pulsar-broker-2.11.0.jar:2.11.0]
at org.apache.pulsar.PulsarStandalone.start(PulsarStandalone.java:299)
~[org.apache.pulsar-pulsar-broker-2.11.0.jar:2.11.0]
at
org.apache.pulsar.PulsarStandaloneStarter.main(PulsarStandaloneStarter.java:141)
~[org.apache.pulsar-pulsar-broker-2.11.0.jar:2.11.0]
```
So this PR fixes it in the same way
### Documentation
<!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. -->
- [ ] `doc` <!-- Your PR contains doc changes. Please attach the local
preview screenshots (run `sh start.sh` at `pulsar/site2/website`) to your PR
description, or else your PR might not get merged. -->
- [ ] `doc-required` <!-- Your PR changes impact docs and you will update
later -->
- [x] `doc-not-needed` <!-- Your PR changes do not impact docs -->
- [ ] `doc-complete` <!-- Docs have been already added -->
### Matching PR in forked repository
PR in forked repository: https://github.com/labuladong/pulsar/pull/7
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
