eolivelli opened a new pull request #14121: URL: https://github.com/apache/pulsar/pull/14121
Fix a StackOverFlowException that we have seen on Pulsar in some non-OSS user application. **Problem** We are calling readMoreEntries() recursively. **Modifications** The fix is to schedule the read in another thread, like we did in apache/pulsar#10696 **Details** The stacktrace is taken from the DataStax fork of Pulsar version 2.8.0_1.1.16, but the problem still applies to master, 2.9 and 2.8 (I didn't check 2.7) ``` > 2022-02-02_17:56:18.396 [pulsar] STDOUT: MultipleConsumers.sendMessagesToConsumers(PersistentStickyKeyDispatcherMultipleConsumers.java:286) ~[com.datastax.oss-pulsar-broker-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.396 [pulsar] STDOUT: at org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers.readEntriesComplete(PersistentDispatcherMultipleConsumers.java:473) ~[com.datastax.oss-pulsar-broker-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.396 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedCursorImpl$11.readEntryComplete(ManagedCursorImpl.java:1251) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.396 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.EntryCacheImpl.asyncReadEntry0(EntryCacheImpl.java:211) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.396 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.EntryCacheImpl.asyncReadEntry(EntryCacheImpl.java:190) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.396 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.asyncReadEntry(ManagedLedgerImpl.java:1916) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.396 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.asyncReadEntry(ManagedLedgerImpl.java:1837) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.396 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedCursorImpl.lambda$asyncReplayEntries$9(ManagedCursorImpl.java:1276) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.396 [pulsar] STDOUT: at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[?:?] 2022-02-02_17:56:18.396 [pulsar] STDOUT: at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177) ~[?:?] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.TreeMap$KeySpliterator.forEachRemaining(TreeMap.java:2739) ~[?:?] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:?] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:?] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) ~[?:?] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedCursorImpl.asyncReplayEntries(ManagedCursorImpl.java:1270) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers.asyncReplayEntriesInOrder(PersistentDispatcherMultipleConsumers.java:365) ~[com.datastax.oss-pulsar-broker-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers.readMoreEntries(PersistentDispatcherMultipleConsumers.java:247) ~[com.datastax.oss-pulsar-broker-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.pulsar.broker.service.persistent.PersistentStickyKeyDispatcherMultipleConsumers.sendMessagesToConsumers(PersistentStickyKeyDispatcherMultipleConsumers.java:286) ~[com.datastax.oss-pulsar-broker-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.pulsar.broker.service.persistent.PersistentDispatcherMultipleConsumers.readEntriesComplete(PersistentDispatcherMultipleConsumers.java:473) ~[com.datastax.oss-pulsar-broker-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedCursorImpl$11.readEntryComplete(ManagedCursorImpl.java:1251) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.EntryCacheImpl.asyncReadEntry0(EntryCacheImpl.java:211) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.EntryCacheImpl.asyncReadEntry(EntryCacheImpl.java:190) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.asyncReadEntry(ManagedLedgerImpl.java:1916) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl.asyncReadEntry(ManagedLedgerImpl.java:1837) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at org.apache.bookkeeper.mledger.impl.ManagedCursorImpl.lambda$asyncReplayEntries$9(ManagedCursorImpl.java:1276) ~[com.datastax.oss-managed-ledger-2.8.0.1.1.16.jar:2.8.0.1.1.16] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) ~[?:?] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177) ~[?:?] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.TreeMap$KeySpliterator.forEachRemaining(TreeMap.java:2739) ~[?:?] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) ~[?:?] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) ~[?:?] 2022-02-02_17:56:18.397 [pulsar] STDOUT: at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?] 2022-02-02_17:56:18.398 [pulsar] STDOUT: at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) ~[?:?] ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
