[
https://issues.apache.org/jira/browse/QPID-8557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17394761#comment-17394761
]
Daniil Kirilyuk commented on QPID-8557:
---------------------------------------
We observed in our tests that broker performs flush on every received message.
Even when setting parameters "qpid.broker.bdbCommiterNotifyThreshold"
(_jobQueueNotifyThreshold) and "qpid.broker.bdbCommiterWaitTimeout"
(_commiterWaitTimeout) to some high values (e.g. 100 messages and 120000 ms),
debug message "flushLog completed in X ms" were seen on every sent message.
Following calls were involved:
{noformat}
at
org.apache.qpid.server.store.berkeleydb.CoalescingCommiter$ThreadNotifyingSettableFuture.get(CoalescingCommiter.java:318)
at org.apache.qpid.server.txn.AsyncCommand.complete(AsyncCommand.java:50) at
org.apache.qpid.server.protocol.v1_0.StandardReceivingLinkEndpoint.receiveComplete(StandardReceivingLinkEndpoint.java:592)
at
org.apache.qpid.server.protocol.v1_0.Session_1_0.lambda$receivedComplete$5(Session_1_0.java:1343)
at java.base/java.lang.Iterable.forEach(Iterable.java:75) at
org.apache.qpid.server.protocol.v1_0.Session_1_0.receivedComplete(Session_1_0.java:1343)
at
org.apache.qpid.server.protocol.v1_0.AMQPConnection_1_0Impl.lambda$receivedComplete$11(AMQPConnection_1_0Impl.java:1356)
at java.base/java.security.AccessController.doPrivileged(Native Method) at
org.apache.qpid.server.protocol.v1_0.AMQPConnection_1_0Impl.receivedComplete(AMQPConnection_1_0Impl.java:1354)
at
org.apache.qpid.server.protocol.v1_0.AMQPConnection_1_0Impl.onReceive(AMQPConnection_1_0Impl.java:1335)
at
org.apache.qpid.server.transport.AbstractAMQPConnection.lambda$received$2(AbstractAMQPConnection.java:573)
at java.base/java.security.AccessController.doPrivileged(Native Method) at
org.apache.qpid.server.transport.AbstractAMQPConnection.received(AbstractAMQPConnection.java:568)
at
org.apache.qpid.server.transport.MultiVersionProtocolEngine.received(MultiVersionProtocolEngine.java:138)
at
org.apache.qpid.server.transport.NonBlockingConnection.processAmqpData(NonBlockingConnection.java:611)
at
org.apache.qpid.server.transport.NonBlockingConnectionPlainDelegate.processData(NonBlockingConnectionPlainDelegate.java:58)
at
org.apache.qpid.server.transport.NonBlockingConnection.doRead(NonBlockingConnection.java:496)
at
org.apache.qpid.server.transport.NonBlockingConnection.doWork(NonBlockingConnection.java:270)
at
org.apache.qpid.server.transport.NetworkConnectionScheduler.processConnection(NetworkConnectionScheduler.java:134)
at
org.apache.qpid.server.transport.SelectorThread$ConnectionProcessor.processConnection(SelectorThread.java:545)
at
org.apache.qpid.server.transport.SelectorThread$SelectionTask.performSelect(SelectorThread.java:345)
at
org.apache.qpid.server.transport.SelectorThread$SelectionTask.run(SelectorThread.java:95)
at
org.apache.qpid.server.transport.SelectorThread.run(SelectorThread.java:503) at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at
org.apache.qpid.server.bytebuffer.QpidByteBufferFactory.lambda$null$0(QpidByteBufferFactory.java:464)
at java.base/java.lang.Thread.run(Thread.java:834)
{noformat}
> [Broker-J] Batching mode for CoalescingCommitter
> ------------------------------------------------
>
> Key: QPID-8557
> URL: https://issues.apache.org/jira/browse/QPID-8557
> Project: Qpid
> Issue Type: Improvement
> Components: Broker-J
> Affects Versions: qpid-java-broker-8.0.5
> Reporter: Daniil Kirilyuk
> Priority: Minor
>
> Although QPID-8547 introduced configurable parameters for
> CoalescingCommitter, BDB log is flushed to disc on every message. That is
> caused by calling future.get() inside of
> org.apache.qpid.server.txn.AsyncCommand.complete(). Such call forces broker
> to wait until CoalescingCommiter calls #CommitThread#processJobs() making the
> operation effectively sync instead of async (every message triggers
> processJobs() and data flush).
> To improve overall performance we suggest to add batching mode for
> CoalescingCommitter, which would consider BDBVirtualHost parameters and flush
> data in chunks of size ${qpid.broker.bdbCommiterNotifyThreshold} or every
> ${qpid.broker.bdbCommiterWaitTimeout} ms (whatever comes first).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
