[
https://issues.apache.org/jira/browse/QPID-8238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16610224#comment-16610224
]
Keith Wall edited comment on QPID-8238 at 9/11/18 7:52 AM:
-----------------------------------------------------------
I think it would be better to separate the FieldTable optimisation and any
LOGBACK-1027 workaround performance improvements into separate JIRAs. This
would make it easier for reviewers to review and users to evaluate the risk of
taking a change (or not). At the moment the FieldTable changes which affects
*all* 0-x users are hidden under a Jira title that talks of topic exchange.
Conversely, the Jira body talks of a 0-9 performance degradation yet the code
change to the topic exchange impacts a shared 0-9, 0-10, 1.0 path. I would
suggest reverting the
[c30e8ae|https://git-wip-us.apache.org/repos/asf?p=qpid-broker-j.git;h=c30e8ae]
and
[8c58e38|https://git-wip-us.apache.org/repos/asf?p=qpid-broker-j.git;h=8c58e38]
from this JIRA.
On the {{FieldTable}} changes, I did like the idea expressed above to make the
FieldTables immutable. I was imagining turning FieldTable into an interface
with two implementations. One that accepts the ByteBuffer and one that adapts
a Java Map. The latter would be used in the BDB upgraders and the conversion
layer. I think this approach would make the implementation tidier and easier
to reason about. Separately, the idea to make the decoding of the short
strings within the FieldTable on-demand sounds appealing. This would benefit
any application that was setting many string properties on a message but only
using a subset for routing. Did you do any testing? (I did wonder if the
approach would also bring benefit to the strings in
BasicContentHeaderProperties - but that would be a separate optimisation).
On the topic exchange changes - could you add some commentary to help explain
your thinking behind this change to allow me to review efficiently.
The turbo filter changes looked reasonable.
was (Author: k-wall):
I think it would be better to separate the FieldTable optimisation and any
LOGBACK-1027 workaround performance improvements into separate JIRAs. This
would make it easier for reviewers to review and users to evaluate the risk of
taking a change (or not). At the moment the FieldTable changes which affects
*all* 0-x users are hidden under a Jira title that talks of topic exchange.
Conversely, the Jira body talks of a 0-9 performance degradation yet the code
change to the topic exchange impacts a shared 0-9, 0-10, 1.0 path. I would
suggest reverting the
[c30e8ae|https://git-wip-us.apache.org/repos/asf?p=qpid-broker-j.git;h=c30e8ae]
and
[8c58e38|https://git-wip-us.apache.org/repos/asf?p=qpid-broker-j.git;h=8c58e38]
from this JIRA.
On the {{FieldTable}} changes, I did like the idea expressed above to make the
FieldTables immutable. I was imagining turning FieldTable into an interface
with two implementations. One that accepts the ByteBuffer and one that adapts
a Java Map. The latter would be used in the BDB upgraders and the conversion
layer. I think this approach would make the implementation tidier and easier
to reason about. Separately, the idea to make the decoding of the short
strings within the FieldTable on-demand sounds appealing. This would benefit
any application that was setting many string properties on a message but only
using a subset for routing. Did you do any testing? (I did wonder if the
approach would also bring benefit to the strings in
BasicContentHeaderProperties - but that would be a separate optimisation).
On the topic exchange changes - could you add some commentary to help explain
your thinking behind this change to allow me to review efficiently.
The turbo filter changes looked reasonable.
> [Broker-J] Improve performance of asynchronous publishing of transient
> messages into topic exchange having queues bound using non-overlapping
> selectors
> --------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: QPID-8238
> URL: https://issues.apache.org/jira/browse/QPID-8238
> Project: Qpid
> Issue Type: Improvement
> Components: Broker-J
> Affects Versions: qpid-java-broker-7.0.6
> Reporter: Alex Rudyy
> Priority: Major
> Fix For: qpid-java-broker-7.1.0, qpid-java-broker-7.0.7
>
> Attachments:
> 0001-QPID-8238-Use-java.lang.String-for-keys-and-values-i.patch
>
>
> The performance of asynchronous publishing of transient messages into topic
> exchange which routes messages into queues bound using non-overlapping
> selectors is 2-3 times slower than performance of 0.32 broker. The
> performance degradation is observed with AMQP 0.9, though, I suspect that the
> AMQP 0-10 protocol could be affected as well.
> I was running tests with 10 concurrent producers publishing messages on
> separate connections using the same routing key into 10 different queues
> (subscribers queues) bound to the exchange using non-overlapping selectors.
> My testing showed that performance of 7.0 broker for this particular use case
> was 2-3 times worse than performance of 0.32 broker.
> The following factors contributed to degradation of performance:
> • Copying data from direct memory into heap memory whilst decoding
> message headers. Due to this factor, the decoding of message headers is
> around twice slower. It seems it contributes around 70% to total performance
> degradation
> • The message routing algorithm is slower due to need to support a new
> feature to route messages into bound exchanges (in addition to queues) using
> replacement routing key.
> • AMQ short strings caching contributes 5-10% to total performance
> degradation. The caching was added to manage heap space more efficiently.
> The numbers provided here could be inaccurate due instrumentation overhead
> whilst profiling the issue.
> Potentially, caching can be turned off but that will not improve performance
> much.
> On other hand, adding of additional caching of strings to amqp-short-strings
> would improve the performance a bit. Whilst evaluating selectors, the fields
> used in selector expressions are represented as java strings but they get
> converted every time into amqp-short-strings when looking up for message
> header values. If 10 queues are bound to the exchange using the same binding
> key, the selector expression is evaluated 10 times for the incoming message.
> Thus, all selector field names are get converted into amqp-short-strings 10
> times as well. It seems adding caching here can improve the performance.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
