[
https://issues.apache.org/jira/browse/ARTEMIS-2852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17170224#comment-17170224
]
Kasper Kondzielski edited comment on ARTEMIS-2852 at 8/3/20, 7:12 PM:
----------------------------------------------------------------------
Let me start from the beginning. In our tests we try to compare various queues.
Obviously most of these queues can be fine tuned to the very extreme in terms
of performance, but this requires a lot of very specialized knowledge. As we
are time constrained we can't allow ourselves to dive so deeply into each
implementation. That's why for most of the cases we just use the default
configuration with some best practice tune ups if they are publicly and easily
available. This is the kind of comparison I had in mind when I opened this
issue. I compared artemis 2.2.0 with all the best practice we knew at the time
of testing with the current version 2.9.0 and all the best practices we know
now. So it is not about comparing isolated binaries with all other things fixed
to some values, but rather giving a practical insight on the performance which
would benefit most of the people.
Still, when I saw the results I was concerned about that decrease. There can be
three causes to that:
* I did something terrible wrong in the configuration
* there is a bug in the current version
* there is nothing wrong either with implementation or the configuration and
the performance decrease was a result of e.g. stabilizing and hardening the
system
In the first two cases I think that it is justified to create this issue.
Although it might not be the best name for this issue, without having the
option to compare these results against previous version I wouldn't even know
that it might be an issue.
In case of the third option it still might be good to let users know about such
characteristic, especially to those who migrated all the way from 2.2.0 up to
2.13.0
was (Author: kkondzielski):
Let me start from the beginning. In our tests we try to compare various queues.
Obviously most of these queues can be fine tuned to the very extreme in terms
of performance, but this requires a lot of very specialized knowledge. As we
are time constrained we can't allow ourselves to dive so deeply into each
implementation. That's way for most of the cases we just use the default
configuration with some best practice tune ups if they are publicly and easily
available. This is the kind of comparison I had in mind when I opened this
issue. I compared artemis 2.2.0 with all the best practice we knew at the time
of testing with the current version 2.9.0 and all the best practices we know
now. So it is not about comparing isolated binaries with all other things fixed
to some values, but rather giving a practical insight on the performance which
would benefit most of the people.
Still, when I saw the results I was concerned about that decrease. There can be
three causes to that:
* I did something terrible wrong in the configuration
* there is a bug in the current version
* there is nothing wrong either with implementation or the configuration and
the performance decrease was a result of e.g. stabilizing and hardening the
system
In the first two cases I think that it is justified to create this issue.
Although it might not be the best name for this issue, without having the
option to compare these results against previous version I wouldn't even know
that it might be an issue.
In case of the third option it still might be good to let users know about such
characteristic, especially to those who migrated all the way from 2.2.0 up to
2.13.0
> Huge performance decrease between versions 2.2.0 and 2.13.0
> -----------------------------------------------------------
>
> Key: ARTEMIS-2852
> URL: https://issues.apache.org/jira/browse/ARTEMIS-2852
> Project: ActiveMQ Artemis
> Issue Type: Bug
> Reporter: Kasper Kondzielski
> Priority: Major
> Attachments: Selection_433.png, Selection_434.png, Selection_440.png,
> Selection_441.png
>
>
> Hi,
> Recently, we started to prepare a new revision of our blog-post in which we
> test various implementations of replicated queues. Previous version can be
> found here: [https://softwaremill.com/mqperf/]
> We updated artemis binary to 2.13.0, regenerated configuration file and
> applied all the performance tricks you told us last time. In particular these
> were:
> * the {{Xmx}} java parameter bumped to {{16G (now bumped to 48G)}}
> * in {{broker.xml}}, the {{global-max-size}} setting changed to {{8G (this
> one we forgot to set, but we suspect that it is not the issue)}}
> * {{journal-type}} set to {{MAPPED}}
> * {{journal-datasync}}, {{journal-sync-non-transactional}} and
> {{journal-sync-transactional}} all set to false
> Apart from that we changed machines' type we use to r5.2xlarge ( 8 cores, 64
> GIB memory, Network bandwidth Up to 10 Gbps, Storage bandwidth Up to 4,750
> Mbps) and we decided to always run twice as much receivers as senders.
> From our tests it looks like version 2.13.0 is not scaling as well, with the
> increase of senders and receivers, as version 2.2.0 (previously tested).
> Basically is not scaling at all as the throughput state almost at the same
> level, while previously it used to grow linearly.
> Here you can find our tests results for both versions:
> [https://docs.google.com/spreadsheets/d/1kr9fzSNLD8bOhMkP7K_4axBQiKel1aJtpxsBCOy9ugU/edit?usp=sharing]
> We are aware that now there is a dedicated page in documentation about
> performance tuning, but we are surprised that same settings as before
> performs much worse.
> Maybe there is an obvious property which we overlooked which should be turned
> on?
> All changes between those versions together with the final configuration can
> be found on this merged PR:
> [https://github.com/softwaremill/mqperf/commit/6bfae489e11a250dc9e6ef59719782f839e8874a]
>
> Charts showing machines' usage in attachments. Memory consumed by artemis
> process didn't exceed ~ 16 GB. Bandwidht and cpu weren't also a bottlenecks.
> p.s. I wanted to ask this question on mailing list/nabble forum first but it
> seems that I don't have permissions to do so even though I registered &
> subscribed. Is that intentional?
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
