[
https://issues.apache.org/jira/browse/QPID-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16488982#comment-16488982
]
Alex Rudyy commented on QPID-7830:
----------------------------------
The caching of strings in AMQP 0-8 and AMQP 1.0 is implemented only for
certain fields. It looks like inconsistent approach to me. I think that caching
should be implemented in the layer decoding the string regardless the string
use case. Caching of string in AMQP 0-10 is implemented in decoder and looks to
me like a more attractive approach than the one used for caching in AMQP 0-8
and AMQP 1.0. Though it is less effective as single-use object would hit the
cache but it is easier to maintain and reason about. Currently, in AMQP 0-8 we
do not cache message header names. Potentially caching of header names in AMQP
0-8 could further reduce memory consumption.
I believe that caching functionality should be moved into
{{StringTypeConstructor}} in AMQP 1.0 and implemented as part of {{construct}},
something like below
{code}
byte[] data = new byte[size];
in.get(data);
ByteBuffer buffer = ByteBuffer.wrap(data);
String cached = CACHE.get().getIfPresent(buffer);
if (cached == null)
{
cached = new String(data, UTF_8);
CACHE.get().put(buffer, cached);
}
return cached;
{code}
AMQP 0-8 {{AMQPShortStrings}} should be cached unconditionally in
{{AMQShortString#readAMQShortString}} similar to code above and calls to
{{intern}} should be removed. That should simplify the code and would allow to
cache more duplicate string values than the current approach.
I am going to attach a patch for the suggested changes. In meant time, I am
going to merge implemented changes into 7.0.x
> Heap dominated by duplicates of common routing values / header values etc
> -------------------------------------------------------------------------
>
> Key: QPID-7830
> URL: https://issues.apache.org/jira/browse/QPID-7830
> Project: Qpid
> Issue Type: Bug
> Components: Broker-J
> Reporter: Keith Wall
> Assignee: Alex Rudyy
> Priority: Major
> Fix For: qpid-java-broker-7.1.0, qpid-java-broker-7.0.4
>
>
> When used for store and forwarding, in some use cases the Broker's heap can
> become dominated by duplicates of common values such as routing information
> (e.g. {{amq.direct}} or an application's queue name) or common header values
> (e.g a application/octet-stream or an application's user id).
> On the 0-8..0-91 paths, every enqueued message gets its own
> {{MessagePublishInfo}} referencing its own {{AMQShortString}} exchange and
> routing keys. For some use-cases, these are drawn from a small set. On the
> AMQP 1.0 path, {{Properties#to}} is an example. 0-10 is probably affected
> too.
> This unnecessarily increases the heap requirements of the Broker.
> The Broker should adopt a sensible intern/caching policy with the same policy
> applying regardless of whether messages follow the on-line enqueue or
> recovery path. Note that in AMQP 1.0, values which are {{Symbols}} have
> their underlying String automatically interned.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
