[
https://issues.apache.org/jira/browse/CAMEL-7635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Raúl Kripalani updated CAMEL-7635:
----------------------------------
Description:
We just had an OOM incident in our dev environments after upgrading from Camel
2.10.3 to 2.13.1. Heap settings have remained untouched.
A heap dump showed millions of DefaultMessageHistory instances retained (see
[^image1.png]), along with their corresponding Date and StopWatch instances.
Obviously our first solution will be to disable message history in all contexts.
Digging deeper, I'm utterly confused because I don't seem to find the GC roots
that are keeping these objects alive.
OQL query for VisualVM:
{code}
select x from org.apache.camel.impl.DefaultMessageHistory x where
count(referrers(x)) > 0
{code}
returns many objects, which is good.
However, they are referenced by some Object[] which in turn has no referrers
(!), see [^image2.png].
Using the "Find nearest GC root" feature yields no results either.
This mysterious Object[] seems to be filled from position 4072 onwards (see
[^image3.png]), it has 9k+ items... Perhaps some kind of Queue? There are many
more Object[] arrays storing DefaultMessageHistory instances too.
was:
We just had an OOM incident in our dev environments after upgrading from Camel
2.10.3 to 2.13.1. Heap settings have remained untouched.
A heap dump showed millions of DefaultMessageHistory instances retained (see
screenshot), along with their corresponding Date and StopWatch instances.
Obviously our first solution will be to disable message history in all contexts.
Digging deeper, I'm utterly confused because I don't seem to find the GC roots
that are keeping these objects alive.
OQL query for VisualVM:
{code}
select x from org.apache.camel.impl.DefaultMessageHistory x where
count(referrers(x)) > 0
{code}
returns many objects, which is good.
However, they are referenced by some Object[] which in turn has no referrers
(!), see screenshot.
Using the "Find nearest GC root" feature yields no results either.
This mysterious Object[] seems to be filled from position 4072 onwards (see
screenshot), it has 9k+ items... Perhaps some kind of Queue? There are many
more Object[] arrays storing DefaultMessageHistory instances too.
> Memory leak in Message History feature
> --------------------------------------
>
> Key: CAMEL-7635
> URL: https://issues.apache.org/jira/browse/CAMEL-7635
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Affects Versions: 2.13.1
> Environment: Apache ServiceMix 5.1.0 w/ Camel 2.13.1
> JDK 7
> Reporter: Raúl Kripalani
> Attachments: image1.png, image2.png, image3.png
>
>
> We just had an OOM incident in our dev environments after upgrading from
> Camel 2.10.3 to 2.13.1. Heap settings have remained untouched.
> A heap dump showed millions of DefaultMessageHistory instances retained (see
> [^image1.png]), along with their corresponding Date and StopWatch instances.
> Obviously our first solution will be to disable message history in all
> contexts.
> Digging deeper, I'm utterly confused because I don't seem to find the GC
> roots that are keeping these objects alive.
> OQL query for VisualVM:
> {code}
> select x from org.apache.camel.impl.DefaultMessageHistory x where
> count(referrers(x)) > 0
> {code}
> returns many objects, which is good.
> However, they are referenced by some Object[] which in turn has no referrers
> (!), see [^image2.png].
> Using the "Find nearest GC root" feature yields no results either.
> This mysterious Object[] seems to be filled from position 4072 onwards (see
> [^image3.png]), it has 9k+ items... Perhaps some kind of Queue? There are
> many more Object[] arrays storing DefaultMessageHistory instances too.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
