Tribe's AtMostOnceInterceptor could lead to OutOfMemory under heavy load and
big messages.
------------------------------------------------------------------------------------------
Key: AXIS2-4749
URL: https://issues.apache.org/jira/browse/AXIS2-4749
Project: Axis2
Issue Type: Bug
Affects Versions: 1.6
Environment: Linux 64 bit, JDK 1.6, WSO2 Carbon 2.0.3
Reporter: Stefano Bruna
Wa are stressing the WSO2 ESB with some cache mediator enabled. Messages are
echanged by tribes through the cluster's nodes. Under heavy load and with some
big xml messages (1 mb per message) the local variable Map<ChannelMessage,
Long> receivedMessages is growing continuously leading to a potential Out of
Memory if the cleaning thread that runs every 5 minutes is not fast enough to
free up memory.
See:
http://svn.apache.org/viewvc/axis/axis2/java/core/trunk/modules/clustering/src/org/apache/axis2/clustering/tribes/AtMostOnceInterceptor.java?view=markup
This is because when an object is passed to a HashMap as a key internally the
original object is kept. The Hasmap is using a private class Entry that has the
hash, the orginal object used to create the hash, and the value that, in this
usage case, is a long.
Somewhere inside HashMap.java
// private variables
final Object key;
Object value;
Entry next;
final int hash;
// constructor
Entry(int i, Object obj, Object obj1, Entry entry)
{
value = obj1; <==== value, ok
next = entry;
key = obj; <======= orginal object !!!
hash = i; <======== key, ok
}
So if the AtMostOnceInterceptor manages for example 50 msg/sec and a message is
1 mb we could have within 5 minutes a memory usage for the messageReceived
object of 6 GB.
A simple solution, if we dont'to accept all this as something by desing, could
be to pass to the HashMap the already calculated hash of the object (that is
also the same method that is called internally int i = hash(obj.hashCode());)
to not give the opportunity to the HashMap to keep the actual object used to
produce the key.
.....
// map
private static final Map<Integer, Long> receivedMessagesHashCodes =
new HashMap<Integer, Long>();
.....
Integer hashCode = new Integer(msg.hashCode());
if (receivedMessagesHashCodes.get(hashCode) == null) { // If it is a new
message, keep track of it
receivedMessagesHashCodes.put(hashCode, System.currentTimeMillis());
super.messageReceived(msg);
} else { // If it is a duplicate message, discard it. i.e. dont call
super.messageReceived
log.info("Duplicate message received from " +
TribesUtil.getName(msg.getAddress()));
}
etc...
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
