On 8/9/11 9:57 PM, Kiran Ayyagari wrote:
On Tue, Aug 9, 2011 at 10:58 PM, Emmanuel Lecharny<[email protected]> wrote:
Hi,
I tried to launch a test with 10000 modifications (modDn test) to see what
could happen, but the test fails after a few thousands loops (around 4000).
I started a profiling session, and I saw that the memory is quickly consumed
and never released.
I tested with 128M, 256M and 512M, the only difference is the number of
loops we can do.
When we reach the memory limit, the GC start to kick hard and so does the
CPU. I have beautiful curves on VisualVM !
We do have a memory leak somewhere, we now have to find where ! IMO, it
seems that the ActiveMQ queues are storing refs in memory, and aren't
cleaned when we pick a message. To be checked.
I think this is quite likely the scenario cause we use in-vm transport
for communicating with queues
I tried to configure ActiveMQ to use what they call File Based Cursor,
with no success
(http://activemq.apache.org/how-do-i-configure-activemq-to-hold-100s-of-millions-of-queue-messages-.html)
Here is the code I added :
...
// Create the system responsible for transmitting the
requests when
// the direct connection to the consumer is closed.
brokerService = new BrokerService();
brokerService.setUseJmx( false );
brokerService.setPersistent( true );
brokerService.setDataDirectory( path );
PolicyMap policyMap = new PolicyMap();
// The Topic policyEntry
PolicyEntry topicPolicyEntry = new PolicyEntry();
topicPolicyEntry.setTopic( ">" );
topicPolicyEntry.setProducerFlowControl( true );
// Limit memory used to 1M
topicPolicyEntry.setMemoryLimit( 1000000L );
PendingSubscriberMessageStoragePolicy
pendingSubscriberPolicy = new FilePendingSubscriberMessageStoragePolicy();
topicPolicyEntry.setPendingSubscriberPolicy(
pendingSubscriberPolicy );
policyMap.put( topicPolicyEntry.getDestination(),
topicPolicyEntry );
// The Queue policyEntry
PolicyEntry queuePolicyEntry = new PolicyEntry();
queuePolicyEntry.setTopic( ">" );
queuePolicyEntry.setProducerFlowControl( true );
// Limit memory used to 1M
queuePolicyEntry.setMemoryLimit( 1000000L );
PendingQueueMessageStoragePolicy pendingQueuePolicy = new
FilePendingQueueMessageStoragePolicy();
queuePolicyEntry.setPendingQueuePolicy( pendingQueuePolicy );
policyMap.put( queuePolicyEntry.getDestination(),
queuePolicyEntry );
brokerService.setDestinationPolicy( policyMap );
...
Didn't changed anything. My understanding of ActiveMQ configuration is
*very* limited, so it needs some more love from someone who knows how
this configuration works.
--
Regards,
Cordialement,
Emmanuel Lécharny
www.iktek.com
