[
https://issues.apache.org/jira/browse/SAMZA-544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14307490#comment-14307490
]
Chris Riccomini commented on SAMZA-544:
---------------------------------------
Ya. We should first try 2048G heap (or whatever the max 32 bit JVM heap is). If
that doesn't work, we could think about decreasing the number of
threads/messages in the test, which would shrink the heap (though it would
change the characteristics of the test).
Another solution would be to exclude this test from 32bit JVMs. I think this
can be done with a combination of:
# http://stackoverflow.com/questions/22209181/gradle-how-to-exclude-some-tests
#
http://stackoverflow.com/questions/2062020/how-can-i-tell-if-im-running-in-64-bit-jvm-or-32-bit-jvm-from-within-a-program
We should be able to use (2) to determine if the JVM is 32 bit, and exclude (1)
the test, if it is. Still, ideally this test would run as-is in a 2G heap.
> TestSamzaContainerPerformance throws OOM occasionally
> ------------------------------------------------------
>
> Key: SAMZA-544
> URL: https://issues.apache.org/jira/browse/SAMZA-544
> Project: Samza
> Issue Type: Bug
> Components: test
> Affects Versions: 0.9.0
> Reporter: Yan Fang
> Fix For: 0.9.0
>
> Attachments: Screen Shot 2015-02-04 at 1.50.54 PM.png
>
>
> Occasionally, can see the OOM error from TestSamzaContainerPerformance. Doubt
> it is because we decreased the memory in SAMZA-426. Log is following.
> {code}
> 11:22:14.338 [DEBUG] [TestEventLogger] 2152 [ThreadJob] INFO
> org.apache.samza.container.SamzaContainer - Starting consumer multiplexer.
> 11:23:57.797 [ERROR] [system.err]
> 11:23:57.798 [ERROR] [system.err] Exception: java.lang.OutOfMemoryError
> thrown from the UncaughtExceptionHandler in thread "MockSystemConsumer-7"
> 11:24:47.237 [DEBUG] [TestEventLogger] 155714 [ThreadJob] INFO
> org.apache.samza.container.SamzaContainer - Shutting down.
> 11:24:47.237 [DEBUG] [TestEventLogger] Exception in thread
> "MockSystemConsumer-6" java.lang.OutOfMemoryError: Java heap space
> 11:24:47.238 [DEBUG] [TestEventLogger] at
> java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:335)
> 11:24:47.238 [DEBUG] [TestEventLogger] at
> org.apache.samza.util.BlockingEnvelopeMap.put(BlockingEnvelopeMap.java:168)
> 11:24:47.238 [DEBUG] [TestEventLogger] at
> org.apache.samza.system.mock.MockSystemConsumer.access$200(MockSystemConsumer.java:43)
> 11:24:47.238 [DEBUG] [TestEventLogger] at
> org.apache.samza.system.mock.MockSystemConsumer$MockSystemConsumerRunnable.run(MockSystemConsumer.java:163)
> 11:24:47.238 [DEBUG] [TestEventLogger] at
> java.lang.Thread.run(Thread.java:745)
> 11:24:47.239 [DEBUG] [TestEventLogger] 155714 [ThreadJob] INFO
> org.apache.samza.container.SamzaContainer - Shutting down consumer
> multiplexer.
> 11:24:47.239 [DEBUG] [TestEventLogger] Exception in thread
> "MockSystemConsumer-2" java.lang.OutOfMemoryError: Java heap space
> 11:24:47.239 [DEBUG] [TestEventLogger] at
> org.apache.samza.system.mock.MockSystemConsumer$MockSystemConsumerRunnable.run(MockSystemConsumer.java:163)
> 11:24:47.239 [DEBUG] [TestEventLogger] at
> java.lang.Thread.run(Thread.java:745)
> 11:24:47.240 [DEBUG] [TestEventLogger] Exception in thread
> "MockSystemConsumer-3" java.lang.OutOfMemoryError: Java heap space
> 11:24:47.240 [DEBUG] [TestEventLogger] Exception in thread
> "MockSystemConsumer-9" java.lang.OutOfMemoryError: Java heap space
> 11:24:47.240 [DEBUG] [TestEventLogger] at
> org.apache.samza.system.mock.MockSystemConsumer$MockSystemConsumerRunnable.run(MockSystemConsumer.java:163)
> 11:24:47.240 [DEBUG] [TestEventLogger] at
> java.lang.Thread.run(Thread.java:745)
> 11:24:47.240 [DEBUG] [TestEventLogger]
> 11:24:47.240 [DEBUG] [TestEventLogger]
> org.apache.samza.test.performance.TestSamzaContainerPerformance >
> testContainerPerformance STANDARD_OUT
> 11:24:47.241 [DEBUG] [TestEventLogger] Got interrupt. Shutting down.
> 11:24:47.241 [DEBUG] [TestEventLogger] Got interrupt. Shutting down.
> 11:24:47.241 [DEBUG] [TestEventLogger] Got interrupt. Shutting down.
> 11:24:47.241 [DEBUG] [TestEventLogger] Got interrupt. Shutting down.
> 11:24:47.241 [DEBUG] [TestEventLogger] Got interrupt. Shutting down.
> 11:24:47.242 [DEBUG] [TestEventLogger] Got interrupt. Shutting down.
> 11:24:47.242 [DEBUG] [TestEventLogger] Got interrupt. Shutting down.
> 11:24:47.242 [DEBUG] [TestEventLogger]
> 11:24:47.242 [DEBUG] [TestEventLogger]
> org.apache.samza.test.performance.TestSamzaContainerPerformance >
> testContainerPerformance STANDARD_ERROR
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
