Niranjan,

On 2/18/21 13:36, Niranjan Rao wrote:
First apologies for non tomcat question. I have seen that there is enough expertise here to provide hints and hints are what I am looking for to solve the problem and question is generic enough. I have tried researching problem to best of my abilities.

It all happens on Ubuntu 20.04 and JDK 15

We have a java program that regularly throws "java.lang.OutOfMemoryError: Java heap space" exception. Puzzling point is it happens only on one VM. We have a set of two VMs/boxes spawned from same AWS image. Machine class/region is exactly same and since they are from same image, they should be mostly identical except stuff like host name, ip address etc.

Number of tasks performed by VMs are comparable and not a significant difference. Yet, one VM never runs of out of memory and other one does. Sometimes it's as soon as half an hour after restarting the process while on the other box process is running for days and no issues.

I took memory dumps from both VMs and they look similar. Program is started with -Xmx1g flag and we have taken regular memory dumps. In many cases eclipse MAT reports total memory usage was less than 100MB when program crashed with out of memory exception.


Has anyone seen anything similar to this? Identical bits of code behaving differently? What else should I be looking for?

What is the load profile of each application/server? You said you aren't running Tomcat, but is load on each of the applications balanced in any way similar to how a web-application load-balancer would work? Sometimes, the answer is simply that one server is doing more work than the other.

We have two application servers which are "identical" except that only one of them handles our email queue.

Maybe though the "types" of tasks are the same for each server, one of them is getting unlucky and is handling a "big" task that fails each time?

Do you have any logging which would indicate which task, ro what kinds of tasks are failing?

Do you have a stack trace of the OOME? Do you have a bunch of them (from many separate events)? Do they all look the same?

The AWS images are the same, have you upgraded the OS on either one after initial launch, or do you always start fresh with the same image and no "apt-get update" on them. Same JVM and everything on each of them?

If you start with -Xmx1G then you should consider also using -Xms1G. IF you have a long-running process which you expect may take up 1G of heap space, go ahead and allocate it all at once instead of wasting time re-sizing the heap a bunch of times on your way up to 1G.

-chris

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org

Reply via email to