I think it would be much better if we can find a root-cause fix instead of altering settings. I would be interested in helping out digging out whatever is causing the memory leak if Guilio is interested in opening a JIRA for this and investigating further.
On Fri, Sep 21, 2018, 11:48 AM Ingo Wolfmayr <ingo.wolfm...@wolfix.at> wrote: > Hallo Guilio, > > I had a similar issue some time ago. My fix was to enable caching for > groovy scripts etc. > > framework/base/config/cache.properties. > > There is a part that starts says: # Development Mode - comment these out > to better cache groovy scripts, etc > > After checking the heat cache analysis I found that for every groovy > script call a new instance of that class was created. Caching fixed it > completely. Maybe it will work for you too. You will then have to clear the > cache in ofbiz if you change something in your groovy scripts. > > Best regards > Ingo > > > -----Ursprüngliche Nachricht----- > Von: Giulio Speri - MpStyle Srl <giulio.sp...@mpstyle.it> > Gesendet: Donnerstag, 20. September 2018 18:43 > An: user@ofbiz.apache.org > Betreff: OFBiz OutOfMemory and stucked JobPoller issue > > Hello everyone, > > hope you're doing good. > I am writing, because I am struggling with a quite strange problem, over > an ofbiz installation, for one of our customers. > This installation is composed by two instances of OFBiz (v13.07.03), > served via an Apache Tomcat webserver, along with a load balancer. > The database server is MariaDB. > > We had the first problems, about 3 weeks ago, when suddenly, the front1 > (ofbiz instance 1), stopped serving web requests; front2, instead, was > still working correctly. > > Obviously we checked the log files, and we saw that async services were > failing; the failure was accompanied by this error line: > > *Thread "AsyncAppender-async" java.lang.OutOfMemoryError: GC overhead limit > exceeded* > > We analyzed the situation with our system specialists, and they told us > that the application was highly stressing machine resources (cpu always at > or near 100%, RAM usage rapidly increasing), until the jvm run out of > memory. > This "resource-high-consumption situation", occurred only when ofbiz1 > instance was started with the JobPoller enabled; if the JobPoller was not > enabled, ofbiz run with low resource usage. > > We then focused on the db, to check first of all the dimensions; the > result was disconcerting; 45GB, mainly divided on four tables: SERVER_HIT > (about > 18 GB), VISIT (about 15 GB), ENTITY_SYNC_REMOVE (about 8 GB), VISITOR > (about 2 GB). > All the other tables had a size in the order of few MB each. > > The first thing we did, was to clear all those tables, reducing > considerably the db size. > After the cleaning, we tried to start ofbiz1 again, with the JobPoller > component enabled; this caused a lot of old scheduled/queued jobs, to > execute. > Except than for the start-up time, the resource usage of the machine, > stabilized around normal to low values (cpu 1-10%). > Ofbiz seemed to work (web request was served), but we noticed taht the > JobPoller did not schedule or run jobs, anymore. > The number of job in "Pending" state in the JobSandbox entity was small > (about 20); no Queued, no Failed, no jobs in other states. > In addition to this, unfortunately, after few hours, jvm run out of memory > again. > > Our jvm has an heap maximum size of 20GB ( we have 32GB on the machine), > so it's not so small, I think. > The next step we're going to do is set-up locally the application over the > same production db to see what happens. > > Now that I explained the situation, I am going to ask if, in your > opinion/experience: > > Could the JobPoller component be the root (and only) cause of the > OutOfMemory of the jvm? > > Could this issue be related to this > https://issues.apache.org/jira/browse/OFBIZ-5710? > > Dumping and analyzing the heap of the jvm could help in some way to > understand what or who fills the memory or is this operation a waste of > time? > > Is there something that we did not considered or missed during the whole > process of problem analysis? > > > I really thank you all for your attention and your help; any suggestion or > advice would really be greatly appreciated. > > Kind regards, > Giulio > > > > > > > -- > Giulio Speri > > > *Mp Styl**e Srl* > via Antonio Meucci, 37 > 41019 Limidi di Soliera (MO) > T 059/684916 > M 334/3779851 > > www.mpstyle.it >
