+1 for solving and +1 for engaging OpenJPA. Marlon
On 6/8/14 7:35 AM, Lahiru Gunathilake wrote: > Hi Chathuri, > > Great work !! > > Regards > Lahiru > > > On Fri, Jun 6, 2014 at 4:20 PM, Suresh Marru <[email protected]> wrote: > >> Very nice, thanks for the detective work Chathuri. >> >> Suresh >> >> On Jun 6, 2014, at 3:55 PM, Chathuri Wimalasena <[email protected]> >> wrote: >> >>> With the help of OpneJPA community, I was able to solve the memory leak >> issue. It is due to the way we close the entity manager. After fixing that, >> we no longer see previous memory leak. Attaching the memory graphs after >> the fix and before the fix. >>> Thanks.. >>> Chathuri >>> >>> >>> >>> On Fri, May 30, 2014 at 3:50 PM, Supun Kamburugamuva <[email protected]> >> wrote: >>> You can use Eclipse memory analyzer for finding the leak.I find it much >> easier than JProfiler. >>> http://www.eclipse.org/mat/ >>> >>> Thanks, >>> Supun.. >>> >>> >>> On Fri, May 30, 2014 at 3:44 PM, Chathuri Wimalasena < >> [email protected]> wrote: >>> I did some performance testing with JConsole. Attaching two images with >> caching and without. I ran getAllUserExperiments 500 times with 180 >> experiments in the database. To run 500 iterations with caching is like 5 >> mins but without caching it took more than a hour. In both cases memory and >> cpu usages are not much difference as I feel. But it seems we have a memory >> leak somewhere in our code since memory usage is increasing over the time. >> I will dig more into it. >>> >>> On Fri, May 30, 2014 at 9:30 AM, Marlon Pierce <[email protected]> wrote: >>> +1 very nice improvement. >>> On 5/29/14 2:41 PM, Saminda Wijeratne wrote: >>>> wow.... thats great news... Could you also monitor memory and CPU usage >>>> impact? See also if there's any drawbacks that'll affect us for using >>>> openJPA caching. >>>> >>>> >>>> On Thu, May 29, 2014 at 11:54 AM, Lahiru Gunathilake < >> [email protected]> >>>> wrote: >>>> >>>>> Sounds great Chathuri, I think we can tweak the caching configuration >> of >>>>> openJPA and find out the best caching configuration and publish it in >> wiki. >>>>> If you can make the caching parameters configurable in >>>>> airavata-server.properties that would be great. >>>>> >>>>> Regards >>>>> Lahiru >>>>> >>>>> >>>>> On Thu, May 29, 2014 at 2:42 PM, Chathuri Wimalasena < >> [email protected] >>>>>> wrote: >>>>>> I enabled openJPA caching and the results are pretty impressive. I >> have >>>>>> 180 experiments in default derby database. I query >> getAllUserExperiments >>>>>> 100 times and average time is around 7000 - 8000 milliseconds without >>>>>> enabling caching. After enabling caching that number reduced to >> average of >>>>>> 400 - 500 milliseconds. >>>>>> >>>>>> >>>>>> On Tue, May 27, 2014 at 4:21 PM, Chathuri Wimalasena < >>>>>> [email protected]> wrote: >>>>>> >>>>>>> We can try openJPA caching [1] and see how it improves the >> performance. >>>>>>> This will have minimal changes to the existing code. >>>>>>> >>>>>>> [1] >>>>>>> >> http://pic.dhe.ibm.com/infocenter/wasinfo/v7r0/index.jsp?topic=%2Fcom.ibm.websphere.express.doc%2Finfo%2Fexp%2Fae%2Ftejb_datcacheconfig.html >>>>>>> >>>>>>> On Tue, May 27, 2014 at 12:38 PM, Supun Kamburugamuva < >> [email protected] >>>>>>>> wrote: >>>>>>>> I'm not 100% percent sure on the architecture and the >> requirements. May >>>>>>>> be you should look in to Redis or Memcache. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Supun.. >>>>>>>> >>>>>>>> >>>>>>>> On Tue, May 27, 2014 at 12:35 PM, Lahiru Gunathilake < >> [email protected] >>>>>>>>> wrote: >>>>>>>>> I think we can implement this from the scratch and won't be a hard >>>>>>>>> thing to do. >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, May 27, 2014 at 12:25 PM, Saminda Wijeratne < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> +1. >>>>>>>>>> >>>>>>>>>> Do you know any good frameworks which supports this? IMO the >> tricky >>>>>>>>>> part is when to know that the data in cache has expired. >> Jackrabbit had a >>>>>>>>>> feature where it does a callback whenever something gets updated >> in some >>>>>>>>>> tree path. If we restrict all registry access through a single >>>>>>>>>> component/layer instance we would be able to do the same by a >> subscription >>>>>>>>>> pattern. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, May 27, 2014 at 9:04 AM, Lahiru Gunathilake < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> Hi Devs, >>>>>>>>>>> >>>>>>>>>>> In our current implementation we have large number of Experiment >>>>>>>>>>> retrieval and experiment storing happen in between experiment >> creation and >>>>>>>>>>> experiment completion. We do not really parse these data-model >> objects >>>>>>>>>>> between component and we simply parse the ids of these >> experiment so every >>>>>>>>>>> component has to retrieve them everytime. I think >> programatically this >>>>>>>>>>> approach looks much cleaner than parsing big objects. But to >> make this more >>>>>>>>>>> efficient we can use a cachedRegistry implementation as another >>>>>>>>>>> implementation of registry and make sure we do not get objects >> all the way >>>>>>>>>>> from the database. >>>>>>>>>>> >>>>>>>>>>> Each component can init its own cache registry object and it >> will >>>>>>>>>>> build a cache on that module and update the cache if some other >> component >>>>>>>>>>> had changed the data-model objects. IMHO if we implement a good >> caching >>>>>>>>>>> layer on our current data-model airavata registry will be more >> efficient. >>>>>>>>>>> WDYT ? >>>>>>>>>>> >>>>>>>>>>> Regards >>>>>>>>>>> Lahiru >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> System Analyst Programmer >>>>>>>>>>> PTI Lab >>>>>>>>>>> Indiana University >>>>>>>>>>> >>>>>>>>> -- >>>>>>>>> System Analyst Programmer >>>>>>>>> PTI Lab >>>>>>>>> Indiana University >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Supun Kamburugamuva >>>>>>>> Member, Apache Software Foundation; http://www.apache.org >>>>>>>> E-mail: [email protected]; Mobile: +1 812 369 6762 >>>>>>>> Blog: http://supunk.blogspot.com >>>>>>>> >>>>>>>> >>>>> -- >>>>> System Analyst Programmer >>>>> PTI Lab >>>>> Indiana University >>>>> >>> >>> >>> >>> >>> -- >>> Supun Kamburugamuva >>> Member, Apache Software Foundation; http://www.apache.org >>> E-mail: [email protected]; Mobile: +1 812 369 6762 >>> Blog: http://supunk.blogspot.com >>> >>> >>> <after_fix.png><before_fix.png> >> >
