Hi Chathuri, Great work !!
Regards Lahiru On Fri, Jun 6, 2014 at 4:20 PM, Suresh Marru <[email protected]> wrote: > Very nice, thanks for the detective work Chathuri. > > Suresh > > On Jun 6, 2014, at 3:55 PM, Chathuri Wimalasena <[email protected]> > wrote: > > > With the help of OpneJPA community, I was able to solve the memory leak > issue. It is due to the way we close the entity manager. After fixing that, > we no longer see previous memory leak. Attaching the memory graphs after > the fix and before the fix. > > > > Thanks.. > > Chathuri > > > > > > > > On Fri, May 30, 2014 at 3:50 PM, Supun Kamburugamuva <[email protected]> > wrote: > > You can use Eclipse memory analyzer for finding the leak.I find it much > easier than JProfiler. > > > > http://www.eclipse.org/mat/ > > > > Thanks, > > Supun.. > > > > > > On Fri, May 30, 2014 at 3:44 PM, Chathuri Wimalasena < > [email protected]> wrote: > > I did some performance testing with JConsole. Attaching two images with > caching and without. I ran getAllUserExperiments 500 times with 180 > experiments in the database. To run 500 iterations with caching is like 5 > mins but without caching it took more than a hour. In both cases memory and > cpu usages are not much difference as I feel. But it seems we have a memory > leak somewhere in our code since memory usage is increasing over the time. > I will dig more into it. > > > > > > On Fri, May 30, 2014 at 9:30 AM, Marlon Pierce <[email protected]> wrote: > > +1 very nice improvement. > > On 5/29/14 2:41 PM, Saminda Wijeratne wrote: > > > wow.... thats great news... Could you also monitor memory and CPU usage > > > impact? See also if there's any drawbacks that'll affect us for using > > > openJPA caching. > > > > > > > > > On Thu, May 29, 2014 at 11:54 AM, Lahiru Gunathilake < > [email protected]> > > > wrote: > > > > > >> Sounds great Chathuri, I think we can tweak the caching configuration > of > > >> openJPA and find out the best caching configuration and publish it in > wiki. > > >> > > >> If you can make the caching parameters configurable in > > >> airavata-server.properties that would be great. > > >> > > >> Regards > > >> Lahiru > > >> > > >> > > >> On Thu, May 29, 2014 at 2:42 PM, Chathuri Wimalasena < > [email protected] > > >>> wrote: > > >>> I enabled openJPA caching and the results are pretty impressive. I > have > > >>> 180 experiments in default derby database. I query > getAllUserExperiments > > >>> 100 times and average time is around 7000 - 8000 milliseconds without > > >>> enabling caching. After enabling caching that number reduced to > average of > > >>> 400 - 500 milliseconds. > > >>> > > >>> > > >>> On Tue, May 27, 2014 at 4:21 PM, Chathuri Wimalasena < > > >>> [email protected]> wrote: > > >>> > > >>>> We can try openJPA caching [1] and see how it improves the > performance. > > >>>> This will have minimal changes to the existing code. > > >>>> > > >>>> [1] > > >>>> > http://pic.dhe.ibm.com/infocenter/wasinfo/v7r0/index.jsp?topic=%2Fcom.ibm.websphere.express.doc%2Finfo%2Fexp%2Fae%2Ftejb_datcacheconfig.html > > >>>> > > >>>> > > >>>> On Tue, May 27, 2014 at 12:38 PM, Supun Kamburugamuva < > [email protected] > > >>>>> wrote: > > >>>>> I'm not 100% percent sure on the architecture and the > requirements. May > > >>>>> be you should look in to Redis or Memcache. > > >>>>> > > >>>>> Thanks, > > >>>>> Supun.. > > >>>>> > > >>>>> > > >>>>> On Tue, May 27, 2014 at 12:35 PM, Lahiru Gunathilake < > [email protected] > > >>>>>> wrote: > > >>>>>> I think we can implement this from the scratch and won't be a hard > > >>>>>> thing to do. > > >>>>>> > > >>>>>> > > >>>>>> On Tue, May 27, 2014 at 12:25 PM, Saminda Wijeratne < > > >>>>>> [email protected]> wrote: > > >>>>>> > > >>>>>>> +1. > > >>>>>>> > > >>>>>>> Do you know any good frameworks which supports this? IMO the > tricky > > >>>>>>> part is when to know that the data in cache has expired. > Jackrabbit had a > > >>>>>>> feature where it does a callback whenever something gets updated > in some > > >>>>>>> tree path. If we restrict all registry access through a single > > >>>>>>> component/layer instance we would be able to do the same by a > subscription > > >>>>>>> pattern. > > >>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> On Tue, May 27, 2014 at 9:04 AM, Lahiru Gunathilake < > > >>>>>>> [email protected]> wrote: > > >>>>>>> > > >>>>>>>> Hi Devs, > > >>>>>>>> > > >>>>>>>> In our current implementation we have large number of Experiment > > >>>>>>>> retrieval and experiment storing happen in between experiment > creation and > > >>>>>>>> experiment completion. We do not really parse these data-model > objects > > >>>>>>>> between component and we simply parse the ids of these > experiment so every > > >>>>>>>> component has to retrieve them everytime. I think > programatically this > > >>>>>>>> approach looks much cleaner than parsing big objects. But to > make this more > > >>>>>>>> efficient we can use a cachedRegistry implementation as another > > >>>>>>>> implementation of registry and make sure we do not get objects > all the way > > >>>>>>>> from the database. > > >>>>>>>> > > >>>>>>>> Each component can init its own cache registry object and it > will > > >>>>>>>> build a cache on that module and update the cache if some other > component > > >>>>>>>> had changed the data-model objects. IMHO if we implement a good > caching > > >>>>>>>> layer on our current data-model airavata registry will be more > efficient. > > >>>>>>>> > > >>>>>>>> WDYT ? > > >>>>>>>> > > >>>>>>>> Regards > > >>>>>>>> Lahiru > > >>>>>>>> > > >>>>>>>> -- > > >>>>>>>> System Analyst Programmer > > >>>>>>>> PTI Lab > > >>>>>>>> Indiana University > > >>>>>>>> > > >>>>>>> > > >>>>>> > > >>>>>> -- > > >>>>>> System Analyst Programmer > > >>>>>> PTI Lab > > >>>>>> Indiana University > > >>>>>> > > >>>>> > > >>>>> > > >>>>> -- > > >>>>> Supun Kamburugamuva > > >>>>> Member, Apache Software Foundation; http://www.apache.org > > >>>>> E-mail: [email protected]; Mobile: +1 812 369 6762 > > >>>>> Blog: http://supunk.blogspot.com > > >>>>> > > >>>>> > > >> > > >> -- > > >> System Analyst Programmer > > >> PTI Lab > > >> Indiana University > > >> > > > > > > > > > > > > -- > > Supun Kamburugamuva > > Member, Apache Software Foundation; http://www.apache.org > > E-mail: [email protected]; Mobile: +1 812 369 6762 > > Blog: http://supunk.blogspot.com > > > > > > <after_fix.png><before_fix.png> > > -- System Analyst Programmer PTI Lab Indiana University
