You can use Eclipse memory analyzer for finding the leak.I find it much easier than JProfiler.
http://www.eclipse.org/mat/ Thanks, Supun.. On Fri, May 30, 2014 at 3:44 PM, Chathuri Wimalasena <[email protected]> wrote: > I did some performance testing with JConsole. Attaching two images with > caching and without. I ran getAllUserExperiments 500 times with 180 > experiments in the database. To run 500 iterations with caching is like 5 > mins but without caching it took more than a hour. In both cases memory and > cpu usages are not much difference as I feel. But it seems we have a memory > leak somewhere in our code since memory usage is increasing over the time. > I will dig more into it. > > > On Fri, May 30, 2014 at 9:30 AM, Marlon Pierce <[email protected]> wrote: > >> +1 very nice improvement. >> On 5/29/14 2:41 PM, Saminda Wijeratne wrote: >> > wow.... thats great news... Could you also monitor memory and CPU usage >> > impact? See also if there's any drawbacks that'll affect us for using >> > openJPA caching. >> > >> > >> > On Thu, May 29, 2014 at 11:54 AM, Lahiru Gunathilake <[email protected] >> > >> > wrote: >> > >> >> Sounds great Chathuri, I think we can tweak the caching configuration >> of >> >> openJPA and find out the best caching configuration and publish it in >> wiki. >> >> >> >> If you can make the caching parameters configurable in >> >> airavata-server.properties that would be great. >> >> >> >> Regards >> >> Lahiru >> >> >> >> >> >> On Thu, May 29, 2014 at 2:42 PM, Chathuri Wimalasena < >> [email protected] >> >>> wrote: >> >>> I enabled openJPA caching and the results are pretty impressive. I >> have >> >>> 180 experiments in default derby database. I query >> getAllUserExperiments >> >>> 100 times and average time is around 7000 - 8000 milliseconds without >> >>> enabling caching. After enabling caching that number reduced to >> average of >> >>> 400 - 500 milliseconds. >> >>> >> >>> >> >>> On Tue, May 27, 2014 at 4:21 PM, Chathuri Wimalasena < >> >>> [email protected]> wrote: >> >>> >> >>>> We can try openJPA caching [1] and see how it improves the >> performance. >> >>>> This will have minimal changes to the existing code. >> >>>> >> >>>> [1] >> >>>> >> http://pic.dhe.ibm.com/infocenter/wasinfo/v7r0/index.jsp?topic=%2Fcom.ibm.websphere.express.doc%2Finfo%2Fexp%2Fae%2Ftejb_datcacheconfig.html >> >>>> >> >>>> >> >>>> On Tue, May 27, 2014 at 12:38 PM, Supun Kamburugamuva < >> [email protected] >> >>>>> wrote: >> >>>>> I'm not 100% percent sure on the architecture and the requirements. >> May >> >>>>> be you should look in to Redis or Memcache. >> >>>>> >> >>>>> Thanks, >> >>>>> Supun.. >> >>>>> >> >>>>> >> >>>>> On Tue, May 27, 2014 at 12:35 PM, Lahiru Gunathilake < >> [email protected] >> >>>>>> wrote: >> >>>>>> I think we can implement this from the scratch and won't be a hard >> >>>>>> thing to do. >> >>>>>> >> >>>>>> >> >>>>>> On Tue, May 27, 2014 at 12:25 PM, Saminda Wijeratne < >> >>>>>> [email protected]> wrote: >> >>>>>> >> >>>>>>> +1. >> >>>>>>> >> >>>>>>> Do you know any good frameworks which supports this? IMO the >> tricky >> >>>>>>> part is when to know that the data in cache has expired. >> Jackrabbit had a >> >>>>>>> feature where it does a callback whenever something gets updated >> in some >> >>>>>>> tree path. If we restrict all registry access through a single >> >>>>>>> component/layer instance we would be able to do the same by a >> subscription >> >>>>>>> pattern. >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> On Tue, May 27, 2014 at 9:04 AM, Lahiru Gunathilake < >> >>>>>>> [email protected]> wrote: >> >>>>>>> >> >>>>>>>> Hi Devs, >> >>>>>>>> >> >>>>>>>> In our current implementation we have large number of Experiment >> >>>>>>>> retrieval and experiment storing happen in between experiment >> creation and >> >>>>>>>> experiment completion. We do not really parse these data-model >> objects >> >>>>>>>> between component and we simply parse the ids of these >> experiment so every >> >>>>>>>> component has to retrieve them everytime. I think >> programatically this >> >>>>>>>> approach looks much cleaner than parsing big objects. But to >> make this more >> >>>>>>>> efficient we can use a cachedRegistry implementation as another >> >>>>>>>> implementation of registry and make sure we do not get objects >> all the way >> >>>>>>>> from the database. >> >>>>>>>> >> >>>>>>>> Each component can init its own cache registry object and it will >> >>>>>>>> build a cache on that module and update the cache if some other >> component >> >>>>>>>> had changed the data-model objects. IMHO if we implement a good >> caching >> >>>>>>>> layer on our current data-model airavata registry will be more >> efficient. >> >>>>>>>> >> >>>>>>>> WDYT ? >> >>>>>>>> >> >>>>>>>> Regards >> >>>>>>>> Lahiru >> >>>>>>>> >> >>>>>>>> -- >> >>>>>>>> System Analyst Programmer >> >>>>>>>> PTI Lab >> >>>>>>>> Indiana University >> >>>>>>>> >> >>>>>>> >> >>>>>> >> >>>>>> -- >> >>>>>> System Analyst Programmer >> >>>>>> PTI Lab >> >>>>>> Indiana University >> >>>>>> >> >>>>> >> >>>>> >> >>>>> -- >> >>>>> Supun Kamburugamuva >> >>>>> Member, Apache Software Foundation; http://www.apache.org >> >>>>> E-mail: [email protected]; Mobile: +1 812 369 6762 >> >>>>> Blog: http://supunk.blogspot.com >> >>>>> >> >>>>> >> >> >> >> -- >> >> System Analyst Programmer >> >> PTI Lab >> >> Indiana University >> >> >> >> > -- Supun Kamburugamuva Member, Apache Software Foundation; http://www.apache.org E-mail: [email protected]; Mobile: +1 812 369 6762 Blog: http://supunk.blogspot.com
