Nice testing, Chathuri. Marlon
On 5/30/14 3:44 PM, Chathuri Wimalasena wrote: > I did some performance testing with JConsole. Attaching two images with > caching and without. I ran getAllUserExperiments 500 times with 180 > experiments in the database. To run 500 iterations with caching is like 5 > mins but without caching it took more than a hour. In both cases memory and > cpu usages are not much difference as I feel. But it seems we have a memory > leak somewhere in our code since memory usage is increasing over the time. > I will dig more into it. > > > On Fri, May 30, 2014 at 9:30 AM, Marlon Pierce <[email protected]> wrote: > >> +1 very nice improvement. >> On 5/29/14 2:41 PM, Saminda Wijeratne wrote: >>> wow.... thats great news... Could you also monitor memory and CPU usage >>> impact? See also if there's any drawbacks that'll affect us for using >>> openJPA caching. >>> >>> >>> On Thu, May 29, 2014 at 11:54 AM, Lahiru Gunathilake <[email protected]> >>> wrote: >>> >>>> Sounds great Chathuri, I think we can tweak the caching configuration of >>>> openJPA and find out the best caching configuration and publish it in >> wiki. >>>> If you can make the caching parameters configurable in >>>> airavata-server.properties that would be great. >>>> >>>> Regards >>>> Lahiru >>>> >>>> >>>> On Thu, May 29, 2014 at 2:42 PM, Chathuri Wimalasena < >> [email protected] >>>>> wrote: >>>>> I enabled openJPA caching and the results are pretty impressive. I have >>>>> 180 experiments in default derby database. I query >> getAllUserExperiments >>>>> 100 times and average time is around 7000 - 8000 milliseconds without >>>>> enabling caching. After enabling caching that number reduced to >> average of >>>>> 400 - 500 milliseconds. >>>>> >>>>> >>>>> On Tue, May 27, 2014 at 4:21 PM, Chathuri Wimalasena < >>>>> [email protected]> wrote: >>>>> >>>>>> We can try openJPA caching [1] and see how it improves the >> performance. >>>>>> This will have minimal changes to the existing code. >>>>>> >>>>>> [1] >>>>>> >> http://pic.dhe.ibm.com/infocenter/wasinfo/v7r0/index.jsp?topic=%2Fcom.ibm.websphere.express.doc%2Finfo%2Fexp%2Fae%2Ftejb_datcacheconfig.html >>>>>> >>>>>> On Tue, May 27, 2014 at 12:38 PM, Supun Kamburugamuva < >> [email protected] >>>>>>> wrote: >>>>>>> I'm not 100% percent sure on the architecture and the requirements. >> May >>>>>>> be you should look in to Redis or Memcache. >>>>>>> >>>>>>> Thanks, >>>>>>> Supun.. >>>>>>> >>>>>>> >>>>>>> On Tue, May 27, 2014 at 12:35 PM, Lahiru Gunathilake < >> [email protected] >>>>>>>> wrote: >>>>>>>> I think we can implement this from the scratch and won't be a hard >>>>>>>> thing to do. >>>>>>>> >>>>>>>> >>>>>>>> On Tue, May 27, 2014 at 12:25 PM, Saminda Wijeratne < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> +1. >>>>>>>>> >>>>>>>>> Do you know any good frameworks which supports this? IMO the tricky >>>>>>>>> part is when to know that the data in cache has expired. >> Jackrabbit had a >>>>>>>>> feature where it does a callback whenever something gets updated >> in some >>>>>>>>> tree path. If we restrict all registry access through a single >>>>>>>>> component/layer instance we would be able to do the same by a >> subscription >>>>>>>>> pattern. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, May 27, 2014 at 9:04 AM, Lahiru Gunathilake < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> Hi Devs, >>>>>>>>>> >>>>>>>>>> In our current implementation we have large number of Experiment >>>>>>>>>> retrieval and experiment storing happen in between experiment >> creation and >>>>>>>>>> experiment completion. We do not really parse these data-model >> objects >>>>>>>>>> between component and we simply parse the ids of these experiment >> so every >>>>>>>>>> component has to retrieve them everytime. I think programatically >> this >>>>>>>>>> approach looks much cleaner than parsing big objects. But to make >> this more >>>>>>>>>> efficient we can use a cachedRegistry implementation as another >>>>>>>>>> implementation of registry and make sure we do not get objects >> all the way >>>>>>>>>> from the database. >>>>>>>>>> >>>>>>>>>> Each component can init its own cache registry object and it will >>>>>>>>>> build a cache on that module and update the cache if some other >> component >>>>>>>>>> had changed the data-model objects. IMHO if we implement a good >> caching >>>>>>>>>> layer on our current data-model airavata registry will be more >> efficient. >>>>>>>>>> WDYT ? >>>>>>>>>> >>>>>>>>>> Regards >>>>>>>>>> Lahiru >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> System Analyst Programmer >>>>>>>>>> PTI Lab >>>>>>>>>> Indiana University >>>>>>>>>> >>>>>>>> -- >>>>>>>> System Analyst Programmer >>>>>>>> PTI Lab >>>>>>>> Indiana University >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Supun Kamburugamuva >>>>>>> Member, Apache Software Foundation; http://www.apache.org >>>>>>> E-mail: [email protected]; Mobile: +1 812 369 6762 >>>>>>> Blog: http://supunk.blogspot.com >>>>>>> >>>>>>> >>>> -- >>>> System Analyst Programmer >>>> PTI Lab >>>> Indiana University >>>> >>
