Hi Tamas, I remember we got some trouble with github repo?
Please try the current official repo: http://git-wip-us.apache.org/repos/asf/incubator-cloudstack.git Thanks! -- Sheng On Fri, Jun 1, 2012 at 11:19 AM, Tamas Monos <tam...@veber.co.uk> wrote: > Hi, > > I have used git to get the cloudstack source from github to have a look at > this problem. > I could not find that part of code so I run a "grep -R usageRecordGeneric > ./*" in the source folder and got no results. > I can't find the code you've pasted earlier. Am I missing something? > > Regards > > > -----Original Message----- > From: Nitin Mehta [mailto:nitin.me...@citrix.com] > Sent: 31 May 2012 17:58 > To: cloudstack-dev@incubator.apache.org > Subject: RE: API usage records and SQL > > Hi Tamas, > Asking for 500 records takes 5-6 seconds and 1000 records taking 12 seconds > shows a linear increase and so doesn't seem like there is any performance > degradation of any sort here. > From a quick glance at the code it seems that your analysis " It seems there > is a conversion/parsing algorithm somewhere in CS that struggles with the > data " might be correct. > I see that after fetching the data from DB, CS iterates over each record and > makes a DB call for setting the usage id (conversion from ID to UUID) > depending on the usage type which Indicates the DB table to look for. Example > below > > for (Object usageRecordGeneric : usageRecords) { > if(usageRecord.getUsageType() == UsageTypes.RUNNING_VM || > usageRecord.getUsageType() == UsageTypes.ALLOCATED_VM){ > //VM Instance ID > > usageRecResponse.setUsageId(identityDao.getIdentityUuid("vm_instance", > usageRecord.getUsageId().toString())); > } else if(usageRecord.getUsageType() == UsageTypes.IP_ADDRESS){ > //IP Address ID > > usageRecResponse.setUsageId(identityDao.getIdentityUuid("user_ip_address", > usageRecord.getUsageId().toString())); > } else if .......// So on for different usage type. > > So for fetching 1000 records its making 1 +1000 Db queries. > > > Thanks, > -Nitin > > -----Original Message----- > From: Tamas Monos [mailto:tam...@veber.co.uk] > Sent: Thursday, May 31, 2012 7:50 PM > To: cloudstack-dev@incubator.apache.org > Subject: API usage records and SQL > > Hi, > > I've spent lots of time trying to debug why does it take so long for the > management server to answer the: "listUsageRecords&pagesize=1000&page=X" > query. > When I ask for 500 records I get results in about 5-6 seconds. If I increase > the pagesize to 1000 I need to wait 12 seconds to start getting any response. > > This means to me somewhere there is a massive bottleneck either in mysql or > the way cs generates the pages for response. > I have tcpdumped and got proof that the sql server responds and delivers 1000 > records in 300ms and after the management "hesitates" for about 10 seconds > before sends the respond down to the client. > It seems there is a conversion/parsing algorithm somewhere in CS that > struggles with the data. I have and account with a month worth usage records > (35k, 35 pages) and takes about 7 minutes (408sec) to get the all the data > via "listUsageRecords&pagesize=1000&page=X". > Every page adds a ~12 seconds delay (408/35=11.6). I have checked the server > load and cpu utilization but nothing. Java is around 10% of CPU and the load > is 0.06. > > This is just one account... > Any advice or suggestion is appreciated. > > Regards > > Tamas Monos DDI > +44(0)2034687012 > Chief Technical Office > +44(0)2034687000 > Veber: The Hosting Specialists Fax +44(0)871 522 7057 > http://www.veber.co.uk<http://www.veber.co.uk/> > > Follow us on Twitter: > www.twitter.com/veberhost<http://www.twitter.com/veberhost> > Follow us on Facebook: > www.facebook.com/veberhost<http://www.facebook.com/veberhost> > > >
