Hi steven, looks like db and app version mismatch. Any idea what might have caused this?
~Rajani On 13-Jun-2014, at 8:30 pm, Steve Searles <ssear...@zimcom.net<mailto:ssear...@zimcom.net>> wrote: I have resolved this my finding the proper schema for the cloud_usage DB and creating a blank db. Thank you everyone. Steven Searles, CTO | ssear...@zimcom.net<mailto:ssear...@zimcom.net> Zimcom Internet Solutions | www.zimcom.net<http://www.zimcom.net/> O: 513.231.9500 | D: 513.233.4130 On Jun 13, 2014, at 10:48 AM, Steve Searles <ssear...@zimcom.net<mailto:ssear...@zimcom.net>> wrote: After adding the virtual_size column to the cloud_usage table the listusagerecords is no longer producing an error, however in the usage.log it appears there is still a missing table, I will create this by hand and report back. 2014-06-13 14:38:55,401 ERROR [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) Usage Manager error com.cloud.utils.exception.CloudRuntimeException: Table 'cloud_usage.vm_disk_statistics' doesn't exist at com.cloud.usage.dao.UsageDaoImpl.saveVmDiskStats(UsageDaoImpl.java:377) at com.cloud.usage.dao.UsageDaoImpl_EnhancerByCloudStack_30c322bf.CGLIB$saveVmDiskStats$11(<generated>) at com.cloud.usage.dao.UsageDaoImpl_EnhancerByCloudStack_30c322bf_FastClassByCloudStack_bdbde463.invoke(<generated>) at net.sf.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) at com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125) at com.cloud.usage.dao.UsageDaoImpl_EnhancerByCloudStack_30c322bf.saveVmDiskStats(<generated>) at com.cloud.usage.UsageManagerImpl.parse(UsageManagerImpl.java:543) at com.cloud.usage.UsageManagerImpl.runInContextInternal(UsageManagerImpl.java:349) at com.cloud.usage.UsageManagerImpl$1.runInContext(UsageManagerImpl.java:291) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at com.cloud.usage.UsageManagerImpl.run(UsageManagerImpl.java:288) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:165) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:701) 2014-06-13 14:38:55,402 INFO [cloud.usage.UsageManagerImpl] (Usage-Job-1:null) usage job complete Steven Searles, CTO | ssear...@zimcom.net<mailto:ssear...@zimcom.net> Zimcom Internet Solutions | www.zimcom.net<http://www.zimcom.net/> O: 513.231.9500 | D: 513.233.4130 On Jun 13, 2014, at 10:25 AM, Steve Searles <ssear...@zimcom.net<mailto:ssear...@zimcom.net>> wrote: It looks like I am getting closer with the help of some of you privately. Here is where we are so far. Using the API calling the generateUsageRecords did nothing but create another usage job. But calling listUsagerecords produced an error, when tracked down we can see that it is caused by a missing DB column. See below. 2014-06-13 00:12:37,597 - requester.py:41 - [DEBUG] ======== START Request ======== 2014-06-13 00:12:37,597 - requester.py:41 - [DEBUG] Requesting command=listUsageRecords, args={'startdate': '2014-06-01', 'enddate': '2014-06-13'} 2014-06-13 00:12:37,598 - requester.py:41 - [DEBUG] Request sent: http://localhost:8080/client/api?apikey=GqlaXiDzG1TWAg8smpO21gzzo_6Uxq oQnk8wdpzCCOyEQ8oGcal9elX-y4R8Tm01sLTuuE40zMCZ7_yeqqZRcA&command=listU sageRecords&enddate=2014-06-13&expires=2014-06-13T04%3A22%3A37%2B0000& response=json&signatureversion=3&startdate=2014-06-01&signature=0kk0CP XBv%2BZrbg3J7oosTk7mYIM%3D 2014-06-13 00:12:37,616 - requester.py:41 - [DEBUG] Response received: None 2014-06-13 00:12:37,616 - requester.py:41 - [DEBUG] Error: : DB Exception on: com.mysql.jdbc.JDBC4PreparedStatement@d02448f: SELECT cloud_usage.id, cloud_usage.zone_id, cloud_usage.account_id, cloud_usage.domain_id, cloud_usage.description, cloud_usage.usage_display, cloud_usage.usage_type, cloud_usage.raw_usage, cloud_usage.vm_instance_id, cloud_usage.vm_name, cloud_usage.offering_id, cloud_usage.template_id, cloud_usage.usage_id, cloud_usage.type, cloud_usage.size, cloud_usage.virtual_size, cloud_usage.network_id, cloud_usage.start_date, cloud_usage.end_date FROM cloud_usage WHERE cloud_usage.start_date BETWEEN '2014-06-01 04:00:00' AND '2014-06-14 03:59:59' AND cloud_usage.end_date BETWEEN '2014-06-01 04:00:00' AND '2014-06-14 03:59:59' ORDER BY cloud_usage.start_date DESC LIMIT 0, 500 2014-06-13 00:12:37,616 - requester.py:41 - [DEBUG] ======== END Request ======== Now we are getting somewhere. When running the query in mysql we can see that a column is missing. mysql> SELECT cloud_usage.id, cloud_usage.zone_id, mysql> cloud_usage.account_id, cloud_usage.domain_id, mysql> cloud_usage.description, cloud_usage.usage_display, mysql> cloud_usage.usage_type, cloud_usage.raw_usage, mysql> cloud_usage.vm_instance_id, cloud_usage.vm_name, mysql> cloud_usage.offering_id, cloud_usage.template_id, mysql> cloud_usage.usage_id, cloud_usage.type, cloud_usage.size, mysql> cloud_usage.virtual_size, cloud_usage.network_id, mysql> cloud_usage.start_date, cloud_usage.end_date FROM cloud_usage mysql> WHERE cloud_usage.start_date BETWEEN '2014-06-01 04:00:00' AND mysql> '2014-06-14 03:59:59' AND cloud_usage.end_date BETWEEN mysql> '2014-06-01 04:00:00' AND '2014-06-14 03:59:59' ORDER BY mysql> cloud_usage.start_date DESC LIMIT 0, 500; ERROR 1054 (42S22): Unknown column 'cloud_usage.virtual_size' in 'field list' mysql> I don't see the virtual_size table being created in the cloud_usage schema file. So it looks like virtual_size and size are missing from my DB. It is my understanding that the DB is created with create-database-premium.sql create-schema-premium.sql. The create database is fine, but the schema is missing virtual_size and size as well. CREATE TABLE `cloud_usage`.`cloud_usage` ( `id` bigint unsigned NOT NULL auto_increment, `zone_id` bigint unsigned NOT NULL, `account_id` bigint unsigned NOT NULL, `domain_id` bigint unsigned NOT NULL, `description` varchar(1024) NOT NULL, `usage_display` varchar(255) NOT NULL, `usage_type` int(1) unsigned, `raw_usage` DOUBLE UNSIGNED NOT NULL, `vm_instance_id` bigint unsigned, `vm_name` varchar(255), `offering_id` bigint unsigned, `template_id` bigint unsigned, `usage_id` bigint unsigned, `type` varchar(32), `size` bigint unsigned, `network_id` bigint unsigned, `start_date` DATETIME NOT NULL, `end_date` DATETIME NOT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; Does anyone have an updated copy of the usageDB schema? I am not sure how anyone’s usage servers are working since this is a fresh install of 4.3 on my end and they should be missing the tables as well. The usage job will populate account and user_statistics because they seem not to be dependent on the cloud_usage table. Any help would be appreciated. Steve Searles On Jun 13, 2014, at 1:46 AM, Shweta Agarwal <shweta.agar...@citrix.com<mailto:shweta.agar...@citrix.com><mailto:shweta.agar...@citrix.com>> wrote: You can use generateUsageRecords api . It Generates usage records. This will generate records only if there any records to be generated, i.e if the scheduled usage job was not run or failed Request parameters Parameter Name Description Required enddate End date range for usage record query. Use yyyy-MM-dd as the date format, e.g. startDate=2009-06-03. true startdate Start date range for usage record query. Use yyyy-MM-dd as the date format, e.g. startDate=2009-06-01. true domainid List events for the specified domain. false Response Tags Response Name Description displaytext any text associated with the success or failure success true if operation is executed successfully Hope this will help . If you can share the logs it will be easier for us to help more. Thanks Shweta -----Original Message----- From: Rajani Karuturi [mailto:rajani.karut...@citrix.com] Sent: Friday, June 13, 2014 10:36 AM To: users@cloudstack.apache.org<mailto:users@cloudstack.apache.org><mailto:users@cloudstack.apache.org> Subject: Re: Usage Server Issues Can you share logs? I don't know if you can start it manually. But, you can definitely tweak the configs to start every 10 min. http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/latest/usage.html ~Rajani On 13-Jun-2014, at 8:41 am, Steve Searles <ssear...@zimcom.net<mailto:ssear...@zimcom.net><mailto:ssear...@zimcom.net>> wrote: Any experts on the usage server out there? I have a weird problem where my accounts table and user_statistics table are populated but the rest of the tables in the DB are empty (with the exception of usage_job). Does anyone know how to manually fire off the usage job from the command line rather than waiting for it to happen? Any pointers on how to debug this? I have about 40 vm's running in my lab that should be making plenty of data. CS 4.3 Steve Searles
