[
https://issues.apache.org/jira/browse/CLOUDSTACK-9783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15951364#comment-15951364
]
ASF GitHub Bot commented on CLOUDSTACK-9783:
--------------------------------------------
Github user nvazquez commented on the issue:
https://github.com/apache/cloudstack/pull/1944
Hi @rhtyd, thanks for this great improvement! Along with @serg38 we've been
testing in our env and got some failures on `listHostsMetrics` and
`listVolumeMetrics`:
````
test_list_clusters_metrics (test_metrics_api.TestMetrics) ... === TestName:
test_list_clusters_metrics | Status : SUCCESS ===
ok
test_list_hosts_metrics (test_metrics_api.TestMetrics) ... === TestName:
test_list_hosts_metrics | Status : EXCEPTION ===
ERROR
test_list_infrastructure_metrics (test_metrics_api.TestMetrics) ... ===
TestName: test_list_infrastructure_metrics | Status : SUCCESS ===
ok
test_list_pstorage_metrics (test_metrics_api.TestMetrics) ... === TestName:
test_list_pstorage_metrics | Status : SUCCESS ===
ok
test_list_vms_metrics (test_metrics_api.TestMetrics) ... === TestName:
test_list_vms_metrics | Status : SUCCESS ===
ok
test_list_volumes_metrics (test_metrics_api.TestMetrics) ... === TestName:
test_list_volumes_metrics | Status : EXCEPTION ===
ERROR
test_list_zones_metrics (test_metrics_api.TestMetrics) ... === TestName:
test_list_zones_metrics | Status : SUCCESS ===
ok
````
````
[root@ussarlabcsmgt41 cloudstack]# cat
/tmp//MarvinLogs//1A23HU/failed_plus_exceptions.txt
2017-03-31 10:21:19,514 - CRITICAL - EXCEPTION: test_list_hosts_metrics:
['Traceback (most recent call last):\n', ' File
"/usr/local/lib/python2.7/unittest/case.py", line 331, in run\n
testMethod()\n', ' File
"/home/t_vazqnadmin/cloudstack/test/integration/smoke/test_metrics_api.py",
line 77, in test_list_hosts_metrics\n host_metric =
self.apiclient.listHostsMetrics(cmd)[0]\n', ' File
"/usr/local/lib/python2.7/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py",
line 3094, in listHostsMetrics\n response =
self.connection.marvinRequest(command, response_type=response,
method=method)\n', ' File
"/usr/local/lib/python2.7/site-packages/marvin/cloudstackConnection.py", line
379, in marvinRequest\n raise e\n', "CloudstackAPIException: Execute cmd:
listhostsmetrics failed, due to: errorCode: 530, errorText:No value specified
for 'Date'\n"]
2017-03-31 10:23:50,276 - CRITICAL - EXCEPTION: test_list_volumes_metrics:
['Traceback (most recent call last):\n', ' File
"/usr/local/lib/python2.7/unittest/case.py", line 331, in run\n
testMethod()\n', ' File
"/home/t_vazqnadmin/cloudstack/test/integration/smoke/test_metrics_api.py",
line 172, in test_list_volumes_metrics\n lvm =
self.apiclient.listVolumesMetrics(cmd)[0]\n', ' File
"/usr/local/lib/python2.7/site-packages/marvin/cloudstackAPI/cloudstackAPIClient.py",
line 954, in listVolumesMetrics\n response =
self.connection.marvinRequest(command, response_type=response,
method=method)\n', ' File
"/usr/local/lib/python2.7/site-packages/marvin/cloudstackConnection.py", line
379, in marvinRequest\n raise e\n', "CloudstackAPIException: Execute cmd:
listvolumesmetrics failed, due to: errorCode: 530, errorText:No value specified
for 'Date'\n"]
````
Exception is thrown by `BeanUtils.copyProperties`, log on management server:
````
2017-03-31 10:28:33,257 ERROR [cloud.api.ApiServer]
(catalina-exec-15:ctx-7fb0b31e ctx-6ded5acb) unhandled exception executing api
command: [Ljava.lang.String;@63f1496d
org.apache.commons.beanutils.ConversionException: No value specified for
'Date'
at
org.apache.commons.beanutils.converters.AbstractConverter.handleMissing(AbstractConverter.java:310)
at
org.apache.commons.beanutils.converters.AbstractConverter.convert(AbstractConverter.java:136)
at
org.apache.commons.beanutils.converters.ConverterFacade.convert(ConverterFacade.java:60)
at
org.apache.commons.beanutils.BeanUtilsBean.convert(BeanUtilsBean.java:1078)
at
org.apache.commons.beanutils.BeanUtilsBean.copyProperty(BeanUtilsBean.java:437)
at
org.apache.commons.beanutils.BeanUtilsBean.copyProperties(BeanUtilsBean.java:286)
at
org.apache.commons.beanutils.BeanUtils.copyProperties(BeanUtils.java:137)
at
org.apache.cloudstack.metrics.MetricsServiceImpl.listHostMetrics(MetricsServiceImpl.java:244)
at
org.apache.cloudstack.api.ListHostsMetricsCmd.execute(ListHostsMetricsCmd.java:48)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150)
at com.cloud.api.ApiServer.queueCommand(ApiServer.java:709)
at com.cloud.api.ApiServer.handleRequest(ApiServer.java:533)
at
com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:299)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:129)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:126)
at com.cloud.api.ApiServlet.doGet(ApiServlet.java:88)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:299)
at
org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcessor.java:889)
at
org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.process(Http11NioProtocol.java:732)
at
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:2274)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
````
In our case it failed for this `hostResponse`:
````
{"id":"be606171-57e0-4cef-99e3-d4f2d47478b4","name":"ussfoldcsesx116.adslab.local","state":"Up","disconnected":"Mar
29, 2017 11:24:21
PM","type":"Routing","ipaddress":"10.140.32.116","zoneid":"0d074f25-ed31-482f-8bc5-44c9314fc417","zonename":"ECS-NSX","podid":"01d4848c-5357-442a-bfce-d0c691fe5bdf","podname":"POD1","version":"4.9.3.0-SNAPSHOT","hypervisor":"VMware","cpusockets":2,"cpunumber":48,"cpuspeed":2596,"cpuallocated":"1.2%","cpuused":"0.3%","cpuwithoverprovisioning":"124608.0","networkkbsread":0,"networkkbswrite":0,"memorytotal":137308676096,"memoryallocated":2147483648,"memoryused":6262890496,"capabilities":"hvm","lastpinged":"Jan
17, 1970 12:27:11
PM","managementserverid":345051565299,"clusterid":"e13b0968-a61e-474f-939e-8d03a088a9cc","clustername":"ussfoldvs6vc027.adslab.local/ECS-CSDEV/CLD100","clustertype":"ExternalManaged","islocalstorageactive":false,"created":"Apr
19, 2016 3:19:24 PM","events":"PingTimeout; StartAgentRebalance; HostDown;
AgentConnected; AgentDisconnected; ManagementServerDown; ShutdownRequested;
Ping;
Remove","hosttags":"test","outofbandmanagement":{"powerstate":"Disabled","enabled":false},"resourcestate":"Enabled","hypervisorversion":"6.0","hahost":false}
````
And `metricsResponse` after failure:
````
{"id":"be606171-57e0-4cef-99e3-d4f2d47478b4","name":"ussfoldcsesx116.adslab.local","disconnected":"Mar
29, 2017 11:24:21
PM","type":"Routing","ipaddress":"10.140.32.116","podid":"01d4848c-5357-442a-bfce-d0c691fe5bdf","podname":"POD1","hypervisor":"VMware","cpusockets":2,"cpunumber":48,"cpuspeed":2596,"cpuallocated":"1.2%","cpuused":"0.3%","cpuwithoverprovisioning":"124608.0","averageload":0,"networkkbsread":0,"networkkbswrite":0,"memorytotal":137308676096,"memoryallocated":2147483648,"memoryused":6262890496,"disksizetotal":0,"disksizeallocated":0,"capabilities":"hvm","lastpinged":"Jan
17, 1970 12:27:11
PM","managementserverid":345051565299,"clusterid":"e13b0968-a61e-474f-939e-8d03a088a9cc","clustername":"ussfoldvs6vc027.adslab.local/ECS-CSDEV/CLD100","clustertype":"ExternalManaged","islocalstorageactive":false,"created":"Apr
19, 2016 3:19:24 PM","events":"PingTimeout; StartAgentRebalance; HostDown;
AgentConnected; AgentDisconnected; ManagementServerDown; ShutdownRequested;
Ping;
Remove","hosttags":"test","hasenoughcapacity":false,"hypervisorversion":"6.0","hahost":false,"jobstatus":0}
````
Did you have a failure like this?
> Improve metrics view performance
> --------------------------------
>
> Key: CLOUDSTACK-9783
> URL: https://issues.apache.org/jira/browse/CLOUDSTACK-9783
> Project: CloudStack
> Issue Type: Improvement
> Security Level: Public(Anyone can view this level - this is the
> default.)
> Reporter: Rohit Yadav
> Assignee: Rohit Yadav
> Fix For: Future, 4.10.0.0, 4.9.3.0
>
>
> Metrics view is a pure frontend feature, where several API calls are made to
> generate the metrics view tabular data. In very large environments, rendering
> of these tables can take a lot of time, especially when there is high
> latency. The improvement task is to reimplement this feature by moving the
> logic to backend so metrics calculations happen at the backend and final
> result can be served by a single API request.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
