I was collecting the metrics pulling from the address http://ambari-server:8080/api/v1/clusters/test-cluster/hosts?fields=Hosts/last_heartbeat_time,metrics every 10 seconds. All the CPU, Memory and Network metrics were fine but the disk throughput was most of the time 0 (calculated by doing [read_bytes(t1+10s) - read_bytes(t1)] / 10s). Now I started to use the Amabari Collector API and it looks better since it has the "read_bps,write_bps" metrics. As you say, I can see from the collector that all the metrics are updated every 10 seconds, is that a parameter that can be configured? What is the retention time for those metrics (I am not sure if one of these is the right one https://docs.hortonworks.com/HDPDocuments/Ambari-2.2.1.1/bk_ambari_reference_guide/content/ams_aggregated_ttl_settings.html )? Thanks.
Regards, Max From: Aravindan Vijayan <[email protected]> To: "[email protected]" <[email protected]> Date: 10/02/2017 21:38 Subject: Re: how to interpret the host metrics retrieved from Rest API Hi Massimilian, Ideally, the disk metrics should be collected every 10 seconds as well. The granularity of the data depends on the time range requested as well as whether the hostname query parameter is passed or not. Can you share the exact GET call that you used to fetch disk, cpu, memory and network metrics from AMS? -- Thanks and Regards, Aravindan Vijayan From: Massimilian Mattetti Reply-To: "[email protected]" Date: Thursday, February 9, 2017 at 9:45 AM To: "[email protected]" Subject: Re: how to interpret the host metrics retrieved from Rest API I managed to calculate the Disk throughput, but I still have the problem of getting stale data. I noticed that the values of the disk metrics change once per minutes, while the values for cpu, memory and network are constantly updated. Is there a way to overcome this problem? Thanks. Best Regards, Max From: Aravindan Vijayan <[email protected]> To: "[email protected]" <[email protected]> Date: 08/02/2017 19:12 Subject: Re: how to interpret the host metrics retrieved from Rest API Hi Massimilian, Ambari Metrics Service uses psutil library to collect system metrics. The metrics's explanation can be found here - https://pythonhosted.org/psutil/ . In case there is no metric in psutil with a given name in AMS, it might have been re-named. Please refer to the following file for renaming - https://github.com/apache/ambari/blob/trunk/ambari-metrics/ambari-metrics-host-monitoring/src/main/python/core/host_info.py . -- Thanks and Regards, Aravindan Vijayan From: Massimilian Mattetti Reply-To: "[email protected]" Date: Wednesday, February 8, 2017 at 12:41 AM To: "[email protected]" Subject: how to interpret the host metrics retrieved from Rest API Hi all, I am trying to monitor the resource usage of a cluster by pulling the metrics from the rest API of Ambari (Version 2.4.2.0). I am sending request with the following path: http://ambari-server:8080/api/v1/clusters/test/hosts?fields=Hosts/last_heartbeat_time,metrics There are few metrics that are really hard to understand. For instance, the metrics of the disk: "disk" : { "disk_free" : 3278.83, "disk_total" : 4948.57, "read_bytes" : 5.0036226048E10, "read_count" : 197716.0, "read_time" : 8.7998397E7, "write_bytes" : 1.0003064832E10, "write_count" : 822311.0, "write_time" : 1.6088341E7 }, How should I interpret the read_time? Why the values take minutes to change (I am pulling every 10 seconds) even if the host is under high disk I/O? Is there a legend that explains the meaning of each one of the metrics? Thanks. Best Regards, Max
