[ https://issues.apache.org/jira/browse/AMBARI-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hari Sekhon updated AMBARI-24295: --------------------------------- Description: Request to integrate OpenTSDB for metrics collection and replace the Ambari Metrics Collector with OpenTSDB. # OpenTSDB is a much more widely used, tested and scaled metrics time series database # OpenTSDB is already run by Hortonworks clients for doing real world metrics collection at scale. For example, on one same cluster where AMS is collecting under 2500 metrics, OpenTSDB is collecting over 250,000 different metrics with much higher dimensionality of tags (AMS monitors a handful of hosts while OpenTSDB monitors nearly all hosts in company) on the same hardware # HBase - both AMS and OpenTSDB already run on HBase, so only AMS Collector needs to be swapped out # Grafana Compatibility - standard centralised Grafana as found in companies doesn't have the data source to query Ambari AMS but can query OpenTSDB natively, so OpenTSDB is more compatible with the outside monitoring ecosystem # Scaling - Ambari should be able to scale OpenTSDB easily from out of the box smaller deployment just to support local cluster, to any scale for entire company solution just by adding RegionServers and TSD instances, further driving Hadoop + HBase adoption and revenue. While AMS can benefit from HBase scaling, I don't think AMS is designed for scale and general purpose like OpenTSDB - there isn't an option for multiple load balancer Metrics Collectors like with OpenTSDB's TSDs # MapR has already been using OpenTSDB for several releases now for their metrics collection was: Request to integrate OpenTSDB for metrics collection and replace the Ambari Metrics Collector with OpenTSDB. # OpenTSDB is a much more widely used, tested and scaled metrics time series database. # OpenTSDB is already run by Hortonworks clients for doing real world metrics collection at scale. For example, on one same cluster where AMS is collecting under 2500 metrics, OpenTSDB is collecting over 250,000 different metrics with much higher dimensionality of tags (AMS monitors a handful of hosts while OpenTSDB monitors nearly all hosts in company) on the same hardware # HBase - both AMS and OpenTSDB already run on HBase, so only AMS Collector needs to be swapped out # Grafana Compatibility - standard centralised Grafana as found in companies doesn't have the data source to query Ambari AMS but can query OpenTSDB natively, so OpenTSDB is more compatible with the outside monitoring ecosystem # Scaling - Ambari should be able to scale OpenTSDB easily from out of the box smaller deployment just to support local cluster, to any scale for entire company solution just by adding RegionServers and TSD instances, further driving Hadoop + HBase adoption and revenue. While AMS can benefit from HBase scaling, I don't think AMS is designed for scale and general purpose like OpenTSDB - there isn't an option for multiple load balancer Metrics Collectors like with OpenTSDB's TSDs # MapR has already been using OpenTSDB for several releases now for their metrics collection. > Integrate OpenTSDB + replace Ambari Metrics Collector > ----------------------------------------------------- > > Key: AMBARI-24295 > URL: https://issues.apache.org/jira/browse/AMBARI-24295 > Project: Ambari > Issue Type: Improvement > Components: ambari-metrics > Affects Versions: 2.5.2 > Environment: HDP 2.6 > Reporter: Hari Sekhon > Priority: Major > > Request to integrate OpenTSDB for metrics collection and replace the Ambari > Metrics Collector with OpenTSDB. > # OpenTSDB is a much more widely used, tested and scaled metrics time series > database > # OpenTSDB is already run by Hortonworks clients for doing real world > metrics collection at scale. For example, on one same cluster where AMS is > collecting under 2500 metrics, OpenTSDB is collecting over 250,000 different > metrics with much higher dimensionality of tags (AMS monitors a handful of > hosts while OpenTSDB monitors nearly all hosts in company) on the same > hardware > # HBase - both AMS and OpenTSDB already run on HBase, so only AMS Collector > needs to be swapped out > # Grafana Compatibility - standard centralised Grafana as found in companies > doesn't have the data source to query Ambari AMS but can query OpenTSDB > natively, so OpenTSDB is more compatible with the outside monitoring ecosystem > # Scaling - Ambari should be able to scale OpenTSDB easily from out of the > box smaller deployment just to support local cluster, to any scale for entire > company solution just by adding RegionServers and TSD instances, further > driving Hadoop + HBase adoption and revenue. While AMS can benefit from HBase > scaling, I don't think AMS is designed for scale and general purpose like > OpenTSDB - there isn't an option for multiple load balancer Metrics > Collectors like with OpenTSDB's TSDs > # MapR has already been using OpenTSDB for several releases now for their > metrics collection -- This message was sent by Atlassian JIRA (v7.6.3#76005)