[
https://issues.apache.org/jira/browse/AMBARI-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hari Sekhon updated AMBARI-24295:
---------------------------------
Description:
Request to integrate OpenTSDB for metrics collection and replace the Ambari
Metrics Collector with OpenTSDB.
# OpenTSDB is a much more widely used, tested and scaled metrics time series
database.
# OpenTSDB is already run by Hortonworks clients for doing real world metrics
collection at scale. For example, on one same cluster where AMS is collecting
under 2500 metrics, OpenTSDB is collecting over 250,000 different metrics with
much higher dimensionality of tags (AMS monitors a handful of hosts while
OpenTSDB monitors nearly all hosts in company) on the same hardware
# HBase - both AMS and OpenTSDB already run on HBase, so only AMS Collector
needs to be swapped out
# Grafana Compatibility - standard centralised Grafana as found in companies
doesn't have the data source to query Ambari AMS but can query OpenTSDB
natively, so OpenTSDB is more compatible with the outside monitoring ecosystem
# Scaling - Ambari should be able to scale OpenTSDB easily from out of the box
smaller deployment just to support local cluster, to any scale for entire
company solution just by adding RegionServers and TSD instances, further
driving Hadoop + HBase adoption and revenue. While AMS can benefit from HBase
scaling, I don't think AMS is designed for scale and general purpose like
OpenTSDB - there isn't an option for multiple load balancer Metrics Collectors
like with OpenTSDB's TSDs
# MapR has already been using OpenTSDB for several releases now for their
metrics collection.
was:
Request to integrate OpenTSDB for metrics collection and replace the Ambari
Metrics Collector with OpenTSDB.
OpenTSDB is a much more widely used, tested and scaled metrics time series
database.
OpenTSDB is already run by Hortonworks clients for doing real world metrics
collection at scale. For example, on one same cluster where AMS is collecting
under 2500 metrics, OpenTSDB is collecting over 250,000 different metrics with
much higher dimensionality of tags (AMS monitors a handful of hosts while
OpenTSDB monitors nearly all hosts in company) on the same hardware.
Both AMS and OpenTSDB already run on HBase, so only AMS Collector needs to be
swapped out.
Grafana Compatibility - standard centralised Grafana as found in companies
doesn't have the data source to query Ambari AMS but can query OpenTSDB
natively, so OpenTSDB is more compatible with the outside monitoring ecosystem.
Ambari should be able to scale OpenTSDB easily from out of the box smaller
deployment just to support local cluster, to any scale for entire company
solution just by adding RegionServers and TSD instances, further driving Hadoop
+ HBase adoption and revenue. While AMS can benefit from HBase scaling, I don't
think AMS is designed for scale and general purpose like OpenTSDB - there isn't
an option for multiple load balancer Metrics Collectors like with OpenTSDB's
TSDs.
MapR has already been using OpenTSDB for several releases now for their metrics
collection.
> Integrate OpenTSDB + replace Ambari Metrics Collector
> -----------------------------------------------------
>
> Key: AMBARI-24295
> URL: https://issues.apache.org/jira/browse/AMBARI-24295
> Project: Ambari
> Issue Type: Improvement
> Components: ambari-metrics
> Affects Versions: 2.5.2
> Environment: HDP 2.6
> Reporter: Hari Sekhon
> Priority: Major
>
> Request to integrate OpenTSDB for metrics collection and replace the Ambari
> Metrics Collector with OpenTSDB.
> # OpenTSDB is a much more widely used, tested and scaled metrics time series
> database.
> # OpenTSDB is already run by Hortonworks clients for doing real world
> metrics collection at scale. For example, on one same cluster where AMS is
> collecting under 2500 metrics, OpenTSDB is collecting over 250,000 different
> metrics with much higher dimensionality of tags (AMS monitors a handful of
> hosts while OpenTSDB monitors nearly all hosts in company) on the same
> hardware
> # HBase - both AMS and OpenTSDB already run on HBase, so only AMS Collector
> needs to be swapped out
> # Grafana Compatibility - standard centralised Grafana as found in companies
> doesn't have the data source to query Ambari AMS but can query OpenTSDB
> natively, so OpenTSDB is more compatible with the outside monitoring ecosystem
> # Scaling - Ambari should be able to scale OpenTSDB easily from out of the
> box smaller deployment just to support local cluster, to any scale for entire
> company solution just by adding RegionServers and TSD instances, further
> driving Hadoop + HBase adoption and revenue. While AMS can benefit from HBase
> scaling, I don't think AMS is designed for scale and general purpose like
> OpenTSDB - there isn't an option for multiple load balancer Metrics
> Collectors like with OpenTSDB's TSDs
> # MapR has already been using OpenTSDB for several releases now for their
> metrics collection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
