[
https://issues.apache.org/jira/browse/AMBARI-24295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hari Sekhon updated AMBARI-24295:
---------------------------------
Description:
Request to integrate OpenTSDB for metrics collection and replace the Ambari
Metrics Collector with OpenTSDB.
OpenTSDB is a much more widely used, tested and scaled metrics time series
database.
OpenTSDB is already run by Hortonworks clients for doing real world metrics
collection at scale. For example, on one same cluster where AMS is collecting
under 2500 metrics, OpenTSDB is collecting over 250,000 different metrics with
much higher dimensionality of tags (AMS monitors a handful of hosts while
OpenTSDB monitors nearly all hosts in company) on the same hardware.
Both AMS and OpenTSDB already run on HBase, so only AMS Collector needs to be
swapped out.
Standard Grafana as you find in companies doesn't have the data source to query
Ambari AMS but can query OpenTSDB natively, so this is more compatible with the
outside monitoring ecosystem.
Ambari should be able to scale this easily just by adding RegionServers and TSD
instances. While AMS can benefit from HBase scaling, I don't think AMS is
designed for scale and general purpose like OpenTSDB - there isn't an option
for multiple load balancer Metrics Collectors like with OpenTSDB's TSDs.
MapR has already been using OpenTSDB for several releases now for their metrics
collection.
was:
Request to integrate OpenTSDB for metrics collection and replace the Ambari
Metrics Collector with OpenTSDB.
OpenTSDB is a much more widely used, tested and scaled metrics time series
database.
OpenTSDB is already run by Hortonworks clients for doing real world metrics
collection at scale. For example, on one same cluster where AMS is collecting
under 2500 metrics, OpenTSDB is collecting over 250,000 different metrics with
much higher dimensionality of tags (AMS monitors a handful of hosts while
OpenTSDB monitors nearly all hosts in company) on the same hardware.
Both AMS and OpenTSDB already run on HBase, so only AMS Collector needs to be
swapped out.
Standard Grafana as you find in companies doesn't have the data source to query
Ambari AMS but can query OpenTSDB natively, so this is more compatible with the
outside monitoring ecosystem.
Ambari should be able to scale this easily just by adding RegionServers and TSD
instances.
> Integrate OpenTSDB + replace Ambari Metrics Collector
> -----------------------------------------------------
>
> Key: AMBARI-24295
> URL: https://issues.apache.org/jira/browse/AMBARI-24295
> Project: Ambari
> Issue Type: Improvement
> Components: ambari-metrics
> Affects Versions: 2.5.2
> Environment: HDP 2.6
> Reporter: Hari Sekhon
> Priority: Major
>
> Request to integrate OpenTSDB for metrics collection and replace the Ambari
> Metrics Collector with OpenTSDB.
> OpenTSDB is a much more widely used, tested and scaled metrics time series
> database.
> OpenTSDB is already run by Hortonworks clients for doing real world metrics
> collection at scale. For example, on one same cluster where AMS is collecting
> under 2500 metrics, OpenTSDB is collecting over 250,000 different metrics
> with much higher dimensionality of tags (AMS monitors a handful of hosts
> while OpenTSDB monitors nearly all hosts in company) on the same hardware.
> Both AMS and OpenTSDB already run on HBase, so only AMS Collector needs to be
> swapped out.
> Standard Grafana as you find in companies doesn't have the data source to
> query Ambari AMS but can query OpenTSDB natively, so this is more compatible
> with the outside monitoring ecosystem.
> Ambari should be able to scale this easily just by adding RegionServers and
> TSD instances. While AMS can benefit from HBase scaling, I don't think AMS is
> designed for scale and general purpose like OpenTSDB - there isn't an option
> for multiple load balancer Metrics Collectors like with OpenTSDB's TSDs.
> MapR has already been using OpenTSDB for several releases now for their
> metrics collection.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
