Marton Greber has uploaded this change for review. (
http://gerrit.cloudera.org:8080/21723
Change subject: Add Prometheus HTTP service discovery
......................................................................
Add Prometheus HTTP service discovery
Prometheus HTTP service discovery enables automatic discovery of all
Kudu Master and TServer instances. This makes setting up monitoring for
Kudu through Prometheus much easier.
This patch implements a new endpoint: '/prometheus-sd' on Master
servers. This returns the Kudu Master and TServers groupped, according
to the Prometheus HTTP SD format [1].
For example running 1 Master and 3 TServers locally, the
'/prometheus-sd' returns the following JSON:
[
{
"targets":[
"127.0.0.1:8765"
],
"labels":{
"job":"Kudu",
"group":"masters"
}
},
{
"targets":[
"127.0.0.1:9871",
"127.0.0.1:9875",
"127.0.0.1:9873"
],
"labels":{
"job":"Kudu",
"group":"tservers"
}
}
]
I added basic unittest to verify the structure of the JSON. Moreover I
tested locally that after starting Kudu, Prometheus can connect
correctly. The steps of the manual testing:
1. Start Kudu
2. Start Prometheus with a custom config:
$ prometheus --config.file=/Users/mgreber/prometheus/prometheus.yml
The content of the config:
scrape_configs:
- job_name: 'kudu'
# Scrape targets from this job every 5 seconds.
scrape_interval: 5s
metrics_path: '/metrics_prometheus'
http_sd_configs:
- url: 'http://127.0.0.1:8765/prometheus-sd'
refresh_interval: 30s
3. Verify on the Prometheus webUI that Kudu has been discovered:
3.1. open webUI -> Status -> Service Discovery -> check that we have
4 active targets under Kudu.
3.2. open webUI -> Status -> Targets -> verify that all 4 Kudu
services are up and scraped.
(4. Just due diligence: checked that by starting Grafana as well, the
Promethes data source is working, one can execute qureys and create
dashboards with it.)
[1]
https://prometheus.io/docs/prometheus/latest/http_sd/#writing-http-service-discovery
Change-Id: I931aa72a7567c0dde43d7b7ed53a2dd0fa8bc1fe
---
M src/kudu/master/master-test.cc
M src/kudu/master/master_path_handlers.cc
M src/kudu/master/master_path_handlers.h
3 files changed, 140 insertions(+), 0 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/23/21723/1
--
To view, visit http://gerrit.cloudera.org:8080/21723
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I931aa72a7567c0dde43d7b7ed53a2dd0fa8bc1fe
Gerrit-Change-Number: 21723
Gerrit-PatchSet: 1
Gerrit-Owner: Marton Greber <[email protected]>
