Thanks, Brian, for the detailed answer. I am referring to the official doc
by datadog(https://www.datadoghq.com/blog/monitor-prometheus-metrics/),
they have only mentioned about getting the metrics from /metrics endpoint
in their example, and there is nothing else(like federation points, etc).
Is the doc incomplete or am I missing something because they have given
nothing about what to do when we need to have the target metrics also?
One more doubt -> /metrics endpoint only shows metrics of Prometheus server
or the metrics of the whole system on which Prometheus is running.
On Monday, March 2, 2020 at 4:37:04 PM UTC+5:30, Brian Candler wrote:
>
> You can't get prometheus to connect to a remote system to "push" metrics
> [except using the remote_write functionality, which is a specific protocol
> the remote system needs to support]
>
> However, other servers can scrape or "pull" metrics from prometheus, by
> scraping the /federate endpoint. That is: you can configure datadog's
> agent to scrape prometheus' /federate endpoint.
>
> The first thing to do is, make sure you scrape the /federate endpoint
> manually using curl. Try something like this:
>
> curl '127.0.0.1:9090/federate?match%5b%5d=%7bjob%3d"node"%7d'
>
> Yes it's ugly, because of URL escaping. That example uses query
> {job="node"}, escaped as follows:
>
> = -> %3d
> [ -> %5b
> ] -> %5d
> { -> %7b
> } -> %7d
>
> If course, if you don't have any metrics labeled with job="node" then
> you'll have to amend this as appropriate. To scrape absolutely everything
> (which probably isn't a good idea), use:
>
> curl 'prometheus:9090/federate?match%5b%5d=%7b__name__%3d%7e".*"%7d'
>
> When that's working, turn your attention to datadog to get it to do the
> same scraping. According to this post
> <https://www.datadoghq.com/blog/monitor-prometheus-metrics/> it should be
> possible. I haven't tested it (since I don't use datadog), but you can try:
>
> instances:
> # The prometheus endpoint to query from
> - prometheus_url:
> http://targethost:9090/federate?match%5b%5d=%7bjob%3d"node"%7d
> namespace: "my_prometheus"
> metrics:
> - "*"
>
>
> This may need adjusting depending on whether datadog does its own escaping
> of URLs. If you can't make it work, then since you're using a commercial
> service, I suggest you talk to their technical support - that's what you're
> paying for after all.
>
> What you showed before was configuring prometheus to scrape its own
> /federate endpoint, and that doesn't make sense.
>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/f2cb0b6e-f965-464a-aa08-e55fbdcff31c%40googlegroups.com.
