Using the PromQL browser in prometheus (or promtool on the command line),
please can you run the query
container_exporter_cpu_usage_percent{container="somecontainer"}
and show all the results returned. That is, so we can see the problem,
together with the *actual* labels of the result.
If you do that, you can show whether the metrics really are duplicated, or
in fact they are separate metrics with distinct label sets. And in the
latter case we can propose some solutions, i.e. how to aggregate the
metrics in a useful way.
On Friday, 26 November 2021 at 18:10:33 UTC [email protected] wrote:
> Hi, thanks for the quick replies. It might very well be that the issue is
> with multiple CPU cores. Now, I am really very new to Prometheus – I just
> took this from a colleague who built it, hence my unspecific question –,
> and I am irritated by the output. For instance, my query for CPU usage
> is container_exporter_cpu_usage_percent{container=~"<containername"}. Or
> for container_exporter_memory_usage{container=~"<containername>"}. The
> metrics are always duplicated, mostly with the same values (but not always,
> so it might really be the CPUs?), although I have checked that for each
> service only one container is running on my host.
> I am now thinking about removing these panels entirely from Grafana since
> I have others services (glance, for instance) which seem to provide
> plausible metrics.
>
> Best,
> Anne
>
> [email protected] schrieb am Freitag, 26. November 2021 um 19:01:49
> UTC+1:
>
>> Node exporter does not provide per container metrics, it looks at the
>> whole node. This is sometimes difficult from inside a container, thus the
>> warning.
>>
>> For metrics about containers, check out cAdvisor:
>> https://github.com/google/cadvisor
>>
>> In Prometheus it is common to have metrics broken out by labels. For
>> example, the CPU metrics in node exporter are all per CPU core, and we
>> expect you to aggregate them into whichever form you need.
>>
>> Can you share your query and what the output is, especially what labels
>> it has?
>>
>> /MR
>>
>>
>> On Fri, Nov 26, 2021, 18:50 Anne Schumann <[email protected]> wrote:
>>
>>> Hi there,
>>>
>>> I am new with Prometheus and I am facing issues with duplicate entries
>>> visualised with grafana. Since I am new, I don't know whether this is a
>>> bug (have seen several issues on GitHub) or an incorrect config. Please
>>> provide me with a couple of hints.
>>>
>>> ### Host operating system: ubuntu
>>> ### Are you running node_exporter in Docker?
>>> Yes, despite the warning, I am using prometheus with docker-compose.
>>> node-exporter is configured as a scraping job. I want to monitor a bunch of
>>> docker containers that run on my host.
>>>
>>> ### What did you do that produced an error?
>>> Queried CPU usage for my containers after restarting my containers on
>>> the machine.
>>>
>>> ### What did you expect to see?
>>> One value per container.
>>>
>>> ### What did you see instead?
>>> Two metrics per container.
>>>
>>> Can anybody help?
>>>
>>> Cheers,
>>> Anne
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Prometheus Users" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/prometheus-users/bdcf0544-d4b5-405c-befb-fcfb5599cd00n%40googlegroups.com
>>>
>>> <https://groups.google.com/d/msgid/prometheus-users/bdcf0544-d4b5-405c-befb-fcfb5599cd00n%40googlegroups.com?utm_medium=email&utm_source=footer>
>>> .
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/cefd7a2c-bf06-49aa-a821-e59fe1f1b82bn%40googlegroups.com.
