Got it. Thank you very much Brian. On Monday, September 13, 2021 at 3:03:24 PM UTC+8 Brian Candler wrote:
> Yes, I showed that before. You write your PromQL query like this: > > (mymetric) * *on(instance) > group_left(hostname,domainname) node_uname_info* > > where "mymetric" is whatever query you would have written before. > > See also: > https://www.robustperception.io/how-to-have-labels-for-machine-roles > https://www.robustperception.io/exposing-the-software-version-to-prometheus > > This works as long as mymetric and node_uname_info have *exactly* the same > value for the "instance" label. In particular, you need to make sure that > the exporter port number (:9100) *doesn't* appear in the instance label > of node_uname info. > If that's not the case, then you need to fix it: > > https://www.robustperception.io/controlling-the-instance-label > https://groups.google.com/forum/#!topic/prometheus-users/I4dVtwI5YrM > > https://groups.google.com/forum/#!msg/prometheus-users/PPwF-2CQiSc/FsSvfCm3AwAJ > > On Monday, 13 September 2021 at 06:47:14 UTC+1 [email protected] wrote: > >> Hi Brian, >> >> We are trying another approach regarding the case. >> >> Our targets are discoverd by file discvoery. If we define some of the >> labels by ourselves, whether the value of the label can be obtained by >> node_uname_info? >> >> [image: Screenshot 2021-09-13 134538.png] >> >> eg.: >> If adding a new label label_A="label_a_value", then whether the value of >> the label can be got with node_uname_info? >> >> On Friday, September 10, 2021 at 4:03:37 PM UTC+8 nina guo wrote: >> >>> Thank you very much Brian >>> On Friday, September 10, 2021 at 3:49:07 PM UTC+8 Brian Candler wrote: >>> >>>> The script will go wherever node_exporter runs. Normally this would be >>>> on the host, but some people run node_exporter inside a pod; if it's in a >>>> pod, then you'll need to bind-mount /etc/sysconfig/network/virtualip so >>>> that the pod can read it. >>>> >>>> There are some bugs in that script. In particular you'll need >>>> double-quotes around the echo which does expansion, and you'll need to put >>>> double-quotes around the label values too. >>>> echo "virtualip{interface=\"$ET\",short_name=\"$SHORT_NAME\"} 1" > >>>> ...etc >>>> >>>> On Friday, 10 September 2021 at 07:23:59 UTC+1 [email protected] >>>> wrote: >>>> >>>>> #!/bin/bash >>>>> # >>>>> # Description: Expose metrics from virtualip. >>>>> # >>>>> # >>>>> OUTPUT=$(cat "/etc/sysconfig/network/virtualip") >>>>> ET=$（cut -d":" -f 1 OUTPUT） >>>>> SHORT_NAME=$(cut -d":" -f 2 OUTPUT) >>>>> >>>>> echo '# HELP show virtualip.' >>>>> echo '# TYPE virtualip' >>>>> >>>>> echo 'virtualip{interface=ET,short_name=SHORT_NAME} 1' > >>>>> /var/lib/node_exporter/textfile_collector/virtualip.prom.$$ >>>>> mv /var/lib/node_exporter/textfile_collector/virtualip.prom.$$ >>>>> /var/lib/node_exporter/textfile_collector/virtualip.prom >>>>> >>>>> >>>>> ------------------------------------------------------------------------------------------------------------------------------------------ >>>>> I'm writing the script as above. >>>>> Our Prometheus solution is deployed in k8s cluster. I have add >>>>> ----collector.textfile.directory to node exporter deployment file. Do we >>>>> also need to copy this script to node exporter POD? >>>>> On Thursday, September 9, 2021 at 8:17:54 PM UTC+8 Brian Candler wrote: >>>>> >>>>>> So you could simply return >>>>>> >>>>>> virtualip{interface="eth0",function_name="foo"} 1 >>>>>> >>>>>> and then prometheus itself would add "instance" and "job" labels when >>>>>> scraping. >>>>>> >>>>>> I have searched for /etc/sysconfig/network/virtualip and I see no >>>>>> reference to it on the Internet, so I am guessing this is a custom file >>>>>> you've created, and your own script is interpreting it. >>>>>> >>>>>> On Thursday, 9 September 2021 at 10:43:30 UTC+1 [email protected] >>>>>> wrote: >>>>>> >>>>>>> >>>>>>> Thank you. The output is as below. >>>>>>> (function_name) username@hostname:~> cat >>>>>>> /etc/sysconfig/network/virtualip >>>>>>> eth0:function_name >>>>>>> On Thursday, September 9, 2021 at 5:06:27 PM UTC+8 Brian Candler >>>>>>> wrote: >>>>>>> >>>>>>>> > We want to get this output "cat >>>>>>>> /etc/sysconfig/network/virtualip". So which collectors may contain >>>>>>>> this >>>>>>>> value? >>>>>>>> >>>>>>>> I am unaware of any. That looks like a RedHat-specific config file. >>>>>>>> >>>>>>>> Remember that prometheus collects *metrics* which are floating >>>>>>>> point numbers only. Anything which is not in that format has to >>>>>>>> appear in >>>>>>>> the label of a metric, and the unique set of labels defines a new >>>>>>>> timeseries. If you don't need any floating-point value for a metric, >>>>>>>> then >>>>>>>> the convention is to return a static value of "1" which makes it easy >>>>>>>> to >>>>>>>> combine with other metrics, using the multiplication operator. >>>>>>>> node_uname_info is a very good example of this, as are others like >>>>>>>> node_exporter_build_info and node_network_info. >>>>>>>> >>>>>>>> If you show us what the content of this file looks like, maybe we >>>>>>>> can suggest what a suitable metric would look like. >>>>>>>> >>>>>>>> > I checked again that the info we require can be shown with >>>>>>>> ifconfig command. We want to get "eth0:AAA" . Is there any metric >>>>>>>> include >>>>>>>> this value? >>>>>>>> >>>>>>>> You can see all of the metrics returned by node_exporter using this >>>>>>>> command: >>>>>>>> >>>>>>>> curl localhost:9100/metrics >>>>>>>> >>>>>>>> I can see metrics giving layer 2 info (MAC addresses): >>>>>>>> node_network_info{address="0a:4e:86:6c:ab:ed",broadcast="ff:ff:ff:ff:ff:ff",device="veth6698c6af",duplex="full",ifalias="",operstate="up"} >>>>>>>> >>>>>>>> 1 >>>>>>>> >>>>>>>> - there is more info here >>>>>>>> <https://www.robustperception.io/network-interface-metrics-from-the-node-exporter> >>>>>>>> >>>>>>>> - but none giving layer 3 info (IP addresses). Either this is >>>>>>>> available in >>>>>>>> an optional node_exporter collector which is disabled by default - I >>>>>>>> pointed you to the documentation on those before - or you'd need to >>>>>>>> create >>>>>>>> a new metric yourself (e.g. with textfile collector), or find another >>>>>>>> exporter that does what you want. >>>>>>>> >>>>>>>> > So we only want to get a specific static value to be shown on >>>>>>>> Grafana, it may not be a generanl metric. >>>>>>>> >>>>>>>> Everything in prometheus is a metric. Static values have to be >>>>>>>> labels on metrics. Again, see how node_uname_info does this. >>>>>>>> >>>>>>>> Of course, since Grafana is a separate piece of software, it >>>>>>>> *might* be possible in Grafana to extract information from some >>>>>>>> other source and combine it with other info in your dashboard. I >>>>>>>> don't >>>>>>>> know how you'd do that, and you'd have to ask elsewhere, because this >>>>>>>> is a >>>>>>>> mailing list for prometheus, not grafana. (Grafana has its own >>>>>>>> discussion >>>>>>>> forum). >>>>>>>> >>>>>>>> On Thursday, 9 September 2021 at 05:30:00 UTC+1 [email protected] >>>>>>>> wrote: >>>>>>>> >>>>>>>>> So we only want to get a specific static value to be shown on >>>>>>>>> Grafana, it may not be a generanl metric. >>>>>>>>> >>>>>>>>> On Thursday, September 9, 2021 at 12:02:01 PM UTC+8 nina guo wrote: >>>>>>>>> >>>>>>>>>> I checked again that the info we require can be shown with >>>>>>>>>> ifconfig command. We want to get "eth0:AAA" . Is there any metric >>>>>>>>>> include >>>>>>>>>> this value? >>>>>>>>>> >>>>>>>>>> username@hostname:~> ifconfig >>>>>>>>>> eth0 Link encap:Ethernet HWaddr FA: >>>>>>>>>> xxxxxxx >>>>>>>>>> .......................... >>>>>>>>>> >>>>>>>>>> *eth0:AAA* Link encap:Ethernet HWaddr FA: >>>>>>>>>> inet addr: Bcast: Mask: >>>>>>>>>> UP BROADCAST RUNNING MULTICAST Metric: >>>>>>>>>> >>>>>>>>>> On Thursday, September 9, 2021 at 10:53:46 AM UTC+8 nina guo >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Thank you very much for your detailed reply. >>>>>>>>>>> >>>>>>>>>>> We want to get this output "cat >>>>>>>>>>> /etc/sysconfig/network/virtualip". So which collectors may contain >>>>>>>>>>> this >>>>>>>>>>> value? >>>>>>>>>>> >>>>>>>>>>> On Wednesday, September 8, 2021 at 6:26:45 PM UTC+8 Brian >>>>>>>>>>> Candler wrote: >>>>>>>>>>> >>>>>>>>>>>> On Wednesday, 8 September 2021 at 10:00:42 UTC+1 >>>>>>>>>>>> [email protected] wrote: >>>>>>>>>>>> >>>>>>>>>>>>> May I know if there is an example for how to write this >>>>>>>>>>>>> cronjob? >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> There are lots of examples in this repo: >>>>>>>>>>>> >>>>>>>>>>>> https://github.com/prometheus-community/node-exporter-textfile-collector-scripts >>>>>>>>>>>> >>>>>>>>>>>> Basically you just write out metrics in the prometheus >>>>>>>>>>>> text-based exposition format >>>>>>>>>>>> <https://prometheus.io/docs/instrumenting/exposition_formats/> >>>>>>>>>>>> to a text file, and enable the node_exporter textfile collector >>>>>>>>>>>> <https://github.com/prometheus/node_exporter#textfile-collector> >>>>>>>>>>>> to read it. >>>>>>>>>>>> >>>>>>>>>>>> > With node exporter, whether we can obtain the info of virtual >>>>>>>>>>>> IP? >>>>>>>>>>>> >>>>>>>>>>>> I don't know, because you haven't said what sort of virtual IPs >>>>>>>>>>>> you're using, except for some CLI tool "virtualip list" which I've >>>>>>>>>>>> never >>>>>>>>>>>> seen before. >>>>>>>>>>>> >>>>>>>>>>>> You can look at the list of node_exporter collectors which are >>>>>>>>>>>> enabled >>>>>>>>>>>> by default >>>>>>>>>>>> <https://github.com/prometheus/node_exporter#enabled-by-default> >>>>>>>>>>>> and disabled by default >>>>>>>>>>>> <https://github.com/prometheus/node_exporter#disabled-by-default> >>>>>>>>>>>> - maybe there's one which meets your needs. For example, if >>>>>>>>>>>> you're using >>>>>>>>>>>> IPVS then I see some metrics for that. If you're using >>>>>>>>>>>> keepalived, then it >>>>>>>>>>>> exports its status via SNMP. (You would run snmpd on that host, >>>>>>>>>>>> and >>>>>>>>>>>> snmp_exporter on your prometheus server). >>>>>>>>>>>> >>>>>>>>>>>> > One more question is with node_uname_info, we get the >>>>>>>>>>>> nodename, this nodename is the hostname of the server by default? >>>>>>>>>>>> >>>>>>>>>>>> I believe it's whatever "uname -n" or "hostname" shows. >>>>>>>>>>>> >>>>>>>>>>> -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/b11be0f6-7554-4617-beb2-d6de35a9fd0an%40googlegroups.com.
