Yes, I showed that before. You write your PromQL query like this: (mymetric) * *on(instance) group_left(hostname,domainname) node_uname_info*
where "mymetric" is whatever query you would have written before. See also: https://www.robustperception.io/how-to-have-labels-for-machine-roles https://www.robustperception.io/exposing-the-software-version-to-prometheus This works as long as mymetric and node_uname_info have *exactly* the same value for the "instance" label. In particular, you need to make sure that the exporter port number (:9100) *doesn't* appear in the instance label of node_uname info. If that's not the case, then you need to fix it: https://www.robustperception.io/controlling-the-instance-label https://groups.google.com/forum/#!topic/prometheus-users/I4dVtwI5YrM https://groups.google.com/forum/#!msg/prometheus-users/PPwF-2CQiSc/FsSvfCm3AwAJ On Monday, 13 September 2021 at 06:47:14 UTC+1 [email protected] wrote: > Hi Brian, > > We are trying another approach regarding the case. > > Our targets are discoverd by file discvoery. If we define some of the > labels by ourselves, whether the value of the label can be obtained by > node_uname_info? > > [image: Screenshot 2021-09-13 134538.png] > > eg.: > If adding a new label label_A="label_a_value", then whether the value of > the label can be got with node_uname_info? > > On Friday, September 10, 2021 at 4:03:37 PM UTC+8 nina guo wrote: > >> Thank you very much Brian >> On Friday, September 10, 2021 at 3:49:07 PM UTC+8 Brian Candler wrote: >> >>> The script will go wherever node_exporter runs. Normally this would be >>> on the host, but some people run node_exporter inside a pod; if it's in a >>> pod, then you'll need to bind-mount /etc/sysconfig/network/virtualip so >>> that the pod can read it. >>> >>> There are some bugs in that script. In particular you'll need >>> double-quotes around the echo which does expansion, and you'll need to put >>> double-quotes around the label values too. >>> echo "virtualip{interface=\"$ET\",short_name=\"$SHORT_NAME\"} 1" > ...etc >>> >>> On Friday, 10 September 2021 at 07:23:59 UTC+1 [email protected] wrote: >>> >>>> #!/bin/bash >>>> # >>>> # Description: Expose metrics from virtualip. >>>> # >>>> # >>>> OUTPUT=$(cat "/etc/sysconfig/network/virtualip") >>>> ET=$（cut -d":" -f 1 OUTPUT） >>>> SHORT_NAME=$(cut -d":" -f 2 OUTPUT) >>>> >>>> echo '# HELP show virtualip.' >>>> echo '# TYPE virtualip' >>>> >>>> echo 'virtualip{interface=ET,short_name=SHORT_NAME} 1' > >>>> /var/lib/node_exporter/textfile_collector/virtualip.prom.$$ >>>> mv /var/lib/node_exporter/textfile_collector/virtualip.prom.$$ >>>> /var/lib/node_exporter/textfile_collector/virtualip.prom >>>> >>>> >>>> ------------------------------------------------------------------------------------------------------------------------------------------ >>>> I'm writing the script as above. >>>> Our Prometheus solution is deployed in k8s cluster. I have add >>>> ----collector.textfile.directory to node exporter deployment file. Do we >>>> also need to copy this script to node exporter POD? >>>> On Thursday, September 9, 2021 at 8:17:54 PM UTC+8 Brian Candler wrote: >>>> >>>>> So you could simply return >>>>> >>>>> virtualip{interface="eth0",function_name="foo"} 1 >>>>> >>>>> and then prometheus itself would add "instance" and "job" labels when >>>>> scraping. >>>>> >>>>> I have searched for /etc/sysconfig/network/virtualip and I see no >>>>> reference to it on the Internet, so I am guessing this is a custom file >>>>> you've created, and your own script is interpreting it. >>>>> >>>>> On Thursday, 9 September 2021 at 10:43:30 UTC+1 [email protected] >>>>> wrote: >>>>> >>>>>> >>>>>> Thank you. The output is as below. >>>>>> (function_name) username@hostname:~> cat >>>>>> /etc/sysconfig/network/virtualip >>>>>> eth0:function_name >>>>>> On Thursday, September 9, 2021 at 5:06:27 PM UTC+8 Brian Candler >>>>>> wrote: >>>>>> >>>>>>> > We want to get this output "cat /etc/sysconfig/network/virtualip". >>>>>>> So which collectors may contain this value? >>>>>>> >>>>>>> I am unaware of any. That looks like a RedHat-specific config file. >>>>>>> >>>>>>> Remember that prometheus collects *metrics* which are floating point >>>>>>> numbers only. Anything which is not in that format has to appear in >>>>>>> the >>>>>>> label of a metric, and the unique set of labels defines a new >>>>>>> timeseries. >>>>>>> If you don't need any floating-point value for a metric, then the >>>>>>> convention is to return a static value of "1" which makes it easy to >>>>>>> combine with other metrics, using the multiplication operator. >>>>>>> node_uname_info is a very good example of this, as are others like >>>>>>> node_exporter_build_info and node_network_info. >>>>>>> >>>>>>> If you show us what the content of this file looks like, maybe we >>>>>>> can suggest what a suitable metric would look like. >>>>>>> >>>>>>> > I checked again that the info we require can be shown with >>>>>>> ifconfig command. We want to get "eth0:AAA" . Is there any metric >>>>>>> include >>>>>>> this value? >>>>>>> >>>>>>> You can see all of the metrics returned by node_exporter using this >>>>>>> command: >>>>>>> >>>>>>> curl localhost:9100/metrics >>>>>>> >>>>>>> I can see metrics giving layer 2 info (MAC addresses): >>>>>>> node_network_info{address="0a:4e:86:6c:ab:ed",broadcast="ff:ff:ff:ff:ff:ff",device="veth6698c6af",duplex="full",ifalias="",operstate="up"} >>>>>>> >>>>>>> 1 >>>>>>> >>>>>>> - there is more info here >>>>>>> <https://www.robustperception.io/network-interface-metrics-from-the-node-exporter> >>>>>>> >>>>>>> - but none giving layer 3 info (IP addresses). Either this is >>>>>>> available in >>>>>>> an optional node_exporter collector which is disabled by default - I >>>>>>> pointed you to the documentation on those before - or you'd need to >>>>>>> create >>>>>>> a new metric yourself (e.g. with textfile collector), or find another >>>>>>> exporter that does what you want. >>>>>>> >>>>>>> > So we only want to get a specific static value to be shown on >>>>>>> Grafana, it may not be a generanl metric. >>>>>>> >>>>>>> Everything in prometheus is a metric. Static values have to be >>>>>>> labels on metrics. Again, see how node_uname_info does this. >>>>>>> >>>>>>> Of course, since Grafana is a separate piece of software, it *might* >>>>>>> be possible in Grafana to extract information from some other source >>>>>>> and >>>>>>> combine it with other info in your dashboard. I don't know how you'd >>>>>>> do >>>>>>> that, and you'd have to ask elsewhere, because this is a mailing list >>>>>>> for >>>>>>> prometheus, not grafana. (Grafana has its own discussion forum). >>>>>>> >>>>>>> On Thursday, 9 September 2021 at 05:30:00 UTC+1 [email protected] >>>>>>> wrote: >>>>>>> >>>>>>>> So we only want to get a specific static value to be shown on >>>>>>>> Grafana, it may not be a generanl metric. >>>>>>>> >>>>>>>> On Thursday, September 9, 2021 at 12:02:01 PM UTC+8 nina guo wrote: >>>>>>>> >>>>>>>>> I checked again that the info we require can be shown with >>>>>>>>> ifconfig command. We want to get "eth0:AAA" . Is there any metric >>>>>>>>> include >>>>>>>>> this value? >>>>>>>>> >>>>>>>>> username@hostname:~> ifconfig >>>>>>>>> eth0 Link encap:Ethernet HWaddr FA: >>>>>>>>> xxxxxxx >>>>>>>>> .......................... >>>>>>>>> >>>>>>>>> *eth0:AAA* Link encap:Ethernet HWaddr FA: >>>>>>>>> inet addr: Bcast: Mask: >>>>>>>>> UP BROADCAST RUNNING MULTICAST Metric: >>>>>>>>> >>>>>>>>> On Thursday, September 9, 2021 at 10:53:46 AM UTC+8 nina guo wrote: >>>>>>>>> >>>>>>>>>> Thank you very much for your detailed reply. >>>>>>>>>> >>>>>>>>>> We want to get this output "cat >>>>>>>>>> /etc/sysconfig/network/virtualip". So which collectors may contain >>>>>>>>>> this >>>>>>>>>> value? >>>>>>>>>> >>>>>>>>>> On Wednesday, September 8, 2021 at 6:26:45 PM UTC+8 Brian Candler >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> On Wednesday, 8 September 2021 at 10:00:42 UTC+1 >>>>>>>>>>> [email protected] wrote: >>>>>>>>>>> >>>>>>>>>>>> May I know if there is an example for how to write this >>>>>>>>>>>> cronjob? >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> There are lots of examples in this repo: >>>>>>>>>>> >>>>>>>>>>> https://github.com/prometheus-community/node-exporter-textfile-collector-scripts >>>>>>>>>>> >>>>>>>>>>> Basically you just write out metrics in the prometheus >>>>>>>>>>> text-based exposition format >>>>>>>>>>> <https://prometheus.io/docs/instrumenting/exposition_formats/> >>>>>>>>>>> to a text file, and enable the node_exporter textfile collector >>>>>>>>>>> <https://github.com/prometheus/node_exporter#textfile-collector> >>>>>>>>>>> to read it. >>>>>>>>>>> >>>>>>>>>>> > With node exporter, whether we can obtain the info of virtual >>>>>>>>>>> IP? >>>>>>>>>>> >>>>>>>>>>> I don't know, because you haven't said what sort of virtual IPs >>>>>>>>>>> you're using, except for some CLI tool "virtualip list" which I've >>>>>>>>>>> never >>>>>>>>>>> seen before. >>>>>>>>>>> >>>>>>>>>>> You can look at the list of node_exporter collectors which are >>>>>>>>>>> enabled >>>>>>>>>>> by default >>>>>>>>>>> <https://github.com/prometheus/node_exporter#enabled-by-default> >>>>>>>>>>> and disabled by default >>>>>>>>>>> <https://github.com/prometheus/node_exporter#disabled-by-default> >>>>>>>>>>> - maybe there's one which meets your needs. For example, if you're >>>>>>>>>>> using >>>>>>>>>>> IPVS then I see some metrics for that. If you're using keepalived, >>>>>>>>>>> then it >>>>>>>>>>> exports its status via SNMP. (You would run snmpd on that host, >>>>>>>>>>> and >>>>>>>>>>> snmp_exporter on your prometheus server). >>>>>>>>>>> >>>>>>>>>>> > One more question is with node_uname_info, we get the >>>>>>>>>>> nodename, this nodename is the hostname of the server by default? >>>>>>>>>>> >>>>>>>>>>> I believe it's whatever "uname -n" or "hostname" shows. >>>>>>>>>>> >>>>>>>>>> -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/6605f7da-e101-4f2b-bcb6-4a45ce6175ebn%40googlegroups.com.
