Sorry Brian disturb you again.
We still faced problems regarding this issue.
With file discovery we defined as below:
Target:[]
Labels: dnshost
Then the dnshost can be shown on Prometheus UI like below:
[image: Screenshot 2021-09-15 180030.png]
*1. Even the labels can be shown on Prometheus UI, we still need to write a
custom metric:*
#!/bin/bash
#
# Description: Expose metrics from virtualip.
#
#
OUTPUT=$(cat "/etc/sysconfig/network/virtualip")
ET=$（cut -d":" -f 1 OUTPUT）
SHORT_NAME=$(cut -d":" -f 2 OUTPUT)
echo '# HELP show virtualip.'
echo '# TYPE virtualip'
echo "virtualip{interface=\"$ET\",short_name=\"$SHORT_NAME\"} 1">
/var/lib/node_exporter/textfile_collector/virtualip.prom.$$
mv /var/lib/node_exporter/textfile_collector/virtualip.prom.$$
/var/lib/node_exporter/textfile_collector/virtualip.prom
*2. with the query (mymetric) * on(instance)
group_left(hostname,domainname) node_uname_info to get the value on
Grafana. *
On Monday, September 13, 2021 at 3:26:37 PM UTC+8 nina guo wrote:
> Got it. Thank you very much Brian.
>
> On Monday, September 13, 2021 at 3:03:24 PM UTC+8 Brian Candler wrote:
>
>> Yes, I showed that before. You write your PromQL query like this:
>>
>> (mymetric) * *on(instance)
>> group_left(hostname,domainname) node_uname_info*
>>
>> where "mymetric" is whatever query you would have written before.
>>
>> See also:
>> https://www.robustperception.io/how-to-have-labels-for-machine-roles
>>
>> https://www.robustperception.io/exposing-the-software-version-to-prometheus
>>
>> This works as long as mymetric and node_uname_info have *exactly* the
>> same value for the "instance" label. In particular, you need to make sure
>> that the exporter port number (:9100) *doesn't* appear in the instance
>> label of node_uname info.
>> If that's not the case, then you need to fix it:
>>
>> https://www.robustperception.io/controlling-the-instance-label
>> https://groups.google.com/forum/#!topic/prometheus-users/I4dVtwI5YrM
>>
>> https://groups.google.com/forum/#!msg/prometheus-users/PPwF-2CQiSc/FsSvfCm3AwAJ
>>
>> On Monday, 13 September 2021 at 06:47:14 UTC+1 [email protected] wrote:
>>
>>> Hi Brian,
>>>
>>> We are trying another approach regarding the case.
>>>
>>> Our targets are discoverd by file discvoery. If we define some of the
>>> labels by ourselves, whether the value of the label can be obtained by
>>> node_uname_info?
>>>
>>> [image: Screenshot 2021-09-13 134538.png]
>>>
>>> eg.:
>>> If adding a new label label_A="label_a_value", then whether the value of
>>> the label can be got with node_uname_info?
>>>
>>> On Friday, September 10, 2021 at 4:03:37 PM UTC+8 nina guo wrote:
>>>
>>>> Thank you very much Brian
>>>> On Friday, September 10, 2021 at 3:49:07 PM UTC+8 Brian Candler wrote:
>>>>
>>>>> The script will go wherever node_exporter runs. Normally this would
>>>>> be on the host, but some people run node_exporter inside a pod; if it's
>>>>> in
>>>>> a pod, then you'll need to bind-mount /etc/sysconfig/network/virtualip so
>>>>> that the pod can read it.
>>>>>
>>>>> There are some bugs in that script. In particular you'll need
>>>>> double-quotes around the echo which does expansion, and you'll need to
>>>>> put
>>>>> double-quotes around the label values too.
>>>>> echo "virtualip{interface=\"$ET\",short_name=\"$SHORT_NAME\"} 1" >
>>>>> ...etc
>>>>>
>>>>> On Friday, 10 September 2021 at 07:23:59 UTC+1 [email protected]
>>>>> wrote:
>>>>>
>>>>>> #!/bin/bash
>>>>>> #
>>>>>> # Description: Expose metrics from virtualip.
>>>>>> #
>>>>>> #
>>>>>> OUTPUT=$(cat "/etc/sysconfig/network/virtualip")
>>>>>> ET=$（cut -d":" -f 1 OUTPUT）
>>>>>> SHORT_NAME=$(cut -d":" -f 2 OUTPUT)
>>>>>>
>>>>>> echo '# HELP show virtualip.'
>>>>>> echo '# TYPE virtualip'
>>>>>>
>>>>>> echo 'virtualip{interface=ET,short_name=SHORT_NAME} 1' >
>>>>>> /var/lib/node_exporter/textfile_collector/virtualip.prom.$$
>>>>>> mv /var/lib/node_exporter/textfile_collector/virtualip.prom.$$
>>>>>> /var/lib/node_exporter/textfile_collector/virtualip.prom
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------------------------------------------------------------------
>>>>>> I'm writing the script as above.
>>>>>> Our Prometheus solution is deployed in k8s cluster. I have add
>>>>>> ----collector.textfile.directory to node exporter deployment file. Do we
>>>>>> also need to copy this script to node exporter POD?
>>>>>> On Thursday, September 9, 2021 at 8:17:54 PM UTC+8 Brian Candler
>>>>>> wrote:
>>>>>>
>>>>>>> So you could simply return
>>>>>>>
>>>>>>> virtualip{interface="eth0",function_name="foo"} 1
>>>>>>>
>>>>>>> and then prometheus itself would add "instance" and "job" labels
>>>>>>> when scraping.
>>>>>>>
>>>>>>> I have searched for /etc/sysconfig/network/virtualip and I see no
>>>>>>> reference to it on the Internet, so I am guessing this is a custom file
>>>>>>> you've created, and your own script is interpreting it.
>>>>>>>
>>>>>>> On Thursday, 9 September 2021 at 10:43:30 UTC+1 [email protected]
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> Thank you. The output is as below.
>>>>>>>> (function_name) username@hostname:~> cat
>>>>>>>> /etc/sysconfig/network/virtualip
>>>>>>>> eth0:function_name
>>>>>>>> On Thursday, September 9, 2021 at 5:06:27 PM UTC+8 Brian Candler
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> > We want to get this output "cat
>>>>>>>>> /etc/sysconfig/network/virtualip". So which collectors may contain
>>>>>>>>> this
>>>>>>>>> value?
>>>>>>>>>
>>>>>>>>> I am unaware of any. That looks like a RedHat-specific config
>>>>>>>>> file.
>>>>>>>>>
>>>>>>>>> Remember that prometheus collects *metrics* which are floating
>>>>>>>>> point numbers only. Anything which is not in that format has to
>>>>>>>>> appear in
>>>>>>>>> the label of a metric, and the unique set of labels defines a new
>>>>>>>>> timeseries. If you don't need any floating-point value for a metric,
>>>>>>>>> then
>>>>>>>>> the convention is to return a static value of "1" which makes it easy
>>>>>>>>> to
>>>>>>>>> combine with other metrics, using the multiplication operator.
>>>>>>>>> node_uname_info is a very good example of this, as are others like
>>>>>>>>> node_exporter_build_info and node_network_info.
>>>>>>>>>
>>>>>>>>> If you show us what the content of this file looks like, maybe we
>>>>>>>>> can suggest what a suitable metric would look like.
>>>>>>>>>
>>>>>>>>> > I checked again that the info we require can be shown with
>>>>>>>>> ifconfig command. We want to get "eth0:AAA" . Is there any metric
>>>>>>>>> include
>>>>>>>>> this value?
>>>>>>>>>
>>>>>>>>> You can see all of the metrics returned by node_exporter using
>>>>>>>>> this command:
>>>>>>>>>
>>>>>>>>> curl localhost:9100/metrics
>>>>>>>>>
>>>>>>>>> I can see metrics giving layer 2 info (MAC addresses):
>>>>>>>>> node_network_info{address="0a:4e:86:6c:ab:ed",broadcast="ff:ff:ff:ff:ff:ff",device="veth6698c6af",duplex="full",ifalias="",operstate="up"}
>>>>>>>>>
>>>>>>>>> 1
>>>>>>>>>
>>>>>>>>> - there is more info here
>>>>>>>>> <https://www.robustperception.io/network-interface-metrics-from-the-node-exporter>
>>>>>>>>>
>>>>>>>>> - but none giving layer 3 info (IP addresses). Either this is
>>>>>>>>> available in
>>>>>>>>> an optional node_exporter collector which is disabled by default - I
>>>>>>>>> pointed you to the documentation on those before - or you'd need to
>>>>>>>>> create
>>>>>>>>> a new metric yourself (e.g. with textfile collector), or find another
>>>>>>>>> exporter that does what you want.
>>>>>>>>>
>>>>>>>>> > So we only want to get a specific static value to be shown on
>>>>>>>>> Grafana, it may not be a generanl metric.
>>>>>>>>>
>>>>>>>>> Everything in prometheus is a metric. Static values have to be
>>>>>>>>> labels on metrics. Again, see how node_uname_info does this.
>>>>>>>>>
>>>>>>>>> Of course, since Grafana is a separate piece of software, it
>>>>>>>>> *might* be possible in Grafana to extract information from some
>>>>>>>>> other source and combine it with other info in your dashboard. I
>>>>>>>>> don't
>>>>>>>>> know how you'd do that, and you'd have to ask elsewhere, because this
>>>>>>>>> is a
>>>>>>>>> mailing list for prometheus, not grafana. (Grafana has its own
>>>>>>>>> discussion
>>>>>>>>> forum).
>>>>>>>>>
>>>>>>>>> On Thursday, 9 September 2021 at 05:30:00 UTC+1 [email protected]
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> So we only want to get a specific static value to be shown on
>>>>>>>>>> Grafana, it may not be a generanl metric.
>>>>>>>>>>
>>>>>>>>>> On Thursday, September 9, 2021 at 12:02:01 PM UTC+8 nina guo
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> I checked again that the info we require can be shown with
>>>>>>>>>>> ifconfig command. We want to get "eth0:AAA" . Is there any metric
>>>>>>>>>>> include
>>>>>>>>>>> this value?
>>>>>>>>>>>
>>>>>>>>>>> username@hostname:~> ifconfig
>>>>>>>>>>> eth0 Link encap:Ethernet HWaddr FA:
>>>>>>>>>>> xxxxxxx
>>>>>>>>>>> ..........................
>>>>>>>>>>>
>>>>>>>>>>> *eth0:AAA* Link encap:Ethernet HWaddr FA:
>>>>>>>>>>> inet addr: Bcast: Mask:
>>>>>>>>>>> UP BROADCAST RUNNING MULTICAST Metric:
>>>>>>>>>>>
>>>>>>>>>>> On Thursday, September 9, 2021 at 10:53:46 AM UTC+8 nina guo
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Thank you very much for your detailed reply.
>>>>>>>>>>>>
>>>>>>>>>>>> We want to get this output "cat
>>>>>>>>>>>> /etc/sysconfig/network/virtualip". So which collectors may contain
>>>>>>>>>>>> this
>>>>>>>>>>>> value?
>>>>>>>>>>>>
>>>>>>>>>>>> On Wednesday, September 8, 2021 at 6:26:45 PM UTC+8 Brian
>>>>>>>>>>>> Candler wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> On Wednesday, 8 September 2021 at 10:00:42 UTC+1
>>>>>>>>>>>>> [email protected] wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> May I know if there is an example for how to write this
>>>>>>>>>>>>>> cronjob?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> There are lots of examples in this repo:
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://github.com/prometheus-community/node-exporter-textfile-collector-scripts
>>>>>>>>>>>>>
>>>>>>>>>>>>> Basically you just write out metrics in the prometheus
>>>>>>>>>>>>> text-based exposition format
>>>>>>>>>>>>> <https://prometheus.io/docs/instrumenting/exposition_formats/>
>>>>>>>>>>>>> to a text file, and enable the node_exporter textfile
>>>>>>>>>>>>> collector
>>>>>>>>>>>>> <https://github.com/prometheus/node_exporter#textfile-collector>
>>>>>>>>>>>>> to read it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> > With node exporter, whether we can obtain the info of
>>>>>>>>>>>>> virtual IP?
>>>>>>>>>>>>>
>>>>>>>>>>>>> I don't know, because you haven't said what sort of virtual
>>>>>>>>>>>>> IPs you're using, except for some CLI tool "virtualip list" which
>>>>>>>>>>>>> I've
>>>>>>>>>>>>> never seen before.
>>>>>>>>>>>>>
>>>>>>>>>>>>> You can look at the list of node_exporter collectors which are
>>>>>>>>>>>>> enabled
>>>>>>>>>>>>> by default
>>>>>>>>>>>>> <https://github.com/prometheus/node_exporter#enabled-by-default>
>>>>>>>>>>>>> and disabled by default
>>>>>>>>>>>>> <https://github.com/prometheus/node_exporter#disabled-by-default>
>>>>>>>>>>>>> - maybe there's one which meets your needs. For example, if
>>>>>>>>>>>>> you're using
>>>>>>>>>>>>> IPVS then I see some metrics for that. If you're using
>>>>>>>>>>>>> keepalived, then it
>>>>>>>>>>>>> exports its status via SNMP. (You would run snmpd on that host,
>>>>>>>>>>>>> and
>>>>>>>>>>>>> snmp_exporter on your prometheus server).
>>>>>>>>>>>>>
>>>>>>>>>>>>> > One more question is with node_uname_info, we get the
>>>>>>>>>>>>> nodename, this nodename is the hostname of the server by default?
>>>>>>>>>>>>>
>>>>>>>>>>>>> I believe it's whatever "uname -n" or "hostname" shows.
>>>>>>>>>>>>>
>>>>>>>>>>>>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/b50437c9-44e6-4379-abbd-bef4e1a6cf67n%40googlegroups.com.
