this is the strace output
[pid 33318] nanosleep({0, 20000}, NULL) = 0
[pid 33318] nanosleep({0, 20000}, NULL) = 0
[pid 33318] nanosleep({0, 20000}, NULL) = 0
[pid 33318] nanosleep({0, 20000}, <unfinished ...>
[pid 33321] write(4, "HTTP/1.1 200 OK\r\nContent-Encodin"..., 2182
<unfinished ...>
[pid 33318] <... nanosleep resumed> NULL) = 0
[pid 33321] <... write resumed> ) = 2182
[pid 33318] nanosleep({0, 20000}, <unfinished ...>
[pid 33321] futex(0x1073648, FUTEX_WAKE_PRIVATE, 1) = 1
[pid 33317] <... futex resumed> ) = 0
[pid 33317] nanosleep({0, 3000}, <unfinished ...>
[pid 33321] read(4, 0xc000204000, 4096) = -1 EAGAIN (Resource temporarily
unavailable)
[pid 33318] <... nanosleep resumed> NULL) = 0
[pid 33321] futex(0xc0000be148, FUTEX_WAIT_PRIVATE, 0, NULL <unfinished ...>
[pid 33318] nanosleep({0, 20000}, <unfinished ...>
[pid 33317] <... nanosleep resumed> NULL) = 0
[pid 33317] futex(0x1073648, FUTEX_WAIT_PRIVATE, 0, NULL <unfinished ...>
[pid 33318] <... nanosleep resumed> NULL) = 0
[pid 33318] futex(0x1072dd0, FUTEX_WAIT_PRIVATE, 0, {60, 0} <unfinished ...>
[pid 42464] <... futex resumed> ) = -1 ETIMEDOUT (Connection timed
out)
[pid 42464] futex(0x1072dd0, FUTEX_WAKE_PRIVATE, 1) = 1
[pid 33318] <... futex resumed> ) = 0
[pid 42464] futex(0xc00055e148, FUTEX_WAIT_PRIVATE, 0, NULL <unfinished ...>
[pid 33318] nanosleep({0, 20000}, NULL) = 0
[pid 33318] futex(0x1072dd0, FUTEX_WAIT_PRIVATE, 0, {60, 0}
On Saturday, 29 February 2020 12:30:58 UTC-6, Bibin John wrote:
>
> both file have same permissions.
>
> /home/nodeexp/db $ ls -ltr
> total 8
> -rwxr-xr-x 1 user user 400 Feb 29 13:13 1.prom
> -rwxr-xr-x 1 user user 396 Feb 29 13:14 2.prom
>
> Please find content of files
>
> /home/nodeexp/db $ cat 1.prom
> kafka_topic_cluster_last_update{cluster="test", ts="1582844402760"}
> 1582844402760
> kafka_topic_rf{cluster="test", atopic="APPC-LCM-READ-REGRESSION-1848"} 3
> kafka_topic_partitioncount{cluster="test",
> atopic="APPC-LCM-READ-REGRESSION-1848"} 8
> kafka_topic_details{cluster="test",
> atopic="APPC-LCM-READ-REGRESSION-1848", bpartition="0", cleader="2",
> dreplicas="2,3,1", eisr="1,2,3", ts="1582844402760"} 3
>
> /home/nodeexp/db $ cat 2.prom
> kafka_topic_cluster_last_update{cluster="dev", ts="1582844402760"}
> 1582844402760
> kafka_topic_rf{cluster="dev", atopic="APPC-LCM-READ-REGRESSION-1848"} 3
> kafka_topic_partitioncount{cluster="dev",
> atopic="APPC-LCM-READ-REGRESSION-1848"} 8
> kafka_topic_details{cluster="dev", atopic="APPC-LCM-READ-REGRESSION-1848",
> bpartition="0", cleader="2", dreplicas="2,3,1", eisr="1,2,3",
> ts="1582844402760"} 3
> /opt/data/tools/node_exporter/data $
>
>
> Data from nodeexporter. this shows, it tried to read from both prom files
> but shows content of only 1 file.
>
> # HELP kafka_topic_cluster_last_update Metric read from
> /home/nodeexp/db/1.prom
> # TYPE kafka_topic_cluster_last_update untyped
> kafka_topic_cluster_last_update{cluster="test",ts="1582844402760"}
> 1.58284440276e+12
> # HELP kafka_topic_details Metric read from /home/nodeexp/db/1.prom
> # TYPE kafka_topic_details untyped
> kafka_topic_details{atopic="APPC-LCM-READ-REGRESSION-1848",bpartition="0",cleader="2",cluster="test",dreplicas="2,3,1",eisr="1,2,3",ts="1582844402760"}
>
> 3
> # HELP kafka_topic_partitioncount Metric read from /home/nodeexp/db/1.prom
> # TYPE kafka_topic_partitioncount untyped
> kafka_topic_partitioncount{atopic="APPC-LCM-READ-REGRESSION-1848",cluster="test"}
>
> 8
> # HELP kafka_topic_rf Metric read from /home/nodeexp/db/1.prom
> # TYPE kafka_topic_rf untyped
> kafka_topic_rf{atopic="APPC-LCM-READ-REGRESSION-1848",cluster="test"} 3
> # HELP node_exporter_build_info A metric with a constant '1' value labeled
> by version, revision, branch, and goversion from which node_exporter was
> built.
> # TYPE node_exporter_build_info gauge
> node_exporter_build_info{branch="",goversion="go1.13.6",revision="",version=""}
>
> 1
> # HELP node_scrape_collector_duration_seconds node_exporter: Duration of a
> collector scrape.
> # TYPE node_scrape_collector_duration_seconds gauge
> node_scrape_collector_duration_seconds{collector="textfile"} 0.000220073
> # HELP node_scrape_collector_success node_exporter: Whether a collector
> succeeded.
> # TYPE node_scrape_collector_success gauge
> node_scrape_collector_success{collector="textfile"} 1
> # HELP node_textfile_mtime_seconds Unixtime mtime of textfiles
> successfully read.
> # TYPE node_textfile_mtime_seconds gauge
> node_textfile_mtime_seconds{file=".prom"} 1.580203921e+09
> node_textfile_mtime_seconds{file="1.prom"} 1.583000029e+09
> node_textfile_mtime_seconds{file="2.prom"} 1.583000042e+09
> # HELP node_textfile_scrape_error 1 if there was an error opening or
> reading a file, 0 otherwise
> # TYPE node_textfile_scrape_error gauge
> node_textfile_scrape_error 0
> # HELP process_cpu_seconds_total Total user and system CPU time spent in
> seconds.
> # TYPE process_cpu_seconds_total counter
> process_cpu_seconds_total 0.03
> # HELP process_max_fds Maximum number of open file descriptors.
> # TYPE process_max_fds gauge
> process_max_fds 200000
> # HELP process_open_fds Number of open file descriptors.
> # TYPE process_open_fds gauge
> process_open_fds 10
> # HELP process_resident_memory_bytes Resident memory size in bytes.
> # TYPE process_resident_memory_bytes gauge
> process_resident_memory_bytes 9.89184e+06
> # HELP process_start_time_seconds Start time of the process since unix
> epoch in seconds.
> # TYPE process_start_time_seconds gauge
> process_start_time_seconds 1.58300043655e+09
> # HELP process_virtual_memory_bytes Virtual memory size in bytes.
> # TYPE process_virtual_memory_bytes gauge
> process_virtual_memory_bytes 7.21330176e+08
> # HELP process_virtual_memory_max_bytes Maximum amount of virtual memory
> available in bytes.
> # TYPE process_virtual_memory_max_bytes gauge
> process_virtual_memory_max_bytes -1
> # HELP promhttp_metric_handler_errors_total Total number of internal
> errors encountered by the promhttp metric handler.
> # TYPE promhttp_metric_handler_errors_total counter
> promhttp_metric_handler_errors_total{cause="encoding"} 0
> promhttp_metric_handler_errors_total{cause="gathering"} 5
> # HELP promhttp_metric_handler_requests_in_flight Current number of
> scrapes being served.
> # TYPE promhttp_metric_handler_requests_in_flight gauge
> promhttp_metric_handler_requests_in_flight 1
> # HELP promhttp_metric_handler_requests_total Total number of scrapes by
> HTTP status code.
> # TYPE promhttp_metric_handler_requests_total counter
> promhttp_metric_handler_requests_total{code="200"} 5
> promhttp_metric_handler_requests_total{code="500"} 0
> promhttp_metric_handler_requests_total{code="503"} 0
>
>
>
>
>
>
>
> On Friday, 28 February 2020 02:52:26 UTC-6, Brian Candler wrote:
>>
>> Look at the stderr output from node_exporter. My guess is that one of
>> the metrics is in an invalid format; if so, textfile_collector will report
>> and abandon the rest of the file (maybe the rest of the directory - I
>> haven't tested this)
>>
>> Another possibility is permissions on the files.
>>
>> You may also get more clues from strace:
>>
>> strace -f -p <pid-of-node-exporter>
>>
>> Look for accesses to /home/nodeexp/db/
>>
>
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/5da738bf-9b0f-40e0-8f92-6d6ffd244e23%40googlegroups.com.
