Grafana is running this query against Prometheus server against the Prometheus endpoint to capture the “Job” values. I have a “gpu-node” job and a “cpu-node” job configured in Prometheus.yml.
I don’t know why the following query (which Grafana is running) to pull both lists of nodes isn’t pulling all systems reporting in both “job”s unless I set change the date range to >24 hours. For example: This is <24hrs in the past from my current time api/v1/series?match%5B%5D=node_uname_info&start=2020-06-26T5:23:00Z&end=2020-06-27T23:23:00Z – this will only return 1 “job” value (cpu-node) job. This is >24 hours in the past. api/v1/series?match%5B%5D=node_uname_info&start=2020-06-25T5:23:00Z&end=2020-06-27T23:23Z – this will return all of the “job” values (both, cpu-node and gpu-node). I have checked and validated that the systems reporting into gpu-node do have data within the last 5 min. Why isn’t the endpoint showing that there is data from these systems? When/How does this endpoint get updated properly that it pulls _all_ “Job” values? Any help would be appreciated. -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/725ad01e-c576-470a-9dc7-fd762f239cc6o%40googlegroups.com.
