On Fri, 26 Jun 2020 at 21:56, Kevin Vasko <[email protected]> wrote: > Grafana is running this query against Prometheus server against the > Prometheus endpoint to capture the “Job” values. > > I have a “gpu-node” job and a “cpu-node” job configured in Prometheus.yml. > > I don’t know why the following query (which Grafana is running) to pull > both lists of nodes isn’t pulling all systems reporting in both “job”s > unless I set change the date range to >24 hours. > > For example: > > This is <24hrs in the past from my current time > > api/v1/series?match%5B%5D=node_uname_info&start=2020-06-26T5:23:00Z&end=2020-06-27T23:23:00Z > – this will only return 1 “job” value (cpu-node) job. > > This is >24 hours in the past. > api/v1/series?match%5B%5D=node_uname_info&start=2020-06-25T5:23:00Z&end=2020-06-27T23:23Z > – this will return all of the “job” values (both, cpu-node and gpu-node). > > I have checked and validated that the systems reporting into gpu-node do > have data within the last 5 min. Why isn’t the endpoint showing that there > is data from these systems? > > When/How does this endpoint get updated properly that it pulls _all_ “Job” > values? >
Things should be added instantly, it's only removing where it's a little more complicated. What version of Prometheus is this? Can you confirm that there is data coming back from a query in the same time ranges for both jobs? -- Brian Brazil www.robustperception.io -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAHJKeLpRxVosQwCNMd3F8dyCR3jXDoD1OLfiMd-xpMtPMuSMtw%40mail.gmail.com.
