On Sat, 27 Jun 2020 at 01:07, Kevin Vasko <[email protected]> wrote: > I am new to Prometheus so i’m sure it’s something I’m overlooking. > > Prometheus is version : 2.18.1 > > There is data as recently as 1 min ago in prometheus if i run this query > (which makes sense, bc it’s pulling every 15s): > > nvidia_gpu_power_usage{job=“gpu_nodes”} > > If i run a query on that API within the last few hours I get nothing back > for machines in “gpu_nodes”. >
http://demo.robustperception.io:9090/api/v1/series?match%5B%5D=up&start=2020-06-27T9:23:00Z&end=2020-06-28T23:23:00Z is working for me (the start is ~15 minutes before I'm writing this), so this doesn't seem to be broken. I'd suggest checking that node_uname_info is indeed coming from the GPU job. -- Brian Brazil www.robustperception.io -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAHJKeLoM_W8zUFX1Mi6DxF96MVvPa-WiLLpJDVRf%2BM3DYLJASg%40mail.gmail.com.
