This is what I suspected. The custom exporter is producing values that drop slightly from scrape to scrape, triggering Prometheus into thinking the counter was reset.
If you did a deriv() function, you would see a negative traffic value. On Tue, Sep 22, 2020, 19:18 Adam <[email protected]> wrote: > Here's the result of `resets(wan_ifInOctets{site=~"$site"}[5m])`: > [image: Screenshot 2020-09-22 131755.png] > > On Tuesday, September 22, 2020 at 12:30:00 PM UTC-4 [email protected] > wrote: > >> What if you graph `resets(wan_ifInOctets{site=~"$site"}[5m])`? >> >> On Tue, Sep 22, 2020 at 6:03 PM Adam <[email protected]> wrote: >> >>> I am trying to troubleshoot the cause of random jumps in the network >>> activity. I have a custom exporter that reports the current network >>> activity from WAN routers via SNMP. The data is scraped every two minutes >>> and this issue only happens on specific routers while others report no >>> issues. >>> >>> Here's a picture to describe the issue I'm facing: >>> [image: Screenshot 2020-09-22 115806.png] >>> *This metrics data is the same from Prometheus' end. I used Grafana to >>> be able to show the issue I'm having.* >>> >>> As you can tell, the raw_172.19.187.146 (yellow line) is going at a >>> smooth increase while raw_172.19.187.146 shows a massive jump for at least >>> 2-3 minutes. >>> >>> Any idea why this issue is happening? Is the query in A row acceptable >>> to show the network activity every five minutes (since Prometheus scrapes >>> it every 2 minutes)? >>> >>> Thank you and I look forward to your response. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Prometheus Users" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/prometheus-users/e43bcf3a-2a20-4ded-b782-1c8d28028643n%40googlegroups.com >>> <https://groups.google.com/d/msgid/prometheus-users/e43bcf3a-2a20-4ded-b782-1c8d28028643n%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/e953ad75-a60a-4fa4-aa96-6e3e39366330n%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-users/e953ad75-a60a-4fa4-aa96-6e3e39366330n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CABbyFmqcd8Z7Hfomm7pxROB%2BjCJTLLVat3ybaFQRFyn0F7qt4A%40mail.gmail.com.
