On 19/11/2020 22:54, Brian Brazil wrote:
On Thu, 19 Nov 2020 at 21:53, Curious <[email protected]
<mailto:[email protected]>> wrote:
Hello Everyone,
I'm using Prometheus metrics to train Facebook Prophet to forecast
data for next 15 minutes.
I plan to put back the forecasting data back into Prometheus.
Tried the following:
1. Started exposing forecasting metrics along with forecasted
timestamp
metric_name [ "{" label_name "=" `"` label_value `"` { ","
label_name "=" `"` label_value `"` } [ "," ] "}" ] value *[
timestamp ] *
But not able to insert them into Prometheus as these are dropped
by Promethues:
level=debug ts=2020-11-19T19:49:34.477Z caller=scrape.go:1255
component="scrape manager" scrape_pool=ml_test
target=http://localhost:4000/ msg="Out of bounds metric"
series="Usage{instance=\"instance01.us.west.local\"}"
Is there a way I can insert the future metrics into prometheus ?
This would then used for comparison of actual v/s forecasted and
generating alerts.
This isn't possible, future data prevents ingestion of current data so
there's a 10m sanity check. You can only really ingest over the past hour.
You could remove the timestamp and just have the system present metrics
when scraped that are actually the forecast for 15 minutes ahead (i.e. a
scrape at 13:45:03 would actual return the forecast for 14:00:03).
Then in your queries you could use the offset modifier
(https://prometheus.io/docs/prometheus/latest/querying/basics/#offset-modifier).
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/7a4898b6-bae6-efb1-1fe6-84960ae94b11%40Jahingo.com.
