Is data that is kept in the WAL considered "historical" in this context?
I was reading that the WAL drops data after ~2 hours. The way our system
works, hosts may be disconnected for a while during logging. Typically
it is only 30-60 minutes, but it can be longer. Is the 2 hour age limit
changeable?
For our application ideally we'd like to be able to ingest and/or import
past data to have it all in one place for analysis. It seems this isn't
possible with Prometheus currently. This is more of a nice to have than
a requirement, but we do have several years of accumulated logs in other
formats. Converting/ingesting these is a separate project vs live
logging, but one we are considering in order to get it all in one place.
Given the limitations around ingestion of historical data maybe remote
write to Victoria Metrics would work better for us. I'll also check out
Cortex and Thanos Remote Write server. Victoria Metrics seems like an
simple install to play around with the storage end at least.
Appreciate the input here folks!
James
On 8/23/20 12:40 AM, Brian Candler wrote:
No, you definitely *cannot* use pushgateway for this.
Pushgateway is only for recording the *last* metric written. It
cannot record a history of measurements; neither can prometheus import
historical measurements (by scraping or any other means).
--
You received this message because you are subscribed to a topic in the
Google Groups "Prometheus Users" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/topic/prometheus-users/i5GZ-sZPJnc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
[email protected]
<mailto:[email protected]>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/93113936-6d72-46b4-b2ac-9cc24c3a6725o%40googlegroups.com
<https://groups.google.com/d/msgid/prometheus-users/93113936-6d72-46b4-b2ac-9cc24c3a6725o%40googlegroups.com?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google Groups
"Prometheus Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/prometheus-users/b789af86-0ccf-12a3-4b0e-6dcc6f638343%40gmail.com.
