Hi Guys, At the moment we're running InfluxDB (v0.13) as our visibility platform backend using Grafana as the UI fronted. Our current ingestion rate is ~10,000 metrics/sec (10 sec granularity per metric) ,this number will probably grow as we expand. At the moment we got no roll-up policies or continuous queries defined - obviously this amount of metrics requires a compaction strategy for data that is older than X days.
In order to support a reasonable SLA at scale, we thought of using DB roll-ups and compact data older than X. We were very surprised to discover the following issue: https://github.com/influxdata/influxdb/issues/6910 <https://github.com/influxdata/influxdb/issues/6910> After seeing this limitation ,the initial idea was to use continuous queries, however this "solution" is far from perfect since: 1) It forces us to write one continuous query for each field we want rolled up. 2) It forces us to create separate schema for data with lower granularity and somehow map the timeranges in Grafana to the relevant schema. 3) It requires write our own service to get the measurements + their keys continuously run this in the background. 4) It doesn't scale (CQ per metric). To be honest, I am quite surprised to see that this issue is being addressed only now (seems like a fix is targeted to v1.1). Will be glad to hear from the folks who run Influx at scale some alternative solutions to this limitation. Thanks, --Paul -- Remember to include the InfluxDB version number with all issue reports --- You received this message because you are subscribed to the Google Groups "InfluxDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/02d812c3-5a96-49d0-8e16-77af64b3ee7c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
