On Tuesday, December 20, 2016 at 2:22:23 AM UTC-8, [email protected] wrote: > I'm running Influx 1.1.1 (and Telegraf 1.1.1), and have been collecting lots > of data for a couple of weeks now. > > I'd like to delete data from hosts that no longer exist (ie. aren't sending > anything new), perhaps after (say) a week or so. That is, I'd like to: > > 1) Identify which hosts haven't sent any data for longer than time X > 2) Delete all the data those hosts sent (but obviously nothing else) > > I've been playing with Retention Policies, but they seem to act on all data, > so will happily remove things older than a certain amount, but not stuff > younger than that which hasn't been updated. > > I'm guessing i'm going to be in to direct queries to do this (Influx queries > are a bit new to me) - that being the case, if a query is going to be > long/slow/expensive, then can I break it into pieces so it only deletes a > chunk of data at a time? > > Anyone got an approach or some tips how I might achieve this sort of thing?
Why do you want to delete the "stale" hosts? Are you writing queries that include the stale hosts? Maybe you can set a specific condition on the query to exclude the hosts you know you don't want. When retention policies are enforced, they act on an entire shard [1] at a time. Depending on your retention policy duration, and your shard duration, you could just wait until the shard has expired, and all series in that shard will be deleted. If you do need to remove an entire series and you can't wait for RP enforcement, DROP SERIES [2] is the command you want. However, the system is not designed to optimize for arbitrary removal of data [3], so it is best to run this command during off-peak load. [1] https://docs.influxdata.com/influxdb/v1.1/concepts/glossary/#shard [2] https://docs.influxdata.com/influxdb/v1.1/query_language/database_management/#drop-series-from-the-index-with-drop-series [3] https://docs.influxdata.com/influxdb/v1.1/concepts/insights_tradeoffs/ -- Remember to include the version number! --- You received this message because you are subscribed to the Google Groups "InfluxData" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/c2f82583-8be8-49de-b8c6-9013972a643e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
