Thank you for such a detailed answer. I'm still working out what our policies should be, but: - most of our data is 'host specific', so when a host is removed, its data is nearly entirely useless (not 100% true, but nearly) - we're quite tight with disk space - we'd love to have 13 months of data available
I may end up putting the important data into specific shards, and then all the other stuff into shorter-lived shards, but until then, I thought it might be useful to delete the old data. One use case where deleting the old data is useful is in Grafana templated dashboards. If you have a template variable called 'host' which does something like: SHOW TAG VALUES FROM system WITH KEY=host ...then it'll show all the hosts that have ever reported stats. If you then select one that's been dead a while, you (seem to) get no results, which confuses people. Since the data is of low value, deleting the host's data means no more confusion. I'd agree that a better solution would be to somehow filter those hosts from the template variable (but I can't see how we could do that). Having looked at this some more and your answer, I'm thinking I'll write some sort of batch job which will: - Get a list of all known hosts - query each one for any stats in the last (say) week - For all hostnames that don't return any data, delete them Since deletion isn't cheap, I'm wondering if I could do them a chunk at a time. If I can, then I could just delete them slowly with multiple small deletes. A quick look at it suggests DROP SERIES is my only choice, so at off-peak times is probably my only option. Thanks again for your help :-) -- Remember to include the version number! --- You received this message because you are subscribed to the Google Groups "InfluxData" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/e165a95b-981c-4dc8-b573-327a6afe4b57%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
