Hi Abrar, When you run delete by query, documents aren't immediately deleted. They are marked as deleted and the data gets removed in the process of merging, which runs asynchronously in the background. There are some settings that make merging more "sensitive" to deletes, have a look here: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/index-modules-merge.html
Alternatively, you can force a merge by running optimize: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/indices-optimize.html#indices-optimize But this is also problematic because it will thrash your I/O and invalidate much of your caches. Usually, a better option if you have expiring data is to have time-based indices (an index per day, for example). You can then use something like the Elasticsearch Curator to expire entire indices as the days go by: https://github.com/elasticsearch/curator Here's a nice blog post about it: http://www.elasticsearch.org/blog/curator-tending-your-time-series-indices/ Using time-based indices is typically better because removing entire indices is super-fast (really remove the associated files and little metadata) and doesn't mess up with the rest of your data and caches. Best regards, Radu -- Performance Monitoring * Log Analytics * Search Analytics Solr & Elasticsearch Support * http://sematext.com/ On Tue, Apr 29, 2014 at 4:04 PM, Abrar Sheikh <[email protected]> wrote: > Hi, > > I would what to know how to rotate old records from ES index, coz when i > delete documents with delete query API it does not free up disk space. Eg. > i want to remove records older than 3 days in my index. For me disk space > is an issue. > > Thanks and Regards, > Abrar. > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/61d35912-72ad-4139-aec3-46fc89f36ac9%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/61d35912-72ad-4139-aec3-46fc89f36ac9%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAHXA0_3VZzmoJ5J8GSh88J%2BXN%2BOGyPh-bwg7W5r6QokKOMsdoQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
