For a full forced merge (mistakenly named “optimize”), the worst case disk space is 3X the size of the index. It is common to need 2X the size of the index.
When I worked on Ultraseek Server 20+ years ago, it had the same merge behavior. I implemented a disk space check that would refuse to merge if there wasn’t enough free space. It would log an error and send an email to the admin. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Jun 16, 2020, at 1:58 PM, David Hastings <hastings.recurs...@gmail.com> > wrote: > > I cant give you a 100% true answer but ive experienced this, and what > "seemed" to happen to me was that the optimize would start, and that will > drive the size up by 3 fold, and if you out of disk space in the process > the optimize will quit since, it cant optimize, and leave the live index > pieces in tact, so now you have the "current" index as well as the > "optimized" fragments > > i cant say for certain thats what you ran into, but we found that if you > get an expanding disk it will keep growing and prevent this from happening, > then the index will contract and the disk will shrink back to only what it > needs. saved me a lot of headaches not needing to ever worry about disk > space > > On Tue, Jun 16, 2020 at 4:43 PM Raveendra Yerraguntla > <raveend...@yahoo.com.invalid> wrote: > >> >> when optimize command is issued, the expectation after the completion of >> optimization process is that the index size either decreases or at most >> remain same. In solr 7.6 cluster with 50 plus shards, when optimize command >> is issued, some of the shard's transient or older segment files are not >> deleted. This is happening randomly across all shards. When unnoticed these >> transient files makes disk full. Currently it is handled through monitors, >> but question is what is causing the transient/older files remains there. >> Are there any specific race conditions which laves the older files not >> being deleted? >> Any pointers around this will be helpful. >> TIA
