Sadly with the size of our complex, spiting and adding more HW is not a viable long term solution. I guess the options we have are to run optimize regularly and/or become aggressive in our merges proactively even before solr cloud gets into this situation. Thanks, Rishi.
-----Original Message----- From: Gili Nachum <gilinac...@gmail.com> To: solr-user <solr-user@lucene.apache.org> Sent: Mon, Apr 27, 2015 4:18 pm Subject: Re: Solr Cloud reclaiming disk space from deleted documents To prevent it from re occurring you could monitor index size and once above a certain size threshold add another machine and split the shard between existing and new machine. On Apr 20, 2015 9:10 PM, "Rishi Easwaran" <rishi.easwa...@aol.com> wrote: > So is there anything that can be done from a tuning perspective, to > recover a shard that is 75%-90% full, other that get rid of the index and > rebuild the data? > Also to prevent this issue from re-occurring, looks like we need make our > system aggressive with segment merges using lower merge factor > > > Thanks, > Rishi. > > > > -----Original Message----- > From: Shawn Heisey <apa...@elyograg.org> > To: solr-user <solr-user@lucene.apache.org> > Sent: Mon, Apr 20, 2015 11:25 am > Subject: Re: Solr Cloud reclaiming disk space from deleted documents > > > On 4/20/2015 8:44 AM, Rishi Easwaran wrote: > > Yeah I noticed that. Looks like > optimize won't work since on some disks we are already pretty full. > > Any > thoughts on increasing/decreasing <mergeFactor>10</mergeFactor> or > ConcurrentMergeScheduler to make solr do merges faster. > > You don't have to do > an optimize to need 2x disk space. Even normal > merging, if it happens just > right, can require the same disk space as a > full optimize. Normal Solr > operation requires that you have enough > space for your index to reach at least > double size on occasion. > > Higher merge factors are better for indexing speed, > because merging > happens less frequently. Lower merge factors are better for > query > speed, at least after the merging finishes, because merging happens > more > frequently and there are fewer total segments at any given moment. > > During a merge, there is so much I/O that query speed is often > negatively > affected. > > Thanks, > Shawn > > > >
