Re: Failure when trying to full sync, out of space ? Doesn't delete old segments before full sync?

On 11/28/2016 11:06 AM, Walter Underwood wrote: > Worst case: > 1. Disable merging. > 2. Delete all the documents. > 3. Add all the documents. > 4. Enable merging. > > After step 3, you have two copies of everything, one deleted copy and one new > copy. > The merge makes a third copy. Just

Re: Failure when trying to full sync, out of space ? Doesn't delete old segments before full sync?

> On Nov 28, 2016, at 9:38 AM, Shawn Heisey wrote: > > […] Typically > a merge or optimize will only require double the space, but there are > certain worst-case scenarios where it can require triple. I do not know > what causes the worst-case situation. Worst case: 1.

Re: Failure when trying to full sync, out of space ? Doesn't delete old segments before full sync?

On 11/28/2016 12:26 PM, Erick Erickson wrote: Well, such checks could be put in, but they don't get past the basic problem. And all this masks your real problem; you didn't have enough disk space to optimize in the first place. Even during regular indexing w/o optimizing, Lucene segment

Re: Failure when trying to full sync, out of space ? Doesn't delete old segments before full sync?

On 11/28/2016 9:39 AM, Michael Joyner wrote: > I'm running out of spacing when trying to restart nodes to get a > cluster back up fully operational where a node ran out of space during > an optimize. > > It appears to be trying to do a full sync from another node, but > doesn't take care to check

Re: Failure when trying to full sync, out of space ? Doesn't delete old segments before full sync?

We've being trying to run at 40% estimated usage when optimized, but are doing a large amount of index updates ... 40% usage in this scenario seems to be too high... On 11/28/2016 12:26 PM, Erick Erickson wrote: Well, such checks could be put in, but they don't get past the basic problem.

Re: Failure when trying to full sync, out of space ? Doesn't delete old segments before full sync?

Well, such checks could be put in, but they don't get past the basic problem. bq: If the segments are out of date and we are pulling from another node before coming "online" why aren't the old segments deleted? because you run the risk of losing _all_ your data and having nothing at all. The

Failure when trying to full sync, out of space ? Doesn't delete old segments before full sync?

Hello all, I'm running out of spacing when trying to restart nodes to get a cluster back up fully operational where a node ran out of space during an optimize. It appears to be trying to do a full sync from another node, but doesn't take care to check available space before starting