Without even looking at the different segment files, things look odd: You say that you optimize every day, yet I see segments up to 4 days old. Also look at all the segments_??? files... each represents a commit point of the index. So it looks like you have 16 snapshots (or commit points) of the index. Do you have a deletion policy configured to do this for some reason?
Anyway, this is why when you changed how you index, you didn't see much of a size increase (comparatively). -Yonik http://lucidimagination.com On Wed, Mar 16, 2011 at 7:46 PM, Robert Petersen <rober...@buy.com> wrote: > Thanks for the reply Yonik, Here are the results of Ls -l on the master > server index folder, also please note we have hundreds of those small > sparsely populated fields and I run optimize once a day at midnight. We > index 24/7 off a queue at a clip of about 200K docs per hour so the index has > had hundreds of commits since last night at midnight. [...]
