Hi Dan, You are looking to throttle the merging? I'd recommend setting ConcurrentMergeScheduler.setMaxThreadCount(1). This way IW.addDocument doesn't wait while a merge occurs (like SerialMergeScheduler) however it should not use as much CPU as only one merge will occur at a time.
In regards to overriding the MS.merge method either way you mentioned would work. -J On Fri, May 15, 2009 at 1:41 PM, Dan OConnor <docon...@acquiremedia.com>wrote: > All: > > I would like to be able to control when an index merge happens (by wall > clock time) so that merges do not occur in the middle of the business day. > > I have a lucene system based on v2.3.2 and we add a couple hundred thousand > documents per day - and we allow searching while documents are being added - > we reopen an IndexReader periodically to expose newly arrived contents. > > There are times when merging causes significant performance impacts on > search results - I've seen cases where merging will cause 200% load on a > system (dual quad core x86_64 running Centos) with a raid-5 disk subsystem > of 15k drives. > > I've seen some info on the MergeScheduler and but not necessarily enough to > attempt a coding effort. > > Looking through the code for ConcurrentMergeScheduler.java, is it as > straightforward as over-riding the mergeScheduler.merge() method with a > method that checks to see if a merge is allowed (by wall clock time)? If a > merge is not allowed at that time, can I just return();? Or do I have to > sleep the thread until the merge is allowed? > > Thanks, > Dan > > > Dan O'Connor > SVP, Engineering > Acquire Media<http://www.acquiremedia.com/> > 77 South Bedford Street, Suite 350< > http://maps.google.com/maps?f=q&hl=en&geocode=&q=77+S+Bedford+St,+Burlington,+MA+01803&sll=37.0625,-95.677068&sspn=32.472848,80.859375&ie=UTF8&ll=42.485517,-71.197935&spn=0.002287,0.005193&t=h&z=18 > > > Burlington, MA 01803< > http://maps.google.com/maps?f=q&hl=en&geocode=&q=77+S+Bedford+St,+Burlington,+MA+01803&sll=37.0625,-95.677068&sspn=32.472848,80.859375&ie=UTF8&ll=42.485517,-71.197935&spn=0.002287,0.005193&t=h&z=18 > > > e: docon...@acquiremedia.com<mailto:docon...@acquiremedia.com> > o: 781-250-0565 > f: 877-861-7724 > >
