On Sun, Sep 19, 2010 at 6:04 AM, Ron Mayer <r...@0ape.com> wrote: > My system which has documents being added pretty much > continually seems pretty well behaved except, it seems, > when large segments get merged. During that time > the system starts really dragging, and queries that took > only a couple seconds are taking dozens. You might wanna look at the MergePolicy since this is the part of Solr / Lucene taking care of the segment merges. There is a recent contribution to the 3.x branch which could be very interesting to you see:
http://svn.apache.org/repos/asf/lucene/dev/branches/branch_3x/lucene/contrib/misc/src/java/org/apache/lucene/index/BalancedSegmentMergePolicy.java you should also look at this post for frequent updates (Near Realtime) http://www.lucidimagination.com/search/document/a17d63d8dcc1cb9d/tuning_solr_caches_with_high_commit_rates_nrt#adf9ee007ce18ba6 Simon > > Some other I/O bound servers seem to have features > that let you throttle how much I/O they take for administrative > background tasks -- for example PostgreSQL's "vacuum_cost_delay" > and related parameters[1], which are described as > > "The intent of this feature is to allow administrators to > reduce the I/O impact of these commands on concurrent > database activity. There are many situations in which it is > not very important that maintenance commands like VACUUM > and ANALYZE finish quickly; however, it is usually very > important that these commands do not significantly > interfere with the ability of the system to perform other > database operations. Cost-based vacuum delay provides > a way for administrators to achieve this." > > Are there any similar features for Solr, where it can sacrifice the > speed of doing a commit in favor of leaving more I/O bandwidth > for users performing searches? > > If not, where in the code might I look to add such a feature? > > Ron > > [1] http://www.postgresql.org/docs/8.4/static/runtime-config-resource.html > > > >
