Yes. Indexing activity suspends until the commit finishes, then starts. Having both queries and indexing on the same Solr will have this memory problem.
Lance On Thu, Oct 21, 2010 at 1:16 PM, Jonathan Rochkind <rochk...@jhu.edu> wrote: > If I do _not_ have any auto-commit enabled, and add 500k documents and > commit at end, no problem. > > If I instead set auto-commit maxDocs to 100000 (pretty large number), and > try to add 500k docs, with autocommits theoretically happening every 100k... > I run into an OutOfMemory error. > > Can anyone think of any reasons that would cause this, and how to resolve > it? > All I can think of is that in the first case, my newSearcher and > firstSearcher warming queries don't run until the 'document add' is > completely done. In the second case, there are newSearcher and firstSearcher > warming queries happening at the same time another process is continuing to > stream 'add's to Solr. Although at a maxDocs of 100000, I shouldn't (I > think) get _overlapping_ warming queries, the warming queries should be done > before the next commit. I think. But nonetheless, just the fact that warming > queries are happening at the same time 'add's are continuing to stream, > could that be enough to somehow increase memory usage enough to run into > OOM? > -- Lance Norskog goks...@gmail.com
