As far as I know you're on the right track, adds are single threaded. You can have multiple threads making indexing requests from your client, but that's primarily aimed at making the I/O not be the bottleneck, at some point the actual indexing of the documents is single-threaded.
It'd be tricky, very tricky to have multiple threads writing to an index at the same time, much less multiple CPUs..... If you're desperate to index quickly, you can index into several cores, even on separate machines and merge the results. Best Erick On Tue, May 31, 2011 at 4:13 PM, Jack Repenning <jrepenn...@collab.net> wrote: > On May 31, 2011, at 12:44 PM, Markus Jelsma wrote: > >> I haven't given it a try but perhaps opening multiple HTTP connections to the >> update handler will end up in multiple threads thus better CPU utilization. > > My original test case had hundreds of HTTP connections (all to the same URL) > doing adds, but seemed to use only one CPU core for adding, or to serialize > the adds somehow, something like that ... at any rate, I couldn't drive CPU > use above ~120% with that configuration. > > This is quite different from queries. For queries (or a rich query-to-add > mix), I can easily drive CPU use into multiple-hundreds of % CPU, with just a > few dozen concurrent query connections (running flat out). But adds resist > that trick. I don't know whether this means that adds really are using a > single thread, or if they're using multiple threads but synchronizing on some > monitor. Actually, I can't say I care much: bottom line seems to be I only > use one CPU core (plus a negligible marginal bit) for adds. > > Since I've confirmed that queries spread neatly, I can live with the > single-thready adds. In production, it seems likely that I'll be more or less > continuously spending one CPU core on adds, and the rest on queries. > > -==- > Jack Repenning > Technologist > Codesion Business Unit > CollabNet, Inc. > 8000 Marina Boulevard, Suite 600 > Brisbane, California 94005 > office: +1 650.228.2562 > twitter: http://twitter.com/jrep > > > > > > > > > >
