The performance results in my previous posting were based on an implementation that performs 2 searches, one for getting 'Hits' and another for getting the BitSet. I reimplemented this in one search using the code in 'SolrIndexSearcher.getDocListAndSetNC' and I'm now getting throughput of 350-375 qps.
This is great stuff Solr guys! I'd love to see the DocSet and DocList features added to Lucene's IndexSearcher. Peter On 6/12/06, Peter Keegan <[EMAIL PROTECTED]> wrote:
I'm seeing query throughput of approx. 290 qps with OpenBitSet vs. 270 with BitSet. I had to reduce the max. HashDocSet size to 2K - 3K (from 10K-20K) to get optimal tradeoff. no. docs in index: 730,000 average no. results returned: 40 average response time: 50 msec (15-20 for counting facets) no. facets: 100 on every query I'm not using the Solr server as we have already developed an infrastructure. Peter On 6/10/06, Yonik Seeley <[EMAIL PROTECTED]> wrote: > > On 6/9/06, Peter Keegan <[EMAIL PROTECTED]> wrote: > > However, my throughput testing shows that the Solr method is at least > 50% > > faster than mine. I'm seeing a big win with the use of the HashDocSet > for > > lower hit counts. On my 64-bit platform, a MAX_SIZE value of 10K-20K > seems > > to provide optimal performance. > > Interesting... how many documents are in your collection? > It would prob be nice to make the HashDocSet cutt-off dynamic rather > than fixed. > Are you using Solr, or just some of it's code? > > > I'm looking forward to trying this with > > OpenBitSet. > > I checked in the OpenBitSet changes today. I imagine this will lower > the optimal max HashDocSet size for performance a little. You might > not see much performance improvement if most of the intersections > involved a HashDocSet... the OpenBitSet improvements only kick in with > bitset<->bitset intersection counts. > > -Yonik > http://incubator.apache.org/solr Solr, the open-source Lucene search > server > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >
