Re: dataset parameters suitable for lucene application

2007-10-02 Thread Chris Harris
ns of mill seconds. > > > > -Original Message- > From: Charlie Jackson [mailto:[EMAIL PROTECTED] > Sent: Wednesday, September 26, 2007 8:53 AM > To: solr-user@lucene.apache.org > Subject: RE: dataset parameters suitable for lucene application > > My experiences so far with this lev

Re: dataset parameters suitable for lucene application

2007-09-26 Thread Mike Klaas
On 26-Sep-07, at 10:50 AM, Law, John wrote: Thanks all! One last question... If I had a collection of 2.5 billion docs and a demand averaging 200 queries per second, what's the confidence that Solr/Lucene could handle this volume and execute search with sub-second response times? No search

RE: dataset parameters suitable for lucene application

2007-09-26 Thread Lance Norskog
; Sent: Wednesday, September 26, 2007 1:32 PM > To: solr-user@lucene.apache.org > Subject: RE: dataset parameters suitable for lucene application > > Sorry, I meant that it maxed out in the sense that my maxDoc field on > the stats page was 8.8 million, which indicates that the most do

Re: dataset parameters suitable for lucene application

2007-09-26 Thread Walter Underwood
ember 26, 2007 1:32 PM > To: solr-user@lucene.apache.org > Subject: RE: dataset parameters suitable for lucene application > > Sorry, I meant that it maxed out in the sense that my maxDoc field on > the stats page was 8.8 million, which indicates that the most docs it > has ever

RE: dataset parameters suitable for lucene application

2007-09-26 Thread Law, John
ailto:[EMAIL PROTECTED] Sent: Wednesday, September 26, 2007 1:32 PM To: solr-user@lucene.apache.org Subject: RE: dataset parameters suitable for lucene application Sorry, I meant that it maxed out in the sense that my maxDoc field on the stats page was 8.8 million, which indicates that the most do

RE: dataset parameters suitable for lucene application

2007-09-26 Thread Charlie Jackson
Solr can handle. -Original Message- From: Chris Harris [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 26, 2007 11:49 AM To: solr-user@lucene.apache.org Subject: Re: dataset parameters suitable for lucene application By "maxed out" do you mean that Solr's performance became un

RE: dataset parameters suitable for lucene application

2007-09-26 Thread Xuesong Luo
PROTECTED] Sent: Wednesday, September 26, 2007 8:53 AM To: solr-user@lucene.apache.org Subject: RE: dataset parameters suitable for lucene application My experiences so far with this level of data have been good. Number of records: Maxed out at 8.8 million Database size: friggin huge (100+ GB) Index

Re: dataset parameters suitable for lucene application

2007-09-26 Thread Chris Harris
By "maxed out" do you mean that Solr's performance became unacceptable beyond 8.8M records, or that you only had 8.8M records to index? If the former, can you share the particular symptoms? On 9/26/07, Charlie Jackson <[EMAIL PROTECTED]> wrote: > My experiences so far with this level of data have

RE: dataset parameters suitable for lucene application

2007-09-26 Thread Charlie Jackson
My experiences so far with this level of data have been good. Number of records: Maxed out at 8.8 million Database size: friggin huge (100+ GB) Index size: ~24 GB 1) It took me about a day to index 8 million docs using a non-optimized program I wrote. It's non-optimized in the sense that it's not

Re: dataset parameters suitable for lucene application

2007-09-26 Thread Walter Underwood
That seems well within Solr's capabilities, though you should come up with a desired queries/sec figure. Solr's query rate varies widely with the configuration -- how many fields, fuzzy search, highlighting, facets, etc. Essentially, Solr uses Lucene, a modern search core. It has performance and