Re: dataset parameters suitable for lucene application

2007-10-02 Thread Chris Harris
- From: Charlie Jackson [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 26, 2007 8:53 AM To: solr-user@lucene.apache.org Subject: RE: dataset parameters suitable for lucene application My experiences so far with this level of data have been good. Number of records: Maxed out at 8.8

Re: dataset parameters suitable for lucene application

2007-09-26 Thread Walter Underwood
That seems well within Solr's capabilities, though you should come up with a desired queries/sec figure. Solr's query rate varies widely with the configuration -- how many fields, fuzzy search, highlighting, facets, etc. Essentially, Solr uses Lucene, a modern search core. It has performance and

RE: dataset parameters suitable for lucene application

2007-09-26 Thread Charlie Jackson
My experiences so far with this level of data have been good. Number of records: Maxed out at 8.8 million Database size: friggin huge (100+ GB) Index size: ~24 GB 1) It took me about a day to index 8 million docs using a non-optimized program I wrote. It's non-optimized in the sense that it's

Re: dataset parameters suitable for lucene application

2007-09-26 Thread Chris Harris
By maxed out do you mean that Solr's performance became unacceptable beyond 8.8M records, or that you only had 8.8M records to index? If the former, can you share the particular symptoms? On 9/26/07, Charlie Jackson [EMAIL PROTECTED] wrote: My experiences so far with this level of data have been

RE: dataset parameters suitable for lucene application

2007-09-26 Thread Charlie Jackson
. -Original Message- From: Chris Harris [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 26, 2007 11:49 AM To: solr-user@lucene.apache.org Subject: Re: dataset parameters suitable for lucene application By maxed out do you mean that Solr's performance became unacceptable beyond 8.8M records

RE: dataset parameters suitable for lucene application

2007-09-26 Thread Law, John
:[EMAIL PROTECTED] Sent: Wednesday, September 26, 2007 1:32 PM To: solr-user@lucene.apache.org Subject: RE: dataset parameters suitable for lucene application Sorry, I meant that it maxed out in the sense that my maxDoc field on the stats page was 8.8 million, which indicates that the most docs

Re: dataset parameters suitable for lucene application

2007-09-26 Thread Walter Underwood
of a maximum number of docs Solr can handle. -Original Message- From: Chris Harris [mailto:[EMAIL PROTECTED] Sent: Wednesday, September 26, 2007 11:49 AM To: solr-user@lucene.apache.org Subject: Re: dataset parameters suitable for lucene application By maxed out do you mean

RE: dataset parameters suitable for lucene application

2007-09-26 Thread Lance Norskog
Subject: RE: dataset parameters suitable for lucene application Sorry, I meant that it maxed out in the sense that my maxDoc field on the stats page was 8.8 million, which indicates that the most docs it has ever had was around 8.8 million. It's down to about 7.8 million currently. I have

Re: dataset parameters suitable for lucene application

2007-09-26 Thread Mike Klaas
On 26-Sep-07, at 10:50 AM, Law, John wrote: Thanks all! One last question... If I had a collection of 2.5 billion docs and a demand averaging 200 queries per second, what's the confidence that Solr/Lucene could handle this volume and execute search with sub-second response times? No