Re: solr performance for documents with hundreds of fields

2008-04-25 Thread Umar Shah
I am just wondering, because having 200 fields seems like too much (for me), I want to know if people actually have such kind of schemas and how well they perform. On Thu, Apr 24, 2008 at 5:10 PM, Grant Ingersoll [EMAIL PROTECTED] wrote: Are you actually seeing performance problems or just

Re: solr performance for documents with hundreds of fields

2008-04-25 Thread Erik Hatcher
That is well within the boundaries of what Solr/Lucene can handle. But, of course, it depends on what you're doing with those fields too. Putting 200 fields into a dismax qf specification, for example, would surely be bad for performance :) But querying on only a handful of fields or

Re: solr performance for documents with hundreds of fields

2008-04-25 Thread Otis Gospodnetic
What Erik said ;) 200 fields is not a problem. Things to watch out for are: - more index file and thus more open file descriptors if you use non-compound Lucene index format and are working with non-optimized indices (on master - optimize your index before it gets to slaves) - slower merging

solr performance for documents with hundreds of fields

2008-04-24 Thread Umar Shah
Hi, I wanted to know what would be the performance of SOLR for the following scenario: the documents contain say 200 fields with say 100 of the fields (containing numbers) and rest containing short strings of 40-50 character length. the sparseness of the data can be assumed to be as

Re: solr performance for documents with hundreds of fields

2008-04-24 Thread Grant Ingersoll
Are you actually seeing performance problems or just wondering if there will be a performance problem? -Grant On Apr 24, 2008, at 7:08 AM, Umar Shah wrote: Hi, I wanted to know what would be the performance of SOLR for the following scenario: the documents contain say 200 fields with