Re: Suggested number of fields limit per Index

2008-01-03 Thread Briggs
Sorry. Anyway, back on track. On Jan 3, 2008 3:25 PM, Chris Hostetter <[EMAIL PROTECTED]> wrote: > > : Ummm, Chris, I don't know why you posted this here. We're all on > : track as far as I can tell. Or is this a trap to say that I have > : changed the subject and am now talking about thread hija

Re: Suggested number of fields limit per Index

2008-01-03 Thread Chris Hostetter
: Ummm, Chris, I don't know why you posted this here. We're all on : track as far as I can tell. Or is this a trap to say that I have : changed the subject and am now talking about thread hijacking? But, I : suppose that would have been you. ;-) This is my standard reply to anyone (i notice) wh

Re: Suggested number of fields limit per Index

2008-01-03 Thread Briggs
Ummm, Chris, I don't know why you posted this here. We're all on track as far as I can tell. Or is this a trap to say that I have changed the subject and am now talking about thread hijacking? But, I suppose that would have been you. ;-) Briggs. On Jan 3, 2008 2:10 PM, Chris Hostetter <[EMAIL

Re: Suggested number of fields limit per Index

2008-01-03 Thread Chris Hostetter
: Subject: Suggested number of fields limit per Index : In-Reply-To: <[EMAIL PROTECTED]> : References: <[EMAIL PROTECTED]> http://people.apache.org/~hossman/#threadhijack Thread Hijacking on Mailing Lists When starting a new discussion on a mailing list, please do not reply to an existing mess

Re: Suggested number of fields limit per Index

2008-01-03 Thread Grant Ingersoll
--Original Message- From: Grant Ingersoll [mailto:[EMAIL PROTECTED] Sent: Thursday, January 03, 2008 1:03 PM To: java-user@lucene.apache.org Subject: Re: Suggested number of fields limit per Index Another issues is how to generate queries. If you have hundreds of fields, you may have

RE: Suggested number of fields limit per Index

2008-01-03 Thread Dai, Chunhe
the line. Thanks again for your help. -Chunhe -Original Message- From: Grant Ingersoll [mailto:[EMAIL PROTECTED] Sent: Thursday, January 03, 2008 1:03 PM To: java-user@lucene.apache.org Subject: Re: Suggested number of fields limit per Index Another issues is how to generate queries

Re: Suggested number of fields limit per Index

2008-01-03 Thread Grant Ingersoll
Another issues is how to generate queries. If you have hundreds of fields, you may have to generate queries (e.g. using the MultfieldQueryParser) across all those fields just to find documents that _could_ have those fields. This can lead to the dreaded TooManyClausesException. That bei

Re: Suggested number of fields limit per Index

2008-01-03 Thread Briggs
I'll give a quick opinion, and remember that is all it is. Without more information of the types of documents your are storing, I would say you are definitely going in the wrong direction. In my opinion, an index should describe the common attributes of all the documents it contains. You should

Re: Suggested number of fields limit per Index

2008-01-03 Thread mark harwood
One thing to watch out for is the "norms" overhead which is one byte per field, per document. These are byte arrays used in scoring to account for the length of fields in individual documents. With hundreds of fields and millions of documents this can eat up memory. The good news is you can opt