One thing to watch out for is the "norms" overhead which is one byte per field,
per document. These are byte arrays used in scoring to account for the length
of fields in individual documents.
With hundreds of fields and millions of documents this can eat up memory.
The good news is you can opt to omit norms for a choice of field or if you have
specify indexed field as untokenized.
Cheers,
Mark
----- Original Message ----
From: "Dai, Chunhe" <[EMAIL PROTECTED]>
To: java-user@lucene.apache.org
Sent: Thursday, 3 January, 2008 4:45:05 PM
Subject: Suggested number of fields limit per Index
I have been searching online could not find an exact answer; and
wondering if anyone here knows whether there is a preferred max number
of fields limit in lucene index?
We are in the process of deciding how our index would look like in our
lucene integration. For one of our approach, we could have a large
number of fields in the index - say maybe several hundred. But, each
Document in the index do not contain every of those fields and would
only have a few fields within those hundreds of fields (Probably in
tens). Does anyone ever have experience with set up like this? I am
wondering whether there is a potential performance issue with indexing
and searching.
Thanks.
Chunhe
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
___________________________________________________________
Support the World Aids Awareness campaign this month with Yahoo! For Good
http://uk.promotions.yahoo.com/forgood/
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
