On 10/19/07, Maximilian Hütter <[EMAIL PROTECTED]> wrote: > Yonik Seeley schrieb: > > On 10/17/07, Maximilian Hütter <[EMAIL PROTECTED]> wrote: > >> I also found this: > >> > >> "Controls the maximum number of terms that can be added to a Field for a > >> given Document, thereby truncating the document. Increase this number if > >> large documents are expected. However, setting this value too high may > >> result in out-of-memory errors." > >> > >> Coming from: http://www.ibm.com/developerworks/library/j-solr2/index.html > >> > >> That might be a problem for me. > >> > >> I was thinking about using copyFields, instead of one large fulltext > >> field. Would that solve my problem, or would the maxFieldLength still > >> apply when using copyFields? > > > > maxFieldLength is a setting on the IndexWriter and applies to all fields. > > If you want more tokens indexed, simply increase the value of > > maxFieldLength to something like 2000000000 and you should be fine. > > > > There's no penalty for setting it higher than the largest field you > > are indexing (no diff between 1M and 2B if all your docs have field > > lengths less than 1M tokens anyway). > > > > -Yonik > > > Yes, that would be an easy solution, as there is no performance penalty > as say. > I am still unsure, if the maxFieldLength applies to copyFields?
maxFieldLength applies to all fields (it's a Lucene concept, not a Solr one). copyField and maxFieldLength are not related. > When using copyFields I get an array back for that field (I copied to). > So it seems to be different. ??? maxFieldLength only applies to the number of tokens indexed. You will always get the complete field back if it's stored, regardless of what maxFieldLength is. > Is there a performance penalty for using copyFields when indexing? copyFields are done as a discrete step before indexing... almost no cost to do that. Indexing itself will have a performance impact if there are more fields to index + store as a result of the copyField commands. > How > about the mixed fieldtypes in the source fields? What happens when I > copy an sint based field and a string based field to a string based field? copyField is done based on the string values, before any analysis. Mixed content should be fine. -Yonik
