Some time ago I posted the results in my peculiar app of using FieldSelector, and it gave dramatic improvements in my case (a factor of about 10). I suspect much of that was peculiar to my index design, so your mileage may vary.
See a thread titled... *Lucene 2.1, using FieldSelector speeds up my app by a factor of 10+....* Best Erick On 5/17/07, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
I haven't tried compression either. I know there was some talk a while ago about deprecating, but that hasn't happened. The current implementation yields the highest level of compression. You might find better results by compressing in your application and storing as a binary field, thus giving you more control over CPU used. This is our current recommendation for dealing w/ compression. If you are not actually displaying that field, you should look into the FieldSelector API (via IndexReader). It allows you to lazily load fields or skip them all together and can yield a pretty significant savings when it comes to loading documents. FieldSelector is available in 2.1. -Grant On May 17, 2007, at 4:01 AM, Paul Elschot wrote: > On Thursday 17 May 2007 08:10, Andreas Guther wrote: >> I am currently exploring how to solve performance problems I >> encounter with >> Lucene document reads. >> >> We have amongst other fields one field (default) storing all >> searchable >> fields. This field can become of considerable size since we are >> indexing >> documents and store the content for display within results. >> >> I noticed that the read can be very expensive. I wonder now if it >> would >> make sense to add this field as Field.Store.Compress to the >> index. Can >> someone tell me if this would speed up the document read or if >> this is >> something only interesting for saving space. > > I have not tried the compression yet, but in my experience a good way > to reduce the costs of document reads from a disk is by reading them > in document number order whenever possible. In this way one saves > on the disk head seeks. > Compression should actually help reducing the costs of disk head seeks > even more. > > Regards, > Paul Elschot > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > -------------------------- Grant Ingersoll Center for Natural Language Processing http://www.cnlp.org/tech/lucene.asp Read the Lucene Java FAQ at http://wiki.apache.org/jakarta-lucene/ LuceneFAQ --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
