Hi, as discussed before, your use case is special and not the natural way of querying Lucene (you should really only get top-ranking hits from store). But Lucene is flexible enough since Lucene 4: Just use another codec to encode stored fields. You need to reindex, but when creating IndexWriter use a FilterCodec to disable compression and use the "old" Lucene 4.0 stored fields which have no LZ4 compression. Don't forget to give your own codec a good name and List it in the META-INF/services, so Lucene can load it when reading indexes!
Uwe ----- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -----Original Message----- > From: Chris Zhang [mailto:zhangjcm...@gmail.com] > Sent: Sunday, July 07, 2013 6:26 PM > To: java-user@lucene.apache.org > Subject: Re: Please Help solve problem of bad read performance in lucene > 4.2.1 > > thianks Adrien, > In my project, almost all hit docs are supposed to be fetched for every query, > what's why I am upset by the poor reading performance. Maybe I should > store field values which are expected to be stored in high performance > storage engine. > In the above test case, time consuming of reading all docs in lucene 3.0 is > about 78 sec, that reading speed is approximately 10MB/s , but 700+ sec in > lucene 4.2.1, which indicates reading speed is less than 1MB/s. So I think > committer of lucene should pay attention to this. > > > On Sun, Jul 7, 2013 at 10:23 PM, Adrien Grand <jpou...@gmail.com> wrote: > > > Indeed, Lucene 4.1+ may be a bit slower for indices that comptelely > > fit in your file-system cache. On the other hand, you should see > > better performance with indices which are larger than the amount of > > physical memory of your machine. Your reading benchmark only measures > > IndexReader.get(int) which should only be used to display summary > > results (that is, only called 10 or 20 times per displayed page). Most > > of time, the bottleneck is rather searching which can be made more > > efficient on small indices by switching to an in-memory postings > > format. > > > > -- > > Adrien > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
