Re: Performance issues with the default field compression

2014-04-10 Thread Alex Parvulescu
Hi Adrien, Thanks for clarifying! We're going to go the custom codec custom visitor route. best, alex On Wed, Apr 9, 2014 at 10:38 PM, Adrien Grand jpou...@gmail.com wrote: Hi Alex, Indeed, one or several (the number depends on the size of your documents) documents need to be fully

Performance issues with the default field compression

2014-04-09 Thread Alex Parvulescu
Hi, I was investigating some performance issues and during profiling I noticed that there is a significant amount of time being spent decompressing fields which are unrelated to the actual field I'm trying to load from the lucene documents. In our benchmark doing mostly a simple full-test search,

Re: Performance issues with the default field compression

2014-04-09 Thread Adrien Grand
Hi Alex, Indeed, one or several (the number depends on the size of your documents) documents need to be fully decompressed in order to read a single field of a single document. Regarding the stored fields visitor, the default one doesn't return STOP when the field has been found because other