Hi Adrien
Thanks a lot for the pointer.
--
Kumaran R
On Wed, Oct 19, 2016 at 8:07 PM, Adrien Grand wrote:
> You would need to override the wrapComponents method in order to wrap the
> tokenstream. See for instance Lucene's LimitTokenCountAnalyzer.
>
> Le mar. 18 oct. 2016
Hi,
adding buffering to ByteBufferIndexInput would not only be an anti-pattern, it
would also slowdown. What is the sense of coping data from memory location A to
memory location B before reading?
I'd suggest to read this and understand what virtual memory and
ByteBufferIndexInput does before
The fact that MMapIndexInput does no buffering is an important
performance gain vs NIOFSDirectory which e.g. on seeking to a term
loads way too many bytes.
Why do you want to add buffering to it?
The OS should already do a good job keeping recently accessed pages
hot, doing the buffering for
Maybe you can contribute the code you have for managing multiple
indices and we can iterate/debug from there?
Somehow we need to expose this failure in a standalone test case so we
can isolate it.
Mike McCandless
http://blog.mikemccandless.com
On Thu, Oct 20, 2016 at 1:57 AM, Trejkaz
When we use NIOFSDirectory, lucene internally uses buffering via
BufferedIndexInput (1KB etc...) while reading from the file..
However, for MmapDirectory (ByteBufferIndexInput) there is no such
buffering & data is read from the mapped bytes directly...
Will it be too much of a performance drag