Re: How to add ASCIIFoldingFilter in ClassicAnalyzer

2016-10-20 Thread Kumaran Ramasubramanian
Hi Adrien Thanks a lot for the pointer. -- Kumaran R On Wed, Oct 19, 2016 at 8:07 PM, Adrien Grand wrote: > You would need to override the wrapComponents method in order to wrap the > tokenstream. See for instance Lucene's LimitTokenCountAnalyzer. > > Le mar. 18 oct. 2016

RE: Can ByteBufferIndexInput use buffering?

2016-10-20 Thread Uwe Schindler
Hi, adding buffering to ByteBufferIndexInput would not only be an anti-pattern, it would also slowdown. What is the sense of coping data from memory location A to memory location B before reading? I'd suggest to read this and understand what virtual memory and ByteBufferIndexInput does before

Re: Can ByteBufferIndexInput use buffering?

2016-10-20 Thread Michael McCandless
The fact that MMapIndexInput does no buffering is an important performance gain vs NIOFSDirectory which e.g. on seeking to a term loads way too many bytes. Why do you want to add buffering to it? The OS should already do a good job keeping recently accessed pages hot, doing the buffering for

Re: ReaderManager, more drama with things not being closed before closing the Directory

2016-10-20 Thread Michael McCandless
Maybe you can contribute the code you have for managing multiple indices and we can iterate/debug from there? Somehow we need to expose this failure in a standalone test case so we can isolate it. Mike McCandless http://blog.mikemccandless.com On Thu, Oct 20, 2016 at 1:57 AM, Trejkaz

Can ByteBufferIndexInput use buffering?

2016-10-20 Thread Ravikumar Govindarajan
When we use NIOFSDirectory, lucene internally uses buffering via BufferedIndexInput (1KB etc...) while reading from the file.. However, for MmapDirectory (ByteBufferIndexInput) there is no such buffering & data is read from the mapped bytes directly... Will it be too much of a performance drag