Re: IndexWriter flush/commit exception

2013-12-18 Thread Ravikumar Govindarajan
Thanks Mike for a great explanation on Flush IOException I was thinking on the perspective of a HDFSDirectory. In addition to the all causes of IOException during flush you have listed, a HDFSDirectory also has to deal with network issues, which is not lucene's problem at all. But I would

Re: Lucene Hierarchial Taxonomy Search

2013-12-18 Thread Nino_87
Hi, thanks for the answer. This could by a solution. But i have more than one hierarchic filed to query and i want to use the CategoryPath indexed in taxonomy. I'm using the DrillDown query: DrillDownQuery luceneQuery = new DrillDownQuery(searchParams.indexingParams); luceneQuery.add(new

Re: IndexWriter flush/commit exception

2013-12-18 Thread Michael McCandless
On Wed, Dec 18, 2013 at 3:15 AM, Ravikumar Govindarajan ravikumar.govindara...@gmail.com wrote: Thanks Mike for a great explanation on Flush IOException You're welcome! I was thinking on the perspective of a HDFSDirectory. In addition to the all causes of IOException during flush you have

RE: Running Lucene tests on a custom Directory subclass

2013-12-18 Thread Scott Schneider
Never mind... the problem was that I compiled my jar against Lucene 3.3, but tried running against Lucene 4.4. It works when I also run against 3.3. (Or, at least, I get test failures that make sense!) Scott -Original Message- From: Scott Schneider

Index Size in bytes

2013-12-18 Thread Siraj Haider
How can I get the size of the whole index in bytes? regards -Siraj (212) 306-0154 This electronic mail message and any attachments may contain information which is privileged, sensitive and/or otherwise exempt from disclosure under applicable law. The

Re: Index Size in bytes

2013-12-18 Thread Michael McCandless
Use Directory.listAll to get all files, then visit each one and call Directory.fileLength, and sum those up? Note that this gives you total size of all commit points, which may be over-counting on Windows in cases where IndexWriter has removed old commit points but IndexReaders still have the

Debugging unit tests with Eclipse

2013-12-18 Thread Scott Schneider
I'm trying to run and debug the unit tests for Lucene 3.3.0 using Eclipse. I loaded src/java, src/test, and src/test-framework into 3 projects in my workspace and got it all compiling. I created a debug configuration for tests, but I get 54 unit test failures. I can copy the list if anyone

Re: IndexWriter flush/commit exception

2013-12-18 Thread Ravikumar Govindarajan
You could make a custom Dir wrapper that always caches in RAM, but that sounds a bit terrifying :) This was exactly what I implemented:) A commit-thread runs periodically every 30 seconds, while RAM-Monitor thread runs every 5 seconds to commit data in-case sizeInBytes=70%-of-maxCachedBytes.

Phrase indexing and searching

2013-12-18 Thread Manjula Wijewickrema
Dear list, My Lucene programme is able to index single words and search the most matching documents (based on term frequencies) documents from a corpus to the input document. Now I want to index two word phrases and search the matching corpus documents (based on phrase frequencies) to the input