Doug Cutting wrote:
Andrzej Bialecki wrote:
[EMAIL PROTECTED] wrote:
Interesting to know. However I never had this good luck, I got everytime a unexpected EOF Exception.
Yeah, that's the symptom of missing index.
I thought i'd fixed this some time ago. One still might get an when iterating through entries from a truncated segment, but no longer when opening it. So it should always be possible to read all the entries that were flushed: an index file should always be present, and EOF on the index file should be trapped, generating only a warning.
I would like to revisit this change... I'm making some tests with ~10mln docs in several segments, and when the MapFile index file is truncated the seeking performance drops drastically - actually, it takes less time if I just delete the incomplete index file, and then re-create it using SegmentReader -fix option...
I propose to change it back to throw an Exception, or to add this choice as an option in the constructor. It's also possible to add an "autofix" option in the constructor. In any case, I think the fact that the index file is truncated should be noted (e.g. isIndexComplete()), so that the applications which depend on fast seeking could make informed decisions whether to fix it or to live with it...
-- Best regards, Andrzej Bialecki
------------------------------------------------- Software Architect, System Integration Specialist CEN/ISSS EC Workshop, ECIMF project chair EU FP6 E-Commerce Expert/Evaluator ------------------------------------------------- FreeBSD developer (http://www.freebsd.org)
------------------------------------------------------- This SF.Net email is sponsored by: InterSystems CACHE FREE OODBMS DOWNLOAD - A multidimensional database that combines robust object and relational technologies, making it a perfect match for Java, C++,COM, XML, ODBC and JDBC. www.intersystems.com/match8 _______________________________________________ Nutch-developers mailing list [EMAIL PROTECTED] https://lists.sourceforge.net/lists/listinfo/nutch-developers
