Hi,
we found serious problem while searching. In some special situation (probably
depends on the index size)
repeating search does not return correct results or event ends with
CLuceneError "read past EOF".
To achieve the error, the following sequence must be called:
1) run a query
2) delete an instance of Analyzer (can be instantiated earlier but in the same
thread) - this causes that the ThreadLocals objects are freed
3) run the same query - THIS FAILS!!
In all the cases where the seach failed we switched off the compound files.
Could some one help us with this issue? It seems that reading the index files
is not working correctly.
Jiri
PS: The following code is a test that demostrated the problem:
/**
* Create index
*/
Directory* prepareDirectory1()
{
const TCHAR * tszDocText = _T( "a b c d e f g h i j k l m n o p q r s t u
v w x y z ab bb cb db eb fb gb hb ib jb kb lb mb nb ob pb qb rb sb tb ub vb wb
xb yb zb ac bc cc dc ec fc gc hc ic jc kc lc mc nc oc pc qc rc sc tc uc vc wc
xc yc zc ad bd cd dd ed fd gd hd id jd kd ld md nd od pd qd rd sd td ud vd wd
xd yd zd ae be ce de ee fe ge he ie je ke le me ne oe pe qe re se te ue ve we
xe ye ze af bf cf df ef ff gf hf if jf kf lf mf" );
char fsdir[CL_MAX_PATH];
_snprintf(fsdir,CL_MAX_PATH,"%s/%s",cl_tempDir, "test.search");
WhitespaceAnalyzer analyzer;
Directory* pDirectory =
(Directory*)FSDirectory::getDirectory(fsdir);
IndexWriter writer( pDirectory, &analyzer, true );
writer.setUseCompoundFile( false );
Document* d = _CLNEW Document();
d->add( *_CLNEW Field( _T("_content"), tszDocText, Field::STORE_NO |
Field::INDEX_TOKENIZED ));
writer.addDocument(d);
_CLDELETE( d );
writer.close();
return pDirectory;
}
/**
* Run test
*/
void testReadPastEOF(CuTest *tc)
{
Directory* pDirectory = prepareDirectory1();
Analyzer * pAnalyzer = NULL;
Hits * pHits = NULL;
IndexReader* pReader = IndexReader::open( pDirectory );
IndexSearcher searcher( pReader );
CLUCENE_ASSERT( pReader->numDocs() == 1 );
Term * t1 = new Term( _T( "_content" ), _T( "ze" ) );
TermQuery * pQry1 = new TermQuery( t1 );
_CLDECDELETE( t1 );
pAnalyzer = new SimpleAnalyzer();
pHits = searcher.search( pQry1 );
_ASSERT( pHits->length() == 1 );
CLUCENE_ASSERT( pHits->length() == 1 );
_CLDELETE( pHits );
// Removing the analyzer causes removing of ThreadLocals - also cached
SegmentTermEnum
_CLDELETE( pAnalyzer );
// THE NEXT CALL WILL FAIL
pHits = searcher.search( pQry1 );
_ASSERT( pHits->length() == 1 );
CLUCENE_ASSERT( pHits->length() == 1 );
_CLDELETE( pHits );
_CLDELETE( pQry1 );
searcher.close();
_CLDELETE( pReader );
pDirectory->close();
_CLDECDELETE( pDirectory );
}
------------------------------------------------------------------------------
Lotusphere 2011
Register now for Lotusphere 2011 and learn how
to connect the dots, take your collaborative environment
to the next level, and enter the era of Social Business.
http://p.sf.net/sfu/lotusphere-d2d
_______________________________________________
CLucene-developers mailing list
CLucene-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/clucene-developers
