I build and executed cl_test and cl_demo again on master and the atomicthreads branch with default cmake settings, see attached files. I included a stack trace for the cl_demo app in both cases.
Klemens 2010/6/13 Itamar Syn-Hershko <ita...@divrei-tora.com>: > Can you please test the master branch (cl_test and cl_demo) with default > cmake settings as well? > > Also, can you send the stacktrace for this deadlock? If you get this on > master, then for master, otherwise for atomicthreads. > > Itamar. > >> -----Original Message----- >> From: Klemens Friedl [mailto:fri...@gmail.com] >> Sent: Sunday, June 13, 2010 11:16 AM >> To: clucene-developers@lists.sourceforge.net >> Subject: Re: [CLucene-dev] clucene - cl_demo stops with error >> >> I tried the cl_test and cl_demo with the atomicthreads branch >> and default cmake settings (except added zlib path vars). >> (see attached log files) >> >> cl_test runs through 102 tests, but fails on first of two UTF8 tests. >> cl_demo indexes all files of the reuters corpa, though it >> deadlocks right after that :/ >> >> >> Kind regards, >> Klemens Friedl >> >> >> >> F:\Home\Search\clucene\atomicthreads\build\bin\Debug>cl_test.exe >> Key: .= pass N=not implemented F=fail >> All CLucene Tests: >> CLucene Atomic Updates Test: .. - 6203ms >> CLucene IndexReader Test: .. - 766ms >> CLucene Reuters Test: ... - 8547ms >> CLucene Analysis Test: . - 0ms >> CLucene Analyzers Test: ......... - 234ms >> CLucene Document Test: ...... - 4563ms >> CLucene Number Tools Test: ... - 422ms >> CLucene Debug Test: . - 0ms >> CLucene IndexWriter Test: ...... - 4281ms >> CLucene IndexModifier Test: . - 56047ms >> CLucene High Frequencies Test: . - 16ms >> CLucene Priority Queue Test: . - 62ms >> CLucene DateTools Test: .. - 0ms >> CLucene Query Parser Test: ............... - 63ms >> CLucene Multi-Field QP Test: .. - 0ms >> CLucene Boolean Tests: .... - 15ms >> CLucene Search Test: .............. - 609ms >> CLucene Queries Test: .. - 16ms >> CLucene Term Vector Test: ..... - 78ms >> CLucene Sort Test: ........... - 79ms >> CLucene Duplicates Test: .. - 125ms >> CLucene DateFilter Test: ... - 78ms >> CLucene Wildcard Test: .. - 0ms >> CLucene Store Test: .. - 297ms >> CLucene UTF8 Test: F. - 187ms >> >> 102 tests run: 101 passed, 1 failed, 0 not implemented. >> >> Tests run in 82843ms >> >> WARNING: stringPool still contains intern'd strings (refcounts): >> contents (10) >> field1 (5) >> field2 (5) >> field3 (5) >> field4 (5) >> id (4) >> >> >> >> >> >> >> F:\Home\Search\clucene\atomicthreads\build\bin\Debug>cl_demo.exe >> adding file 1: >> ..\src\test\data\reuters-21578/all-exchanges-strings.lc.txt >> adding file 2: ..\src\test\data\reuters-21578/all-orgs-strings.lc.txt >> adding file 3: >> ..\src\test\data\reuters-21578/all-people-strings.lc.txt >> adding file 4: >> ..\src\test\data\reuters-21578/all-places-strings.lc.txt >> adding file 5: >> ..\src\test\data\reuters-21578/all-topics-strings.lc.txt >> adding file 6: >> ..\src\test\data\reuters-21578/cat-descriptions_120396.txt >> adding file 7: >> ..\src\test\data\reuters-21578/feldman-cia-worldfactbook-data.txt >> >> adding file 8: ..\src\test\data\reuters-21578/LEWIS.DTD >> adding file 9: ..\src\test\data\reuters-21578/README.TXT >> adding file 10: ..\src\test\data\reuters-21578/reut2-000.sgm >> adding file 11: ..\src\test\data\reuters-21578/reut2-001.sgm >> adding file 12: ..\src\test\data\reuters-21578/reut2-002.sgm >> adding file 13: ..\src\test\data\reuters-21578/reut2-003.sgm >> adding file 14: ..\src\test\data\reuters-21578/reut2-004.sgm >> adding file 15: ..\src\test\data\reuters-21578/reut2-005.sgm >> adding file 16: ..\src\test\data\reuters-21578/reut2-006.sgm >> adding file 17: ..\src\test\data\reuters-21578/reut2-007.sgm >> adding file 18: ..\src\test\data\reuters-21578/reut2-008.sgm >> adding file 19: ..\src\test\data\reuters-21578/reut2-009.sgm >> adding file 20: ..\src\test\data\reuters-21578/reut2-010.sgm >> adding file 21: ..\src\test\data\reuters-21578/reut2-011.sgm >> adding file 22: ..\src\test\data\reuters-21578/reut2-012.sgm >> adding file 23: ..\src\test\data\reuters-21578/reut2-013.sgm >> adding file 24: ..\src\test\data\reuters-21578/reut2-014.sgm >> adding file 25: ..\src\test\data\reuters-21578/reut2-015.sgm >> adding file 26: ..\src\test\data\reuters-21578/reut2-016.sgm >> adding file 27: ..\src\test\data\reuters-21578/reut2-017.sgm >> adding file 28: ..\src\test\data\reuters-21578/reut2-018.sgm >> adding file 29: ..\src\test\data\reuters-21578/reut2-019.sgm >> adding file 30: ..\src\test\data\reuters-21578/reut2-020.sgm >> adding file 31: ..\src\test\data\reuters-21578/reut2-021.sgm >> >> >> Debug Assertion Failed! >> Expression: _BLOCK_TYPE_IS_VALID(pHead->nBlockUse) >> >> VS 2008 debugger reports a deadlock in: >> atomicthreads\clucene\src\core\CLucene\util\Array.h (line 139) >> >> >> >> 2010/6/12 Klemens Friedl <fri...@gmail.com>: >> > I forgot to mention that I ran the cl_test app earlier today, it >> > stopped with an failure at test 97. >> > (although, I may have used slightly different cmake settings) >> > >> > I will try out that branch tomorrow, as it's already late there. >> > >> > Klemens >> > >> > >> > 2010/6/12 Itamar Syn-Hershko <ita...@divrei-tora.com>: >> >> I'm running cl_test on a similar environment without any problem >> >> (using the default CMake config). One of the tests there >> indexes the >> >> reuters corpus too. Can you try running that? >> >> >> >> The actual error looks like something we fixed in the >> atomicthreads >> >> branch, and wasn't merged into master yet due to lack of feedback. >> >> Can you try running demo from that branch (after trying >> cl_test too)? >> >> >> >> Itamar. >> >> >> >>> -----Original Message----- >> >>> From: Klemens Friedl [mailto:fri...@gmail.com] >> >>> Sent: Saturday, June 12, 2010 10:19 PM >> >>> To: clucene-developers@lists.sourceforge.net >> >>> Subject: [CLucene-dev] clucene - cl_demo stops with error >> >>> >> >>> clucene - cl_demo stops with error while indexing reuters corpus >> >>> >> >>> clucene version: current git current master >> >>> platform: WinXP SP3 >> >>> build system: VS 2008 SP1 >> >>> cmake: 2.8.1 >> >>> cmake settings: see cmakecache.txt file (attached to email) >> >>> >> >>> >> >>> cl_demo app stops with error: >> >>> (one code line changed only to meet path to reuters-21578 corpa >> >>> directory) >> >>> >> >>> >> >>> F:\Home\Search\clucene\build\bin\Debug>cl_demo.exe >> >>> adding file 1: >> >>> src\test\data\reuters-21578/all-exchanges-strings.lc.txt >> >>> adding file 2: src\test\data\reuters-21578/all-orgs-strings.lc.txt >> >>> adding file 3: >> src\test\data\reuters-21578/all-people-strings.lc.txt >> >>> adding file 4: >> src\test\data\reuters-21578/all-places-strings.lc.txt >> >>> adding file 5: >> src\test\data\reuters-21578/all-topics-strings.lc.txt >> >>> adding file 6: >> >>> src\test\data\reuters-21578/cat-descriptions_120396.txt >> >>> adding file 7: >> >>> src\test\data\reuters-21578/feldman-cia-worldfactbook-data.txt >> >>> adding file 8: src\test\data\reuters-21578/LEWIS.DTD >> >>> adding file 9: src\test\data\reuters-21578/README.TXT >> >>> adding file 10: src\test\data\reuters-21578/reut2-000.sgm >> >>> adding file 11: src\test\data\reuters-21578/reut2-001.sgm >> >>> >> >>> => VS 2008 SP1 debugger: >> >>> Unhandled exception at 0x10099e4f (clucene-cored.dll) in >> cl_demo.exe: >> >>> 0xC0000005: Access violation writing location 0x01034f74. >> >>> >> >>> file: >> >>> clucene\src\core\CLucene\index\DocumentsWriterThreadState.cpp >> >>> (line 642) >> >>> >> >>> >> >>> the lucene index file, (output from "dir" command): >> >>> >> >>> F:\Home\Search\clucene\build\bin\Debug\data>dir >> >>> Verzeichnis von F:\Home\Search\clucene\build\bin\Debug\data >> >>> >> >>> 12.06.2010 20:56 <DIR> . >> >>> 12.06.2010 20:56 <DIR> .. >> >>> 12.06.2010 20:56 20 segments.gen >> 12.06.2010 20:56 >> >>> 45 segments_3 12.06.2010 20:56 0 write.lock >> >>> 12.06.2010 20:56 536.020 _0.cfs 12.06.2010 >> 20:58 >> >>> 114.688 _1.fdt 12.06.2010 20:56 0 _1.fdx >> >>> 6 Datei(en) 650.773 Bytes >> >>> >> >>> >> >>> >> >>> If I remove half of the reuters-21578 corpa files of the corpa >> >>> directory, the cl_demo runs through fine !! >> >>> >> >>> >> >>> I tried various settings with cmake. I am using >> gnuwin32's zlib. I >> >>> am not using iconv - as it appeared to me as optional component. >> >>> What are the prefered and tested cmake settings for a common >> >>> environment? >> >>> I need unicode support, multithreading would be a nice to >> have, if >> >>> possible i would like to avoid iconv. >> >>> >> >>> >> >>> Kind regards, >> >>> Klemens Friedl >> >>> >> >>> >> >>> btw. >> >>> the _LUCENE_THREAD_FUNC(atomicIndexTest, _writer) and >> >>> _LUCENE_THREAD_FUNC(atomicSearchTest, _directory) may >> need a return >> >>> statement, as VS informed me, while testing other cmake settings. >> >>> file: clucene\src\test\index\TestThreading.cpp (line 18, 51) >> >>> >> >> >> >> >> >> >> >> >> --------------------------------------------------------------------- >> >> --------- ThinkGeek and WIRED's GeekDad team up for the Ultimate >> >> GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the lucky >> >> parental unit. See the prize list and enter to win: >> >> http://p.sf.net/sfu/thinkgeek-promo >> >> _______________________________________________ >> >> CLucene-developers mailing list >> >> CLucene-developers@lists.sourceforge.net >> >> https://lists.sourceforge.net/lists/listinfo/clucene-developers >> >> >> > >> > > > > ------------------------------------------------------------------------------ > ThinkGeek and WIRED's GeekDad team up for the Ultimate > GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the > lucky parental unit. See the prize list and enter to win: > http://p.sf.net/sfu/thinkgeek-promo > _______________________________________________ > CLucene-developers mailing list > CLucene-developers@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/clucene-developers >
F:\Home\Search\clucene\source\build\bin\Debug>cl_test.exe
Key: .= pass N=not implemented F=fail
All CLucene Tests:
CLucene IndexReader Test: . - 250ms
CLucene Reuters Test: ... - 8343ms
CLucene Analysis Test: . - 0ms
CLucene Analyzers Test: ........ - 125ms
CLucene Document Test: ...... - 4812ms
CLucene Number Tools Test: ... - 438ms
CLucene Debug Test: . - 0ms
CLucene IndexWriter Test: ..... - 1766ms
CLucene High Frequencies Test: . - 16ms
CLucene Priority Queue Test: . - 63ms
CLucene DateTools Test: .. - 0ms
CLucene Query Parser Test: ............... - 46ms
CLucene Multi-Field QP Test: .. - 0ms
CLucene Boolean Tests: .... - 16ms
CLucene Search Test: .............. - 641ms
CLucene Queries Test: .. - 15ms
CLucene Term Vector Test: ..... - 78ms
CLucene Sort Test: ........... - 62ms
CLucene Duplicates Test: .. - 141ms
CLucene DateFilter Test: ... - 31ms
CLucene Wildcard Test: .. - 16ms
CLucene Store Test: ... - 265ms
CLucene UTF8 Test: F. - 157ms
97 tests run: 96 passed, 1 failed, 0 not implemented.
Tests run in 17375ms
ERROR: stringPool still contains intern'd strings (refcounts):
contents (12)
field1 (1)
field2 (1)
field3 (1)
field4 (1)
F:\Home\Search\clucene\source\build\bin\Debug>
F:\Home\Search\clucene\source\build\bin\Debug>cl_demo.exe
adding file 1: ..\src\test\data\reuters-21578/all-exchanges-strings.lc.txt
adding file 2: ..\src\test\data\reuters-21578/all-orgs-strings.lc.txt
adding file 3: ..\src\test\data\reuters-21578/all-people-strings.lc.txt
adding file 4: ..\src\test\data\reuters-21578/all-places-strings.lc.txt
adding file 5: ..\src\test\data\reuters-21578/all-topics-strings.lc.txt
adding file 6: ..\src\test\data\reuters-21578/cat-descriptions_120396.txt
adding file 7: ..\src\test\data\reuters-21578/feldman-cia-worldfactbook-data.txt
adding file 8: ..\src\test\data\reuters-21578/LEWIS.DTD
adding file 9: ..\src\test\data\reuters-21578/README.TXT
adding file 10: ..\src\test\data\reuters-21578/reut2-000.sgm
adding file 11: ..\src\test\data\reuters-21578/reut2-001.sgm
Unhandled exception at 0x1009be5f (clucene-cored.dll) in cl_demo.exe:
0xC0000005: Access violation writing location 0x010256b4.
src\core\CLucene\index\DocumentsWriterThreadState.cpp (line 642)
Call Stack:
>
> clucene-cored.dll!lucene::index::DocumentsWriter::ThreadState::writeProxByte(unsigned
> char b='') Line 642 + 0x15 bytes C++
clucene-cored.dll!lucene::index::DocumentsWriter::ThreadState::writeProxVInt(int
vi=670) Line 622 C++
clucene-cored.dll!lucene::index::DocumentsWriter::ThreadState::FieldData::addPosition(lucene::analysis::Token
* token=0x003eccc0) Line 1174 C++
clucene-cored.dll!lucene::index::DocumentsWriter::ThreadState::FieldData::invertField(lucene::document::Field
* field=0x003eb960, lucene::analysis::Analyzer * analyzer=0x00dafefc, const
int maxFieldLength=10000) Line 902 C++
clucene-cored.dll!lucene::index::DocumentsWriter::ThreadState::FieldData::processField(lucene::analysis::Analyzer
* analyzer=0x00dafefc) Line 797 C++
clucene-cored.dll!lucene::index::DocumentsWriter::ThreadState::processDocument(lucene::analysis::Analyzer
* analyzer=0x00dafefc) Line 554 + 0x1a bytes C++
clucene-cored.dll!lucene::index::DocumentsWriter::updateDocument(lucene::document::Document
* doc=0x00dafe58, lucene::analysis::Analyzer * analyzer=0x00dafefc,
lucene::index::Term * delTerm=0x00000000) Line 934 + 0xc bytes C++
clucene-cored.dll!lucene::index::DocumentsWriter::addDocument(lucene::document::Document
* doc=0x00dafe58, lucene::analysis::Analyzer * analyzer=0x00dafefc) Line 919
C++
clucene-cored.dll!lucene::index::IndexWriter::addDocument(lucene::document::Document
* doc=0x00dafe58, lucene::analysis::Analyzer * analyzer=0x00dafefc) Line 670
+ 0x13 bytes C++
cl_demo.exe!indexDocs(lucene::index::IndexWriter * writer=0x003e6088,
const char * directory=0x00410fa0) Line 80 + 0x11 bytes C++
cl_demo.exe!IndexFiles(const char * path=0x00410fa0, const char *
target=0x00410f98, const bool clearIndex=true) Line 116 + 0xd bytes C++
cl_demo.exe!main(int argc=1, char * * argv=0x003e5e68) Line 44 + 0x10
bytes C++
cl_demo.exe!__tmainCRTStartup() Line 586 + 0x19 bytes C
cl_demo.exe!mainCRTStartup() Line 403 C
kernel32.dll!7c817077()
[Frames below may be incorrect and/or missing, no symbols loaded for
kernel32.dll]
<<attachment: error.PNG>>
F:\Home\Search\clucene\atomicthreads\build3\bin\Debug>cl_demo.exe
adding file 1: ..\src\test\data\reuters-21578/all-exchanges-strings.lc.txt
adding file 2: ..\src\test\data\reuters-21578/all-orgs-strings.lc.txt
adding file 3: ..\src\test\data\reuters-21578/all-people-strings.lc.txt
adding file 4: ..\src\test\data\reuters-21578/all-places-strings.lc.txt
adding file 5: ..\src\test\data\reuters-21578/all-topics-strings.lc.txt
adding file 6: ..\src\test\data\reuters-21578/cat-descriptions_120396.txt
adding file 7: ..\src\test\data\reuters-21578/feldman-cia-worldfactbook-data.txt
adding file 8: ..\src\test\data\reuters-21578/LEWIS.DTD
adding file 9: ..\src\test\data\reuters-21578/README.TXT
adding file 10: ..\src\test\data\reuters-21578/reut2-000.sgm
adding file 11: ..\src\test\data\reuters-21578/reut2-001.sgm
adding file 12: ..\src\test\data\reuters-21578/reut2-002.sgm
adding file 13: ..\src\test\data\reuters-21578/reut2-003.sgm
adding file 14: ..\src\test\data\reuters-21578/reut2-004.sgm
adding file 15: ..\src\test\data\reuters-21578/reut2-005.sgm
adding file 16: ..\src\test\data\reuters-21578/reut2-006.sgm
adding file 17: ..\src\test\data\reuters-21578/reut2-007.sgm
adding file 18: ..\src\test\data\reuters-21578/reut2-008.sgm
adding file 19: ..\src\test\data\reuters-21578/reut2-009.sgm
adding file 20: ..\src\test\data\reuters-21578/reut2-010.sgm
adding file 21: ..\src\test\data\reuters-21578/reut2-011.sgm
adding file 22: ..\src\test\data\reuters-21578/reut2-012.sgm
adding file 23: ..\src\test\data\reuters-21578/reut2-013.sgm
adding file 24: ..\src\test\data\reuters-21578/reut2-014.sgm
adding file 25: ..\src\test\data\reuters-21578/reut2-015.sgm
adding file 26: ..\src\test\data\reuters-21578/reut2-016.sgm
adding file 27: ..\src\test\data\reuters-21578/reut2-017.sgm
adding file 28: ..\src\test\data\reuters-21578/reut2-018.sgm
adding file 29: ..\src\test\data\reuters-21578/reut2-019.sgm
adding file 30: ..\src\test\data\reuters-21578/reut2-020.sgm
adding file 31: ..\src\test\data\reuters-21578/reut2-021.sgm
error message from debugger, see screenshot file!
clucene\src\core\CLucene\util\Array.h (line 139)
Call Stack:
ntdll.dll!7c91e514()
[Frames below may be incorrect and/or missing, no symbols loaded for
ntdll.dll]
ntdll.dll!7c91df4a()
kernel32.dll!7c8648a2()
kernel32.dll!7c8438fa()
kernel32.dll!7c839b39()
ntdll.dll!7c9132a8()
ntdll.dll!7c91327a()
ntdll.dll!7c93aa0f()
ntdll.dll!7c91e48a()
>
> clucene-cored.dll!lucene::util::ObjectArray<lucene::index::DocumentsWriter::Posting>::deleteValues()
> Line 139 + 0x18 bytes C++
clucene-cored.dll!lucene::util::ObjectArray<lucene::index::DocumentsWriter::Posting>::~ObjectArray<lucene::index::DocumentsWriter::Posting>()
Line 148 C++
clucene-cored.dll!lucene::index::DocumentsWriter::~DocumentsWriter()
Line 122 + 0xa2 bytes C++
clucene-cored.dll!lucene::index::DocumentsWriter::`scalar deleting
destructor'() + 0x16 bytes C++
clucene-cored.dll!lucene::index::IndexWriter::closeInternal(bool
waitForMerges=true) Line 541 + 0x2e bytes C++
clucene-cored.dll!lucene::index::IndexWriter::close(bool
waitForMerges=true) Line 480 C++
cl_demo.exe!IndexFiles(const char * path=0x00410fa0, const char *
target=0x00410f98, const bool clearIndex=true) Line 118 + 0xd bytes C++
cl_demo.exe!main(int argc=1, char * * argv=0x003e5e58) Line 44 + 0x10
bytes C++
cl_demo.exe!__tmainCRTStartup() Line 586 + 0x19 bytes C
cl_demo.exe!mainCRTStartup() Line 403 C
kernel32.dll!7c817077()
F:\Home\Search\clucene\atomicthreads\build3\bin\Debug>cl_test.exe
Key: .= pass N=not implemented F=fail
All CLucene Tests:
CLucene Atomic Updates Test: .. - 6203ms
CLucene IndexReader Test: .. - 766ms
CLucene Reuters Test: ... - 8796ms
CLucene Analysis Test: . - 0ms
CLucene Analyzers Test: ......... - 63ms
CLucene Document Test: ...... - 4937ms
CLucene Number Tools Test: ... - 422ms
CLucene Debug Test: . - 0ms
CLucene IndexWriter Test: ...... - 4812ms
CLucene IndexModifier Test: . - 58766ms
CLucene High Frequencies Test: . - 16ms
CLucene Priority Queue Test: . - 78ms
CLucene DateTools Test: .. - 0ms
CLucene Query Parser Test: ............... - 62ms
CLucene Multi-Field QP Test: .. - 16ms
CLucene Boolean Tests: .... - 16ms
CLucene Search Test: .............. - 609ms
CLucene Queries Test: .. - 16ms
CLucene Term Vector Test: ..... - 78ms
CLucene Sort Test: ........... - 62ms
CLucene Duplicates Test: .. - 141ms
CLucene DateFilter Test: ... - 31ms
CLucene Wildcard Test: .. - 16ms
CLucene Store Test: .. - 281ms
CLucene UTF8 Test: F. - 266ms
102 tests run: 101 passed, 1 failed, 0 not implemented.
Tests run in 86469ms
WARNING: stringPool still contains intern'd strings (refcounts):
contents (10)
field1 (5)
field2 (5)
field3 (5)
field4 (5)
id (4)
------------------------------------------------------------------------------ ThinkGeek and WIRED's GeekDad team up for the Ultimate GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the lucky parental unit. See the prize list and enter to win: http://p.sf.net/sfu/thinkgeek-promo
_______________________________________________ CLucene-developers mailing list CLucene-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/clucene-developers
