LuceneError when executed should reproduce the failure. The contrib/benchmark libraries are required. MultiThreadDocAdd is a multithreaded indexing utility class.
On Wed, Mar 25, 2009 at 1:06 PM, Jason Rutherglen < jason.rutherg...@gmail.com> wrote: > Each document is being created in a single thread, and the fields of the > document are not being updated elsewhere. I haven't posted the full code > yet as it needs to cleaned up. Thanks Mike! > > > On Tue, Mar 24, 2009 at 2:43 PM, Michael McCandless < > luc...@mikemccandless.com> wrote: > >> It looks like you are reusing a Field (the f.setValue(...) calls); are >> you sure you're not changing a Document/Field while another thread is >> adding it to the index? >> >> If you can post the full code, then I can try to run it on my >> wikipedia dump locally. >> >> Mike >> >> Jason Rutherglen <jason.rutherg...@gmail.com> wrote: >> > Mike, >> > >> > It only happens when at least 1 million documents are indexed in a >> > multithreaded fashion. Maybe I should post the code? I will try >> indexing >> > without the payload field, I assume it won't fail because I indexed >> > wikipedia before with no issues. >> > >> > Thanks! >> > >> > Jason >> > >> > On Tue, Mar 24, 2009 at 12:25 PM, Michael McCandless < >> > luc...@mikemccandless.com> wrote: >> > >> >> Hmmmm. >> >> >> >> Jason is this easily/compactly repeated? EG, try to index the N docs >> >> before that one. >> >> >> >> If you remove the SinglePayloadTokenStream field, does the exception >> >> still happen? >> >> >> >> Mike >> >> >> >> Jason Rutherglen <jason.rutherg...@gmail.com> wrote: >> >> > While indexing using >> >> > contrib/org.apache.lucene.benchmark.byTask.feeds.EnwikiDocMaker. The >> >> > asserion error is from >> TermsHashPerField.comparePostings(RawPostingList >> >> p1, >> >> > RawPostingList p2). A Payload is added to the document representing >> a >> >> UID. >> >> > Only 1-2 out of 1 million documents indexed generates this error. >> >> > >> >> > java.lang.AssertionError >> >> > problem adding >> >> > >> doc:Document<stored/uncompressed,indexed,tokenized<body:[[Image:Croatia, >> >> > Washington.JPG|right|250px|thumb|The Croatian embassy]] The >> '''Croatian >> >> > Embassy in Washington''' is the [[embassy]] of [[Croatia]] in >> >> [[Washington, >> >> > D.C.]] It is located on [[Embassy Row]] at 2343 [[Massachusetts >> Avenue >> >> > (Washington, DC)|Massachusetts Avenue]], [[Washington DC >> >> > (northwest)|Northwest]] near [[Dupont Circle]]. Previously the >> building >> >> had >> >> > been home to the [[Austrian Embassy in Washington|Austrian embassy]], >> but >> >> > they left for larger quarters and sold the structure to Croatia in >> 1993. >> >> > The purchase and renovation of the building was largely paid for by >> the >> >> > [[Croatian-American]] community. In front of the embassy is a large >> >> > sculpture of [[St. Jerome]] by Croatian sculptor [[Ivan Me?trovi?]]. >> >> > ==External link== *[http://www.croatiaemb.org/ Official site] >> >> > [[Category:Embassies in Washington|Croatia]] [[Category:Foreign >> relations >> >> of >> >> > Croatia]]> stored/uncompressed,indexed,tokenized<doctitle:Embassy of >> >> Croatia >> >> > in Washington> >> stored/uncompressed,indexed,tokenized<docdate:29-JUN-2006 >> >> > 07:27:44.000> stored/uncompressed,indexed,omitNorms<docid:1703107> >> >> > >> >> >> indexed,tokenized<_ID:proj.zoie.api.zoieindexreader$singlepayloadtokenstr...@e7b3cf >> >> > >> >> > indexed<id:667162>> ex: java.lang.AssertionError >> >> > at >> >> > >> >> >> org.apache.lucene.index.TermsHashPerField.comparePostings(TermsHashPerField.java:228) >> >> > at >> >> > >> >> >> org.apache.lucene.index.TermsHashPerField.quickSort(TermsHashPerField.java:144) >> >> > at >> >> > >> >> >> org.apache.lucene.index.TermsHashPerField.sortPostings(TermsHashPerField.java:136) >> >> > at >> >> > >> >> >> org.apache.lucene.index.FreqProxFieldMergeState.<init>(FreqProxFieldMergeState.java:51) >> >> > at >> >> > >> >> >> org.apache.lucene.index.FreqProxTermsWriter.appendPostings(FreqProxTermsWriter.java:202) >> >> > at >> >> > >> >> >> org.apache.lucene.index.FreqProxTermsWriter.flush(FreqProxTermsWriter.java:132) >> >> > at org.apache.lucene.index.TermsHash.flush(TermsHash.java:145) >> >> > at org.apache.lucene.index.DocInverter.flush(DocInverter.java:74) >> >> > at >> >> > >> >> >> org.apache.lucene.index.DocFieldConsumers.flush(DocFieldConsumers.java:75) >> >> > at >> >> > >> >> >> org.apache.lucene.index.DocFieldProcessor.flush(DocFieldProcessor.java:60) >> >> > at >> >> > >> org.apache.lucene.index.DocumentsWriter.flush(DocumentsWriter.java:574) >> >> > at >> org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3533) >> >> > at >> org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3442) >> >> > at >> >> > >> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1922) >> >> > at >> >> > >> org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1880) >> >> > >> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> >> >> >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> >> >
--------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
