[
https://issues.apache.org/jira/browse/SOLR-3484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290348#comment-13290348
]
Aaron Simmons commented on SOLR-3484:
-------------------------------------
I have read the related tickets [SOLR-2694, SOLR-2804, SOLR-2694] .. and I
still think this is a bug in LogUpdateProcessor, or maybe SolrWriter's usage of
LogUpdateProcessor?!?
My environment is a Solr Cloud instance with 21 nodes.
(solr-spec-version=4.0.0.2012.03.14.16.31.07)
When we kicked off a regular full-import of a data source (~5 million rows) ..
it starts off fine, but then the import crashes. Looking over the log files, I
find this on one of the nodes:
INFO: [core_NAME_slice11_shard1] webapp=/solr path=/select
params={command=status&qt=/dataimport} status=0 QTime=1
Jun 5, 2012 7:24:22 PM org.apache.solr.handler.dataimport.SolrWriter upload
WARNING: Error creating document : SolrInputDocument[{ --LOTS OF DATA HERE-- }]
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793)
at java.util.HashMap$KeyIterator.next(HashMap.java:828)
at org.apache.solr.common.util.IteratorChain.next(IteratorChain.hava:73)
at
org.apache.solr.common.params.ModifiableSolrParams.add(ModifiableSolrParams.java:116)
at
org.apache.solr.common.params.ModifiableSolrParams.<init>(ModifiableSolrParams.java:56)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:268)
at
org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessor.java:115)
at
org.apache.solr.handler.dataimport.SolrWriter.upload(SolrWriter.java:70)
at
org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHandler.java:289)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:722)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:618)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:324)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:222)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:382)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:450)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:431)
We are using the 'out of the box' DataImportHandler, connecting to a SQL
Server, etc. This seems like a real issue/bug that in
org.apache.solr.update.processor.LogUpdateProcessor.
I saw the patch from SOLR-2804 on 1/1/2012 ... and this seems like a valid fix.
But some people made comments and indicated it was not a valid fix, perhaps
communicating that it's not a real problem? And SOLR-2804 was closed with
"Won't Fix" ... and this bug report is marked as "Not a Problem".
I see this as a very real problem .. but maybe I'm not understanding something.
Thoughts anyone?
> LogUpdateProcessor throws ConcurrentModificationException under
> multi-threading calls
> --------------------------------------------------------------------------------------
>
> Key: SOLR-3484
> URL: https://issues.apache.org/jira/browse/SOLR-3484
> Project: Solr
> Issue Type: Bug
> Components: update
> Affects Versions: 4.0
> Environment: linux
> Reporter: Ethan Tao
>
> Using the LogUpdateProcessor in a singleton chain for concurrent processing
> throws exception. The issue has been reported in SOLR-2694 (closed), and an
> unoffical patch can be found in related bug-id SOLR-2804 patch dated 1/1/12.
> If the patch won't become official for LogUpdateProcessor, suggested to have
> new class "ConcurrentLogUpdateProcessorFactory" to address the thread safe
> issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
