Re: Issue in parallel Indexing using multiple csv files

[zaheer.java]

Ran more tests. It works.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Issue-in-parallel-Indexing-using-multiple-csv-files-tp4092452p4092873.html
Sent from the Solr - User mailing list archive at Nabble.com.

Issue in parallel Indexing using multiple csv files

[zaheer.java]

Using SOLR 4.4

I'm trying to index solr core using a csv file of around 1 million records.
To improve the performance, I've split the csv files into smaller sizes and
tried to  use csv update handler for each file to run in a separate thread.
The outcome was weird. The total count of Solr Documents doesn't match with
the total number of records in the csv files. But, when I run these in
sequential manner, the outcome is as expected.

So, the question is if it is a good option to run these csv files in
parallel? Does it even work?



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Issue-in-parallel-Indexing-using-multiple-csv-files-tp4092452.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SOLR Error: Document is missing mandatory uniqueKey field

[zaheer.java]

Thank you for your feedback Shawn. Looking closely into the error made me
realize that it was indeed an application error and not related to SolrJ or
SOLR.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/SOLR-Error-Document-is-missing-mandatory-uniqueKey-field-tp4062177p4062535.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: SOLR Error: Document is missing mandatory uniqueKey field

[zaheer.java]

Here is the stack trace:

DEBUG - 2013-05-09 18:53:06.411;
org.apache.solr.update.processor.LogUpdateProcessor; PRE_UPDATE
add{,id=(null)} {wt=javabinversion=2}
DEBUG - 2013-05-09 18:53:06.411;
org.apache.solr.update.processor.LogUpdateProcessor; PRE_UPDATE FINISH
{wt=javabinversion=2}
INFO  - 2013-05-09 18:53:06.412;
org.apache.solr.update.processor.LogUpdateProcessor; [orderitemsStage]
webapp=/solr path=/update params={wt=javabinversion=2}
{add=[488653_0_0_141_388 (1434610076088270848), 488653_0_0_141_388
(1434610076090368000), 488653_0_0_141_388 (1434610076091416576),
488653_0_0_141_388 (1434610076091416577), 488653_0_0_141_388
(1434610076092465152), 488653_0_0_141_388 (1434610076093513728),
488653_0_0_141_388 (1434610076094562304), 488653_0_0_141_388
(1434610076094562305), 488653_0_0_141_388 (1434610076095610880),
488653_0_0_141_388 (1434610076096659456), ... (4031 adds)]} 0 2790
ERROR - 2013-05-09 18:53:06.412; org.apache.solr.common.SolrException;
org.apache.solr.common.SolrException: Document is missing mandatory
uniqueKey field: orderItemKey
at
org.apache.solr.update.AddUpdateCommand.getIndexedId(AddUpdateCommand.java:88)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:517)
at
org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:396)
at
org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
at
org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:246)
at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:173)
at
org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1816)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:656)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:359)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)



--
View this message in context: 
http://lucene.472066.n3.nabble.com/SOLR-Error-Document-is-missing-mandatory-uniqueKey-field-tp4062177p4062178.html
Sent from the Solr - User mailing list archive at Nabble.com.

SOLR Error: Document is missing mandatory uniqueKey field

[zaheer.java]

I repeatedly get this error while adding documents to SOLR using SOLRJ
Document is missing mandatory uniqueKey field: orderItemKey.  This field
is defined as uniqueKey in the Document Schema. I've made sure that I'm
passing this field from Java by logging it upfront. 

As suggested somwhere, I've tried upgrading from 4.0 to 4.3, and also made
the field as required=false. 

Please help me debug or get a resolution to this problem.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/SOLR-Error-Document-is-missing-mandatory-uniqueKey-field-tp4062177.html
Sent from the Solr - User mailing list archive at Nabble.com.