Just extend XPathEntityProcessor override nextRow() after 100
return null. Use it as your processor
--Noble
On Tue, Jun 24, 2008 at 10:45 AM, Noble Paul നോബിള് नोब्ळ्
<[EMAIL PROTECTED]> wrote:
> Just extend XPathEntityProcessor override nextRow() after 100 . Use
> it as your processor
> return null;
>
> On Tue, Jun 24, 2008 at 10:23 AM, mike segv <[EMAIL PROTECTED]> wrote:
>>
>> That fixed it.
>>
>> If I'm inserting millions of documents, how do I control docs/update? E.g.
>> if there are 50K docs per file, I'm thinking that I should probably code up
>> my own DataSource that allows me to stipulate docs/update. Like say, 100
>> instead of 50K. Does this make sense?
>>
>> Mike
>>
>>
>> Noble Paul നോബിള് नोब्ळ् wrote:
>>>
>>> hi ,
>>> You have not registered any datasources . the second entity needs a
>>> datasource.
>>> Remove the dataSource="null" and add a name for the second entity
>>> (good practice). No need for baseDir attribute for second entity .
>>> See the modified xml added below
>>> --Noble
>>>
>>> <dataConfig>
>>> <dataSource type="FileDataSource"/>
>>> <document>
>>> <entity name="f" processor="FileListEntityProcessor" fileName=".*xml"
>>> newerThan="'NOW-10DAYS'" recursive="true" rootEntity="false"
>>> dataSource="null" baseDir="/san/tomcat-services/solr-medline">
>>> <entity name="x" processor="XPathEntityProcessor"
>>> forEach="/MedlineCitation"
>>> url="${f.fileAbsolutePath}" >
>>> <field column="pmid" xpath="/MedlineCitation/PMID"/>
>>> </entity>
>>> </entity>
>>> </document>
>>> </dataConfig>
>>>
>>> On Tue, Jun 24, 2008 at 6:39 AM, mike segv <[EMAIL PROTECTED]> wrote:
>>>>
>>>> I'm trying to use the fileListEntityProcessor to add some xml documents
>>>> to a
>>>> solr index. I'm running a nightly version of solr-1.3 with SOLR-469 and
>>>> SOLR-563. I've been able to successfuly run the slashdot httpDataSource
>>>> example. My data-config.xml file loads without errors. When I attempt
>>>> the
>>>> full-import command I get the exception below. Thanks for any help.
>>>>
>>>> Mike
>>>>
>>>> WARNING: No lockType configured for
>>>> /san/tomcat-services/solr-medline/solr/data/index/ assuming 'simple'
>>>> Jun 23, 2008 7:59:49 PM org.apache.solr.handler.dataimport.DataImporter
>>>> doFullImport
>>>> SEVERE: Full Import failed
>>>> java.lang.RuntimeException: java.lang.NullPointerException
>>>> at
>>>> org.apache.solr.handler.dataimport.XPathRecordReader.streamRecords(XPathRecordReader.java:97)
>>>> at
>>>> org.apache.solr.handler.dataimport.XPathEntityProcessor.initQuery(XPathEntityProcessor.java:212)
>>>> at
>>>> org.apache.solr.handler.dataimport.XPathEntityProcessor.fetchNextRow(XPathEntityProcessor.java:166)
>>>> at
>>>> org.apache.solr.handler.dataimport.XPathEntityProcessor.nextRow(XPathEntityProcessor.java:149)
>>>> at
>>>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:286)
>>>> at
>>>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:312)
>>>> at
>>>> org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:179)
>>>> at
>>>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:140)
>>>> at
>>>> org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:335)
>>>> at
>>>> org.apache.solr.handler.dataimport.DataImporter.rumCmd(DataImporter.java:386)
>>>> at
>>>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:377)
>>>> Caused by: java.lang.NullPointerException
>>>> at java.io.Reader.<init>(Reader.java:61)
>>>> at java.io.BufferedReader.<init>(BufferedReader.java:76)
>>>> at com.bea.xml.stream.MXParser.checkForXMLDecl(MXParser.java:775)
>>>> at com.bea.xml.stream.MXParser.setInput(MXParser.java:806)
>>>> at
>>>> com.bea.xml.stream.MXParserFactory.createXMLStreamReader(MXParserFactory.java:261)
>>>> at
>>>> org.apache.solr.handler.dataimport.XPathRecordReader.streamRecords(XPathRecordReader.java:93)
>>>> ... 10 more
>>>>
>>>> Here is my data-config:
>>>>
>>>> <dataConfig>
>>>> <document>
>>>> <entity name="f" processor="FileListEntityProcessor" fileName=".*xml"
>>>> newerThan="'NOW-10DAYS'" recursive="true" rootEntity="false"
>>>> dataSource="null" baseDi
>>>> r="/san/tomcat-services/solr-medline">
>>>> <entity processor="XPathEntityProcessor" forEach="/MedlineCitation"
>>>> url="${f.fileAbsolutePath}" dataSource="null">
>>>> <field column="pmid" xpath="/MedlineCitation/PMID"/>
>>>> </entity>
>>>> </entity>
>>>> </document>
>>>> </dataConfig>
>>>>
>>>> And a snippet from an xml file:
>>>> <MedlineCitation Owner="PIP" Status="MEDLINE">
>>>> <PMID>12236137</PMID>
>>>> <DateCreated>
>>>> <Year>1980</Year>
>>>> <Month>01</Month>
>>>> <Day>03</Day>
>>>> </DateCreated>
>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://www.nabble.com/Attempting-dataimport-using-FileListEntityProcessor-tp18081671p18081671.html
>>>> Sent from the Solr - User mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> --Noble Paul
>>>
>>>
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Attempting-dataimport-using-FileListEntityProcessor-tp18081671p18083747.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
>
>
>
> --
> --Noble Paul
>
--
--Noble Paul
