On Thu, Oct 16, 2008 at 2:08 PM, Florian Aumeier
<[EMAIL PROTECTED]> wrote:
> Noble Paul നോബിള് नोब्ळ् schrieb:
>>
>> The delta implementation is a bit fragile in DIH for complex queries
>>
>>
>
> that's too bad. It's a nice interface and less complex to configure than to
> go the XML /update way.
>
>
> Well, when doing the way you described below (full-import with the delta
> query), the '${dataimporter.last_index_time}' timestamp is empty:
I guess this was fixed post 1.3 . probably you can take
dataimporthandler.jar from a nightly build (you may also need to add
slf4j.jar)
>
> Oct 16, 2008 10:14:53 AM org.apache.solr.handler.dataimport.DataImporter
> doFullImport
> SEVERE: Full Import failed
> org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
> execute query: SELECT a.id AS article_id,a.stub AS article_stub,a.ref AS
> article_ref,a.id_blogs,a.title AS article_title, a.normalized_text, au.url
> AS article_url, bu.url AS blog_url, b.title AS blog_title,b.subtitle AS
> blog_subtitle, r.rank, coalesce(a.updated,a.published,a.added) as ts, a.stub
> as article_stub FROM articles a join blogs b on a.id_blogs = b.id join urls
> au on a.id_urls = au.id join urls bu on b.id_urls = bu.id LEFT OUTER JOIN
> ranks r on a.id = r.id_articles WHERE b.id_urls is not null AND a.hidden is
> false AND b.hidden is false AND a.ref is not null AND b.ref is not null and
> (rankid in (SELECT rankid FROM ranks order by rankid desc limit 1) OR rankid
> is null) AND coalesce(a.updated,a.published,a.added) > '' Processing
> Document # 1
>
> Regards
> Florian
>
>
>> I recommend you do delta-import using a full-import
>>
>> it can be done as follows
>> define a diffferent entity
>>
>> <dataConfig>
>> <dataSource type="JdbcDataSource" driver="org.postgresql.Driver"
>> url="jdbc:postgresql://bm02:5432/bm" user="user" />
>>
>> <document name="articles">
>> <entity name="articles-full" ..>
>> </entity>
>>
>> <entity name="articles-delta rootEntity="false"
>> query="<your-delta-query-here>">
>> <!-- this following entity can be a copy articles-full entity
>> without any delta query because rootEntity=false for
>> articles-delta the following will be used for creating
>> documents. all other rules are same-->
>> <entity name="anyname" ..>
>> </entity>
>> </entity>
>> </document>
>>
>> when you wish to do a full-import pass the request parameter
>> entity=articles-full
>>
>> for delta-import use the request parameter
>> entity=articles-delta&clean=false (command has to be full-import only)
>>
>>
>>
>> On Wed, Oct 15, 2008 at 1:42 PM, Florian Aumeier
>> <[EMAIL PROTECTED]> wrote:
>>
>>>
>>> Shalin Shekhar Mangar schrieb:
>>>
>>>>
>>>> You are missing the "pk" field (primary key). This is used for delta
>>>> imports.
>>>>
>>>>
>>>
>>> I added the pk field and rebuild the index yesterday. However, when I run
>>> the delta-import, I still have this error message in the log:
>>>
>>> INFO: Starting delta collection.
>>> Oct 15, 2008 9:37:27 AM org.apache.solr.handler.dataimport.DocBuilder
>>> collectDelta
>>> INFO: Running ModifiedRowKey() for Entity: articles
>>> Oct 15, 2008 9:37:27 AM
>>> org.apache.solr.handler.dataimport.JdbcDataSource$1
>>> call
>>> INFO: Creating a connection for entity articles with URL:
>>> jdbc:postgresql://bm02:5432/bm
>>> Oct 15, 2008 9:37:27 AM
>>> org.apache.solr.handler.dataimport.JdbcDataSource$1
>>> call
>>> INFO: Time taken for getConnection(): 43
>>> Oct 15, 2008 9:37:36 AM org.apache.solr.core.SolrCore execute
>>> INFO: [db] webapp=/solr path=/dataimport params={} status=0 QTime=0
>>> Oct 15, 2008 9:44:51 AM org.apache.solr.core.SolrCore execute
>>> INFO: [db] webapp=/solr path=/dataimport params={} status=0 QTime=0
>>> Oct 15, 2008 9:50:43 AM org.apache.solr.handler.dataimport.DocBuilder
>>> collectDelta
>>> INFO: Completed ModifiedRowKey for Entity: articles rows obtained : 4584
>>> Oct 15, 2008 9:50:43 AM org.apache.solr.handler.dataimport.DocBuilder
>>> collectDelta
>>> INFO: Running DeletedRowKey() for Entity: articles
>>> Oct 15, 2008 9:50:43 AM org.apache.solr.handler.dataimport.DocBuilder
>>> collectDelta
>>> INFO: Completed DeletedRowKey for Entity: articles rows obtained : 0
>>> Oct 15, 2008 9:50:43 AM org.apache.solr.handler.dataimport.DocBuilder
>>> collectDelta
>>> INFO: Completed parentDeltaQuery for Entity: articles
>>> Oct 15, 2008 9:50:43 AM org.apache.solr.handler.dataimport.DataImporter
>>> doDeltaImport
>>> SEVERE: Delta Import Failed
>>> java.lang.NullPointerException
>>> at
>>>
>>> org.apache.solr.handler.dataimport.SqlEntityProcessor.getDeltaImportQuery(SqlEntityProcessor.java:153)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.SqlEntityProcessor.getQuery(SqlEntityProcessor.java:125)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:285)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:211)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:133)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:359)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:388)
>>> at
>>>
>>> org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:377)
>>> Oct 15, 2008 9:50:58 AM org.apache.solr.core.SolrCore execute
>>> INFO: [db] webapp=/solr path=/dataimport params={} status=0 QTime=0
>>>
>>> Regards
>>> Florian
>>>
>>>
>>
>>
>>
>>
>
>
> --
> Media Ventures GmbH Entwicklung Blogmonitor.de
>
> Jabber-ID [EMAIL PROTECTED]
> Telefon +49 (0) 2236 480 10 22
>
>
--
--Noble Paul
