Noble Paul നോബിള് नोब्ळ् schrieb:
The delta implementation is a bit fragile in DIH for complex queries
that's too bad. It's a nice interface and less complex to configure than
to go the XML /update way.
Well, when doing the way you described below (full-import with the delta
query), the '${dataimporter.last_index_time}' timestamp is empty:
Oct 16, 2008 10:14:53 AM org.apache.solr.handler.dataimport.DataImporter
doFullImport
SEVERE: Full Import failed
org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to
execute query: SELECT a.id AS article_id,a.stub AS article_stub,a.ref AS
article_ref,a.id_blogs,a.title AS article_title, a.normalized_text,
au.url AS article_url, bu.url AS blog_url, b.title AS
blog_title,b.subtitle AS blog_subtitle, r.rank,
coalesce(a.updated,a.published,a.added) as ts, a.stub as article_stub
FROM articles a join blogs b on a.id_blogs = b.id join urls au on
a.id_urls = au.id join urls bu on b.id_urls = bu.id LEFT OUTER JOIN
ranks r on a.id = r.id_articles WHERE b.id_urls is not null AND a.hidden
is false AND b.hidden is false AND a.ref is not null AND b.ref is not
null and (rankid in (SELECT rankid FROM ranks order by rankid desc limit
1) OR rankid is null) AND coalesce(a.updated,a.published,a.added) > ''
Processing Document # 1
Regards
Florian
I recommend you do delta-import using a full-import
it can be done as follows
define a diffferent entity
<dataConfig>
<dataSource type="JdbcDataSource" driver="org.postgresql.Driver"
url="jdbc:postgresql://bm02:5432/bm" user="user" />
<document name="articles">
<entity name="articles-full" ..>
</entity>
<entity name="articles-delta rootEntity="false"
query="<your-delta-query-here>">
<!-- this following entity can be a copy articles-full entity
without any delta query because rootEntity=false for
articles-delta the following will be used for creating
documents. all other rules are same-->
<entity name="anyname" ..>
</entity>
</entity>
</document>
when you wish to do a full-import pass the request parameter
entity=articles-full
for delta-import use the request parameter
entity=articles-delta&clean=false (command has to be full-import only)
On Wed, Oct 15, 2008 at 1:42 PM, Florian Aumeier
<[EMAIL PROTECTED]> wrote:
Shalin Shekhar Mangar schrieb:
You are missing the "pk" field (primary key). This is used for delta
imports.
I added the pk field and rebuild the index yesterday. However, when I run
the delta-import, I still have this error message in the log:
INFO: Starting delta collection.
Oct 15, 2008 9:37:27 AM org.apache.solr.handler.dataimport.DocBuilder
collectDelta
INFO: Running ModifiedRowKey() for Entity: articles
Oct 15, 2008 9:37:27 AM org.apache.solr.handler.dataimport.JdbcDataSource$1
call
INFO: Creating a connection for entity articles with URL:
jdbc:postgresql://bm02:5432/bm
Oct 15, 2008 9:37:27 AM org.apache.solr.handler.dataimport.JdbcDataSource$1
call
INFO: Time taken for getConnection(): 43
Oct 15, 2008 9:37:36 AM org.apache.solr.core.SolrCore execute
INFO: [db] webapp=/solr path=/dataimport params={} status=0 QTime=0
Oct 15, 2008 9:44:51 AM org.apache.solr.core.SolrCore execute
INFO: [db] webapp=/solr path=/dataimport params={} status=0 QTime=0
Oct 15, 2008 9:50:43 AM org.apache.solr.handler.dataimport.DocBuilder
collectDelta
INFO: Completed ModifiedRowKey for Entity: articles rows obtained : 4584
Oct 15, 2008 9:50:43 AM org.apache.solr.handler.dataimport.DocBuilder
collectDelta
INFO: Running DeletedRowKey() for Entity: articles
Oct 15, 2008 9:50:43 AM org.apache.solr.handler.dataimport.DocBuilder
collectDelta
INFO: Completed DeletedRowKey for Entity: articles rows obtained : 0
Oct 15, 2008 9:50:43 AM org.apache.solr.handler.dataimport.DocBuilder
collectDelta
INFO: Completed parentDeltaQuery for Entity: articles
Oct 15, 2008 9:50:43 AM org.apache.solr.handler.dataimport.DataImporter
doDeltaImport
SEVERE: Delta Import Failed
java.lang.NullPointerException
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.getDeltaImportQuery(SqlEntityProcessor.java:153)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.getQuery(SqlEntityProcessor.java:125)
at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:285)
at
org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:211)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:133)
at
org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:359)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:388)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:377)
Oct 15, 2008 9:50:58 AM org.apache.solr.core.SolrCore execute
INFO: [db] webapp=/solr path=/dataimport params={} status=0 QTime=0
Regards
Florian
--
Media Ventures GmbH
Entwicklung Blogmonitor.de
Jabber-ID [EMAIL PROTECTED]
Telefon +49 (0) 2236 480 10 22
