Hi, I am using Delta DataImport feature to partially refresh the indexes.
Note - full-import is working perfectly fine. But I need to do delta-import as I do not want to rebuild all the indexes. My database structure is same as mentioned in the example at http://wiki.apache.org/solr/DataImportHandler. http://www.nabble.com/file/p21200498/example-schema.png --------------------------------------------------------------------------- The dataimport.properties has the value as - #Mon Dec 29 14:08:12 IST 2008 last_index_time=2008-12-29 14\:08\:12 --------------------------------------------------------------------------- I have changed the last_modified for the 'tem' table as current timestamp. And when I query the DB, I get the following result - http://www.nabble.com/file/p21200498/data.jpg ------------------------------------------------------------------------------------------------- But when I call delta data-import, it returns me the following response - Server Response: {responseHeader={status=0,QTime=16},initArgs={defaults={config=db-data-config.xml}},command=delta-import,status=idle,importResponse=,statusMessages={},WARNING=This response format is experimental. It is likely to change in the future.} --------------------------------------------------------------------------------------------------- And when I query for data for '*:*', I get the following - Note - Data returned by query doesnt reflect the correct timestamp from database. SolrDocument(1) Field Name & Value: includes=12 Field Name & Value: features=[demo feature, demo feature 1] Field Name & Value: price=12.0 Field Name & Value: timestamp=Mon Dec 29 13:57:54 IST 2008 Field Name & Value: manu=manu-12 Field Name & Value: sku=1 Field Name & Value: id=1 Field Name & Value: popularity=1 Field Name & Value: weight=12.0 SolrDocument(2) Field Name & Value: includes=43 Field Name & Value: features=demo feature 2 Field Name & Value: price=10.0 Field Name & Value: timestamp=Mon Dec 29 13:57:54 IST 2008 Field Name & Value: manu=Demo - 12 Field Name & Value: sku=2 Field Name & Value: id=2 Field Name & Value: popularity=5 Field Name & Value: weight=12.0 SolrDocument(3) Field Name & Value: includes=1 Field Name & Value: features=demo -3 Field Name & Value: price=1.0 Field Name & Value: timestamp=Mon Dec 29 13:57:54 IST 2008 Field Name & Value: manu=manu - 36 Field Name & Value: sku=3 Field Name & Value: id=3 Field Name & Value: popularity=1 Field Name & Value: weight=1.0 SolrDocument(4) Field Name & Value: includes=2 Field Name & Value: features=demo - 4 Field Name & Value: price=2.0 Field Name & Value: timestamp=Mon Dec 29 13:57:55 IST 2008 Field Name & Value: manu=manu - 46 Field Name & Value: sku=4 Field Name & Value: id=4 Field Name & Value: popularity=2 Field Name & Value: weight=2.0 SolrDocument(5) Field Name & Value: includes=3 Field Name & Value: features=demo - 5 Field Name & Value: price=3.0 Field Name & Value: timestamp=Mon Dec 29 13:57:55 IST 2008 Field Name & Value: manu=manu - 56 Field Name & Value: sku=5 Field Name & Value: id=5 Field Name & Value: popularity=3 Field Name & Value: weight=3.0 SolrDocument(6) Field Name & Value: includes=4 Field Name & Value: features=demo - 66 Field Name & Value: price=4.0 Field Name & Value: timestamp=Mon Dec 29 13:57:55 IST 2008 Field Name & Value: manu=manu - 66 Field Name & Value: sku=6 Field Name & Value: id=6 Field Name & Value: popularity=4 Field Name & Value: weight=4.0 ------------------------------------------------------------------------------------------ My Java code that calls delta import is as follows - public static void main(String[] args) throws IOException, SolrServerException { CommonsHttpSolrServer server = new CommonsHttpSolrServer("http://localhost:8983/solr";); BinaryResponseParser parser = new BinaryResponseParser(); server.setParser(parser); ModifiableSolrParams params = new ModifiableSolrParams(); params.set("qt", "/dataimport"); params.set("command", "delta-import"); params.set("commit", "true"); params.set("wt","json"); try{ QueryResponse response = server.query(params); server.commit(); System.out.println("Server Response: " + response); SolrQuery query = new SolrQuery("*:*"); QueryResponse results = server.query(query); SolrDocumentList list = results.getResults(); int index = 1; for (SolrDocument solrDocument : list) { System.out.println("SolrDocument("+index+")"); Iterator<Entry<String, Object>> iterator = solrDocument.iterator(); while(iterator.hasNext()){ System.out.println("Field Name & Value: "+iterator.next()); } index++; } }catch(Exception ex){ System.out.println("Exception Occured:"+ex); } } --------------------------------------------------------------------------- Does SOLR do some kind of caching? I dont understand as why the updated values are not queried correctly from DB? There are no errors in the logs. But I can see an error on the server console - SEVERE: Delta Import Failed java.lang.NullPointerException at org.apache.solr.handler.dataimport.SqlEntityProcessor.getDeltaImportQuery(SqlEntityProcessor.java:153) at org.apache.solr.handler.dataimport.SqlEntityProcessor.getQuery(SqlEntityProcessor.java:125) at org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73) at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:285) at org.apache.solr.handler.dataimport.DocBuilder.doDelta(DocBuilder.java:211) at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:133) at org.apache.solr.handler.dataimport.DataImporter.doDeltaImport(DataImporter.java:359) at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:388) at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:377) What can be the reason for this error? Thanks, Manu -- View this message in context: http://www.nabble.com/Delta-DataImport-is-not-picking-the-modified-value-in-DB-tp21200498p21200498.html Sent from the Solr - User mailing list archive at Nabble.com.
