may be you can try "postImportDeleteQuery" (not yet documented ,
SOLR-801) on a root entity.
You can keep a timestamp in the fields which can keep the value of
${dataimporter.index_start_time} as a field . Use that to remove old
docs which may exist in the index before the indexing started
--Noble
On Tue, Feb 17, 2009 at 4:42 PM, Steffen B. <s.baumg...@fhtw-berlin.de> wrote:
>
> Hi there,
> I've got a pretty simple question regarding the DIH full-import command.
> I have a SOLR server running that has a full index with lots of documents in
> it. Once a day, a full-import is run, which uses the default parameters
> (clean=true, because it's not an incremental index).
> When I run a full-import, the first step is cleaning up the whole index:
>
> Feb 7, 2009 2:12:01 AM org.apache.solr.update.DirectUpdateHandler2 deleteAll
> INFO: [] REMOVING ALL DOCUMENTS FROM INDEX
>
> After that, suppose the import suddenly fails for one reason or another (ie.
> SQL error), which initiates a rollback:
>
> Feb 7, 2009 2:12:02 AM org.apache.solr.handler.dataimport.DataImporter
> doFullImport
> SEVERE: Full Import failed
> [...]
> Feb 7, 2009 2:12:02 AM org.apache.solr.update.DirectUpdateHandler2 rollback
> INFO: start rollback
> Feb 7, 2009 2:12:02 AM org.apache.solr.update.DirectUpdateHandler2 rollback
> INFO: end_rollback
> Feb 7, 2009 2:12:02 AM org.apache.solr.update.DirectUpdateHandler2 commit
> INFO: start commit(optimize=false,waitFlush=false,waitSearcher=true)
>
> Unfortunately, this rollback does not "refill" the index with the old data,
> and neither keeps the old index from being overwritten with the new,
> erroneous index. Now my question is: is there anything I can do to keep Solr
> from trashing my index on a full-import when there is a problem with the
> database?
> Or should I use clean=false, even though 99% of the imported documents are
> not incremental but the same documents that already were in the index, only
> with new data?
> Any tips will be greatly appreciated! :)
> - Steffen
> --
> View this message in context:
> http://www.nabble.com/DIH-full-import-with-clean%3Dtrue-fails-and-rollback-empties-index-tp22055065p22055065.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
--
--Noble Paul
