I am having some problem importing data via dataimporter. I have a database
(data_archive) containing records for the last 9 years. I am unable to
import all the data via dataimporter. The solr stops. However, I can import
data for the last 12 months and it takes around 20 minutes. How can I add
all the data from the data_archive database and then just add new records
that get added everyday in the data_archive database.
The script I am using is -
<entity name="data_archive" dataSource="ds8" pk="posting_id"
query="SELECT * FROM data_archive WHERE doc_date BETWEEN (CURDATE() -
INTERVAL 12 MONTH) AND CURDATE()"
deltaImportQuery="SELECT * FROM data_archive
WHERE ID = '${dataimporter.delta.posting_id}' AND doc_date BETWEEN
(CURDATE() - INTERVAL 12 MONTH) AND CURDATE()"
deltaQuery="SELECT posting_id FROM data_archive
WHERE last_modified > '${dataimporter.last_index_time}' AND
doc_date BETWEEN (CURDATE() - INTERVAL 12 MONTH) AND CURDATE()">
</entity>
Can someone please modify the above script.
Regards,
Anuj
