[
https://issues.apache.org/jira/browse/SOLR-469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12570591#action_12570591
]
Shalin Shekhar Mangar commented on SOLR-469:
--------------------------------------------
Hi Otis,
Thanks for showing interest in this issue and your feedback.
Originally we started developing this to be a pure DB Import tool. But our own
requirements led to us to keep this general enough to be used with other kinds
of data sources. For example, we're using this internally for reading from REST
API's (including RSS/ATOM feeds). Therefore, we kept the name as
DataImportHandler on purpose. Previously, our data source was JdbcDataSource
and EntityProcessor was called SqlEntityProcessor. We later extracted
interfaces out of them as DataSource and EntityProcessor to make them as
generic as possible. Also note that the DataImportHandler does not care about
the name of data-config.xml. It could be called anything, all we need is that
it should be specified in solrconfig.xml
We're developing our generic REST datasources and entity processors and plan to
contribute them as well. We too are looking forward to see this in Solr and
we're committed to do whatever it takes to make sure it becomes a part of Solr.
> DB Import RequestHandler
> ------------------------
>
> Key: SOLR-469
> URL: https://issues.apache.org/jira/browse/SOLR-469
> Project: Solr
> Issue Type: New Feature
> Components: update
> Affects Versions: 1.3
> Reporter: Noble Paul
> Priority: Minor
> Fix For: 1.3
>
> Attachments: SOLR-469.patch, SOLR-469.patch, SOLR-469.patch
>
>
> We need a RequestHandler Which can import data from a DB or other dataSources
> into the Solr index .Think of it as an advanced form of SqlUpload Plugin
> (SOLR-103).
> The way it works is as follows.
> * Provide a configuration file (xml) to the Handler which takes in the
> necessary SQL queries and mappings to a solr schema
> - It also takes in a properties file for the data source
> configuraution
> * Given the configuration it can also generate the solr schema.xml
> * It is registered as a RequestHandler which can take two commands
> do-full-import, do-delta-import
> - do-full-import - dumps all the data from the Database into the
> index (based on the SQL query in configuration)
> - do-delta-import - dumps all the data that has changed since last
> import. (We assume a modified-timestamp column in tables)
> * It provides a admin page
> - where we can schedule it to be run automatically at regular
> intervals
> - It shows the status of the Handler (idle, full-import,
> delta-import)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
