You could use separate DIH config files for each of your three tables. This
might be overkill, but it would keep them separate. The DIH is not limited
to one request handler setup, so you could create a unique handler for each
case with a unique name:
<requestHandler name="/indexer/table1"
class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str name="config">table1-config.xml</str>
</lst>
</requestHandler>
<requestHandler name="/indexer/table2"
class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str name="config">table2-config.xml</str>
</lst>
</requestHandler>
<requestHandler name="/indexer/table3"
class="org.apache.solr.handler.dataimport.DataImportHandler">
<lst name="defaults">
<str name="config">table3-config.xml</str>
</lst>
</requestHandler>
When you go to ...solr/admin/dataimport.jsp you should see a list of all
DataImportHandlers that are configured, and can select them individually, if
that works for your needs.
-Jay
http://www.lucidimagination.com
On Fri, Oct 9, 2009 at 10:57 AM, solr.searcher <solr.searc...@gmail.com>wrote:
>
> Hi all,
>
> First of all, please accept my apologies if this has been asked and
> answered
> before. I tried my best to search and couldn't find anything on this.
>
> The problem I am trying to solve is as follows. I have multiple tables with
> identical schema - table_a, table_b, table_c ... and I am trying to create
> one big index with the data from each of these tables. The idea was to
> programatically create the data-config file (just changing the table name)
> and do a reload-config followed by a full-import with clean set to false.
> In
> other words:
>
> 1. publish the data-config file
> 2. do a reload-config
> 3. do a full-import with clean = false
> 4. commit, optimize
> 5. repeat with new table name
>
> I wanted to then follow the same procedure for delta imports. The problem
> is
> that after I do a reload-config and then do a full-import, the old data in
> the index is getting lost.
>
> What am I missing here? Please note that I am new to solr.
>
> INFO: [] webapp=/solr path=/dataimport
> params={command=reload-config&clean=false} status=0 QTime=4
> Oct 9, 2009 10:17:30 AM org.apache.solr.core.SolrCore execute
> INFO: [] webapp=/solr path=/dataimport
> params={command=full-import&clean=false} status=0 QTime=1
> Oct 9, 2009 10:17:30 AM org.apache.solr.handler.dataimport.SolrWriter
> readIndexerProperties
> INFO: Read dataimport.properties
> Oct 9, 2009 10:17:30 AM org.apache.solr.handler.dataimport.DataImporter
> doFullImport
> INFO: Starting Full Import
> Oct 9, 2009 10:17:30 AM org.apache.solr.handler.dataimport.SolrWriter
> readIndexerProperties
> INFO: Read dataimport.properties
> Oct 9, 2009 10:17:30 AM org.apache.solr.handler.dataimport.JdbcDataSource$1
> call
> INFO: Creating a connection for entity <blah blah blah>
> Oct 9, 2009 10:17:30 AM org.apache.solr.handler.dataimport.JdbcDataSource$1
> call
> INFO: Time taken for getConnection(): 12
> Oct 9, 2009 10:17:31 AM org.apache.solr.core.SolrDeletionPolicy onInit
> INFO: SolrDeletionPolicy.onInit: commits:num=1
>
>
> commit{dir=/blah/blah/index,segFN=segments_1z,version=1255032607825,generation=71,filenames=[segments_1z,
> _cl.cfs]
> Oct 9, 2009 10:17:31 AM org.apache.solr.core.SolrDeletionPolicy
> updateCommits
> INFO: last commit = 1255032607825
>
> Any help will be greatly appreciated. Is there any other way to
> automaticaly
> slurp data from multiple, identical tables?
>
> Thanks a lot.
>
> --
> View this message in context:
> http://www.nabble.com/Dynamic-Data-Import-from-multiple-identical-tables-tp25825381p25825381.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
