Dear Wiki user, You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.
The following page has been changed by ShalinMangar: http://wiki.apache.org/solr/DataImportHandler The comment on the change is: Cleaned up text, added detailed steps ------------------------------------------------------------------------------ In order to get data from the database, our design philosophy revolves around 'templatized sql' entered by the user for each entity. This gives the user the entire power of SQL if he needs it. The root entity is the central table whose primary key can be used to join this table with other child entities. - = Example = + == Using full-import command == + Full Import operation can be started by hitting the URL [http://localhost:8983/solr/dataimport?command=full-import]. This operation will be started in a new thread and the ''status'' attribute in the response should be shown ''busy'' now. Depending on the size of your data set, this operation may take some time. At any time, you can hit [http://localhost:8983/solr/dataimport] to see the status flag. + + When full-import command is executed, it stores the start time of the operation in a file located at ''conf/dataimport.properties''. This stored timestamp is used when a delta-import operation is executed. + + === Full Import Example === Let us consider an example. Suppose we have the following schema in our database @@ -63, +68 @@ This is a relational model of the same schema that SOLR currently ships with. We will use this as an example to build a data-config.xml for DataImportHandler. We've created a sample database with this schema in HSQLDB. To run it, do the following steps: - * Download this [http://wiki.apache.org/solr-data/attachments/DataImportHandler/attachments/hsqldb-database.zip hsqldb-database.zip] to execute this example. Extract the downloaded zip file into c:\temp. Also save the following xml as c:\temp\example-data-config.xml + * Download [http://wiki.apache.org/solr-data/attachments/DataImportHandler/attachments/hsqldb-database.zip hsqldb-database.zip] to execute this example. This zip file contains all the hsqldb related files. Extract the downloaded zip file into c:\temp. Also save the following xml as c:\temp\example-data-config.xml {{{ <dataConfig> @@ -96, +101 @@ </dataConfig> }}} - * Now modify the dataimport section in solrconfig.xml to the following: - {{{ - <requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler"> - <lst name="defaults"> - <str name="config">c:/temp/example-data-config.xml</str> - <lst name="datasource"> - <str name="driver">org.hsqldb.jdbcDriver</str> - <str name="url">jdbc:hsqldb:/temp/example/ex</str> - <str name="user">sa</str> - <str name="password"></str> - </lst> - </lst> - </requestHandler> - }}} - - * [http://www.hsqldb.org/ Download] hsqldb driver jar and add it to c:\apache-solr-nightly\lib - * Use "ant dist" to rebuild the apache-solr-1.3.war - * Use the Solr jetty start.jar to startup solr - * Follow full-import section to do a full import - Here, the root entity is a table called "item" whose primary key is a column "id". Data can be read from this table with the query "select * from item". Each item can have multiple "features" which are in the table ''feature'' inside the column ''description''. Note the query in ''feature'' entity: {{{ <entity name="feature" pk="ITEM_ID" @@ -134, +119 @@ </entity> }}} + * [http://www.hsqldb.org/ Download] hsqldb driver jar and add it to ''lib'' folder of your solr folder e.g. c:\apache-solr-nightly\lib + * Use "ant example" to build the example application + * Goto ''example/solr/conf'' folder in your solr installation directory + * Now modify the dataimport section in solrconfig.xml to the following: + {{{ + <requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler"> + <lst name="defaults"> + <str name="config">c:/temp/example-data-config.xml</str> + <lst name="datasource"> + <str name="driver">org.hsqldb.jdbcDriver</str> + <str name="url">jdbc:hsqldb:/temp/example/ex</str> + <str name="user">sa</str> + <str name="password"></str> + </lst> + </lst> + </requestHandler> + }}} + + * Goto ''example'' folder and start Solr using the embedded jetty server using "java -jar start.jar". This will start Solr on port ''8983'' + * Hit [http://localhost:8983/solr/dataimport] with a browser to verify the configuration. + * Hit [http://localhost:8983/solr/dataimport?command=full-import] to do a full import. + '''NOTE''' * The data-config in this example can also be written with only one entity ''item'' using SQL joins. In that case columns in ''category'' and ''feature'' tables can directly be read from the ''item'' entity. * This example does not use the delta features. We will add more examples soon. - - == Using full-import command == - Full Import operation can be started by hitting the URL [http://localhost:8983/solr/dataimport?command=full-import]. This operation will be started in a new thread and the ''status'' attribute in the response should be shown ''busy'' now. Depending on the size of your data set, this operation may take some time. At any time, you can hit [http://localhost:8983/solr/dataimport] to see the status flag. - - When full-import command is executed, it stores the start time of the operation in a file located at ''conf/dataimport.properties''. This stored timestamp is used when a delta-import operation is executed. == Using delta-import command == Delta Import operation can be started by hitting the URL [http://localhost:8983/solr/dataimport?command=delta-import]. This operation will be started in a new thread and the ''status'' attribute in the response should be shown ''busy'' now. Depending on the size of your data set, this operation may take some time. At any time, you can hit [http://localhost:8983/solr/dataimport] to see the status flag.
