Dear Wiki user, You have subscribed to a wiki page or wiki category on "Solr Wiki" for change notification.
The following page has been changed by NoblePaul: http://wiki.apache.org/solr/DataImportHandler ------------------------------------------------------------------------------ * Open the DataImportHandler page to verify if everything is in order [http://localhost:8983/solr/dataimport] * Use full-import command to do a full import from the database and add to SOLR index * Use delta-import command to do a delta import (get new inserts/updates) and add to SOLR index + + [[Anchor(dsconfig)]] == Configuring DataSources == Add the tag 'dataSource' directly under the 'dataConfig' tag. @@ -325, +327 @@ X!PathEntityprocessor supports this with a transformer. If transformer returns a row which contains a field '''`$hasMore`''' with a the value `"true"` the Processor makes another request with the same url template (The actual value is recomputed before invoking ). A transformer can pass a totally new url too for the next call by returning a row which contains a field '''`$nextUrl`''' whose value must be the complete url for the next call. The X!PathEntityProcessor implements a streaming parser which supports a subset of xpath syntax. Complete xpath syntax is not supported but most of the common use cases are covered. + == HttpDataSource Example == @@ -517, +520 @@ * '''`olderThan`''' : A date param . Same rules as above example: {{{ - <entity name="f" processor="FileListEntityProcessor" fileName=".*xml" newerThan="'NOW-3DAYS'" recursive="true" rootEntity="false"> + <entity name="f" processor="FileListEntityProcessor" fileName=".*xml" newerThan="'NOW-3DAYS'" recursive="true" rootEntity="false" dataSource="null"> <entity processor="XPathEntityProcessor" forEach="/the/record/xpath" url="${f.fileAbsolutePath}"> <field column="full_name" xpath="/field/xpath"/> </entity> @@ -562, +565 @@ public void init(Context context, Properties initProps); - /**Get a records for the given query. This is designed to stream records using an iterator + /**Get a records for the given query. This is designed to stream records - * @param query . The query string . can be an sql for RDBMS . + * @param query . The query string . can be an sql for RDBMS . or a url for http etc etc * @return an Object which the Entityprocessor understands. For instance, JdbcDataSource returns an Iterator<Map<String,Object>> and HttpDataSource and FileDataSource returs a java.io.reader */ public T getData(String query); @@ -575, +578 @@ <dataSource type="com.foo.FooDataSource" prop1="hello"/> }}} and it can be used in the entities like a standard one + + === JdbcdataSource === + This is the default. See the [#jdbcdatasource example] . The signature is as follows + {{{ + public class JdbcDataSource implements DataSource<Iterator<Map<String, Object>>> + }}} + + It is designed to iterate rows in DB one by one. A row is represented as a Map. + === HttpDataSource === + This is used by X!PathEntityProcessor . See the documentation [#httpds here] . The signature is as follows + {{{ + public class HttpDataSource implements DataSource<Reader> + }}} + === FileDataSource === + This can be used like an !HttpDataSource . The signature is as follows + {{{ + public class FileDataSource implements DataSource<Reader> + }}} + + The attributes are: + * '''`basePath`''': (optional) The base path relative to which the value is evaluated + * '''`encoding`''': (optional) If the files are to be read in an encoding that is not same as the platform encoding == Adding datasource in solrconfig.xml == [[Anchor(solrconfigdatasource)]]
