thanks Jonathan. I overlooked the transformer part altogether for CachedSqlEntityProcessor. I shall add a testcase and cut a patch for the same.
On Wed, Jul 23, 2008 at 4:29 PM, Jonathan Lee (JIRA) <[EMAIL PROTECTED]> wrote: > > [ > https://issues.apache.org/jira/browse/SOLR-469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615952#action_12615952 > ] > > jonjlee edited comment on SOLR-469 at 7/23/08 3:59 AM: > ------------------------------------------------------------ > > This patch has been a wonderful addition to solr - thanks for all the work! > > I believe that there is a bug in CachedSqlEntityProcessor that causes > transformers to be ignored. Here is a patch that worked for me, but I am not > sure it is entirely correct: > > {noformat} > --- > contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/CachedSqlEntityProcessor.java > +++ > contrib.new/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/CachedSqlEntityProcessor.java > @@ -43,18 +43,23 @@ > } > > public Map<String, Object> nextRow() { > - if (rowcache != null) > - return getFromRowCache(); > - if (!isFirst) > + Map<String, Object> r; > + if (rowcache != null) { > + r = getFromRowCache(); > + } else if (!isFirst) { > return null; > - String query = > resolver.replaceTokens(context.getEntityAttribute("query")); > - isFirst = false; > - if (simpleCache != null) { > - return getSimplCacheData(query); > } else { > - return getIdCacheData(query); > + String query = > resolver.replaceTokens(context.getEntityAttribute("query")); > + isFirst = false; > + if (simpleCache != null) { > + r = getSimplCacheData(query); > + } else { > + r = getIdCacheData(query); > + } > } > - > + if (r == null) > + return null; > + return applyTransformer(r); > } > > protected List<Map<String, Object>> getAllNonCachedRows() { > {noformat} > > was (Author: jonjlee): > This patch has been a wonderful addition to solr - thanks for all the work! > > I believe that there is a bug in CachedSqlEntityProcessor that causes > transformers to be ignored. Here is a patch that worked for me, but I am not > sure it is entirely correct: > > --- > contrib/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/CachedSqlEntityProcessor.java > +++ > contrib.new/dataimporthandler/src/main/java/org/apache/solr/handler/dataimport/CachedSqlEntityProcessor.java > @@ -43,18 +43,23 @@ > } > > public Map<String, Object> nextRow() { > - if (rowcache != null) > - return getFromRowCache(); > - if (!isFirst) > + Map<String, Object> r; > + if (rowcache != null) { > + r = getFromRowCache(); > + } else if (!isFirst) { > return null; > - String query = > resolver.replaceTokens(context.getEntityAttribute("query")); > - isFirst = false; > - if (simpleCache != null) { > - return getSimplCacheData(query); > } else { > - return getIdCacheData(query); > + String query = > resolver.replaceTokens(context.getEntityAttribute("query")); > + isFirst = false; > + if (simpleCache != null) { > + r = getSimplCacheData(query); > + } else { > + r = getIdCacheData(query); > + } > } > - > + if (r == null) > + return null; > + return applyTransformer(r); > } > > protected List<Map<String, Object>> getAllNonCachedRows() { > > >> Data Import RequestHandler >> -------------------------- >> >> Key: SOLR-469 >> URL: https://issues.apache.org/jira/browse/SOLR-469 >> Project: Solr >> Issue Type: New Feature >> Components: update >> Affects Versions: 1.3 >> Reporter: Noble Paul >> Assignee: Grant Ingersoll >> Fix For: 1.3 >> >> Attachments: SOLR-469-contrib.patch, SOLR-469-contrib.patch, >> SOLR-469-contrib.patch, SOLR-469-contrib.patch, SOLR-469-contrib.patch, >> SOLR-469-contrib.patch, SOLR-469-contrib.patch, SOLR-469-contrib.patch, >> SOLR-469.patch, SOLR-469.patch, SOLR-469.patch, SOLR-469.patch, >> SOLR-469.patch, SOLR-469.patch, SOLR-469.patch, SOLR-469.patch, >> SOLR-469.patch >> >> >> We need a RequestHandler Which can import data from a DB or other >> dataSources into the Solr index .Think of it as an advanced form of >> SqlUpload Plugin (SOLR-103). >> The way it works is as follows. >> * Provide a configuration file (xml) to the Handler which takes in the >> necessary SQL queries and mappings to a solr schema >> - It also takes in a properties file for the data source >> configuraution >> * Given the configuration it can also generate the solr schema.xml >> * It is registered as a RequestHandler which can take two commands >> do-full-import, do-delta-import >> - do-full-import - dumps all the data from the Database into the >> index (based on the SQL query in configuration) >> - do-delta-import - dumps all the data that has changed since last >> import. (We assume a modified-timestamp column in tables) >> * It provides a admin page >> - where we can schedule it to be run automatically at regular >> intervals >> - It shows the status of the Handler (idle, full-import, >> delta-import) > > -- > This message is automatically generated by JIRA. > - > You can reply to this email to add a comment to the issue online. > > -- --Noble Paul
