Hi Mitch, thanks for that suggestion. I wasn't aware of that. I've already added a temporary field in my ScriptTransformer that does basically the same.
However, with this approach indexing time went up from 20min to more than 5 hours. The new approach is to query the solr index for that other database that I've already setup. This is only a bit slower than the original query (20min). (I'm using URLDataSource to be 1.4.1 conform.) As with the db entity before, for every document a request is sent to the solr core even if it is useless because the input variable is empty. It seems that once an entity processor kicks in you cannot avoid the initial request to its data source? Thanks, Chantal On Mon, 2010-07-26 at 16:22 +0200, MitchK wrote: > Hi Chantal, > > did you tried to write a http://wiki.apache.org/solr/DIHCustomFunctions > custom DIH Function ? > If not, I think this will be a solution. > Just check, whether "${prog.vip}" is an empty string or null. > If so, you need to replace it with a value that never can response anything. > > So the vip-field will always be empty for such queries. > Maybe that helps? > > Hopefully, the variable resolver is able to resolve something like > ${dih.functions.getReplacementIfNeeded(prog.vip). > > Kind regards, > - Mitch > > > > Chantal Ackermann wrote: > > > > Hi, > > > > my use case is the following: > > > > In a sub-entity I request rows from a database for an input list of > > strings: > > <entity name="prog" ...> > > <field name="vip" ...> /* multivalued, not required */ > > <entity name="ssc_entry" dataSource="ssc" onError="continue" > > query="select SSC_VALUE from SSC_VALUE > > where SSC_ATTRIBUTE_ID=1 > > and SSC_VALUE in (${prog.vip})"> > > <field column="SSC_VALUE" name="vip_ssc" /> > > </entity> > > </entity> > > > > The root entity is "prog" and it has an optional multivalued field > > called "vip". When the list of "vip" values is empty, the SQL for the > > sub-entity above throws an SQLException. (Working with Oracle which does > > not allow an empty expression in the "in"-clause.) > > > > Two things: > > (A) best would be not to run the query whenever ${prog.vip} is null or > > empty. > > (B) From the documentation, it is not clear that onError is only checked > > in the transformer runs but not checked when the SQL for the entity > > throws an exception. (Trunk version JdbcDataSource lines 250pp). > > > > IMHO, (A) is the better fix, and if so, (B) is the right decision. (If > > (A) is not easily fixable, making (B) work would be helpful.) > > > > Looking through the code, I've realized that the replacement of the > > variables is done in a very generic way. I've not yet seen an > > appropriate way to check on those variables in order to stop the > > processing of the entity if the variable is empty. > > Is there a way to do this? Or maybe there is a completely different way > > to get my use case working. Any help most appreciated! > > > > Thanks, > > Chantal > > > > > >
