[
https://issues.apache.org/jira/browse/SOLR-994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12668350#action_12668350
]
Jared Flatow commented on SOLR-994:
-----------------------------------
The transformer must know somehow when $hasMore should be true. If the
transformer always give $hasMore a value "true", will there be infinite
requests made or will it stop on the first empty request? Using the
EnumeratedEntityTransformer, a user can specify from the config xml when
$hasMore should be true using the chunkSize attribute. This solves a general
case of "request N rows at a time until no more are available". I agree, a
combination of 'rowsFetchedCount' and a HasMoreUntilEmptyTransformer would also
make this doable from the configuration.
> EnumeratedEntityTransformer
> ---------------------------
>
> Key: SOLR-994
> URL: https://issues.apache.org/jira/browse/SOLR-994
> Project: Solr
> Issue Type: New Feature
> Components: contrib - DataImportHandler
> Affects Versions: 1.4
> Reporter: Jared Flatow
> Priority: Minor
> Fix For: 1.4
>
> Attachments: SOLR-994.patch
>
>
> An EnumeratedEntityTransformer gives the Nth entity an accessible
> ${<entity>.n} == N. In addition, the entity may specify a chunkSize
> attribute, which will cause the chunkSize'th entity to gain the attribute
> $hasMore=true. A template for a nextUrl may also be specified on the entity,
> that is different from the url template.
> Consider an API:
> http://host:port/path/to/resource?maximum_number_returned=50&return_start_index=0
> an entity could specify:
> <entity name="myEntity" processor="XPathEntityProcessor"
> transformer="EnumeratedEntityTransformer"
> url="http://host:port/path/to/resource?maximum_number_returned=50&return_start_index=${myEntity.n}";
> chunkSize=50>...</entity>
> This allows for fetching entities in chunks until there are < chunkSize
> returned.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
