Aklakan commented on issue #1387: URL: https://github.com/apache/jena/issues/1387#issuecomment-1158920943
> Did you consider batching inside the service execution reading from the nextStage(Binding)? Batching based on "push-events" to `nextStage(Binding)` is extremely complex to implement when the rest of ARQ is "pull" based - i.e. operators are implemented as iterators that pull as many items as needed from their underlying inputs in order to produce the next item. As an example why I am not blindly batching by n bindings: It would be easily possible adding a subclass of `QueryIterRepeatApplyBulk` (or maybe it could be called simply QueryIterFlatMap) that calls an abstract nextStage(Binding[] batch). However, e.g. in #1314 I created conceptually a `IterBatchByService` which is an iterator over batches targeted at the same service and which is sourced by the query iterator of the lhs. It aims to create batches of a fixed size but it also adheres to a read ahead limit as to avoid consuming all of the lhs while attempting to fill a batch. This is a bit smarter than blindly batching unrelated bindings. Doing this with a push-based API (that expects immediate return values) would be hell. > Why is single treated different from a bulk of one? Well the single API is a special case of the bulk one - and the bulk one delegates to it. So its not really treating it differently but rather having two reasonable layers of abstraction (That's why I let QueryIterRepeatApply extend from QueryIterRepeatApplyBulk) - maybe the latter should be called simply `QueryIterFlatMap`. Single is certainly more easy to implement - and so far this was how SERVICE was implemented in the first place! Bulk is more powerful because it hands out the QueryIterator but that may require more effort (and knowledge) to work with. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
