[
https://issues.apache.org/jira/browse/JENA-109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13096919#comment-13096919
]
Andy Seaborne commented on JENA-109:
------------------------------------
It's a reasonable thing to add.
Setting the window to 100000 interacts with JENA-44, which gives parallelism in
the sort. I suggest a more cautious limit of a few hundred items. Some
performance comparision can then be done.
About the patch:
There's no need to change (TopN). Instead, rewrite the query to have a slice
and
SELECT ..{ } ORDER BY ... OFFSET X LIMIT N
(slice X _
(top (N ...)
(...)
))
that is, put a slice over the "top" and apply the top to N+X. This keeps (top)
focused on what it does - a fixed length sort/limit.
i.e.
SELECT *
{ SELECT ... { } ORDER BY ... LIMIT N+X }
OFFSET X
There's no need for an outer LIMIT (which would be N)
> Optimise ORDER BY + OFFSET + LIMIT queries
> ------------------------------------------
>
> Key: JENA-109
> URL: https://issues.apache.org/jira/browse/JENA-109
> Project: Jena
> Issue Type: Improvement
> Components: ARQ
> Reporter: Paolo Castagna
> Assignee: Paolo Castagna
> Priority: Minor
> Labels: optimization, sparql
> Attachments: JENA-109_ARQ_r1164652.patch
>
>
> The benefits of JENA-89 gets lost as soon as someone uses OFFSET, even for
> low values.
> Maybe we can assume users will not hit 'next page' too many times.
> We can increase the TOPN_LIMIT_THRESHOLD in TransformTopN to 100000 (?) and
> apply the same TopN optimization we discussed in JENA-89 even when we have
> OFFSET (when OFFSET + LIMIT < TOPN_LIMIT_THRESHOLD).
> In QueryIterTopN we simply return IteratorArray.create(y, offset,
> offset+limit) instead of IteratorArray.create(y).
> This way we can avoid total sort for a few number of small pages (all within
> the 100000 threshold).
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
