[
https://issues.apache.org/jira/browse/IGNITE-12291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17340331#comment-17340331
]
Maksim Timonin edited comment on IGNITE-12291 at 5/11/21, 12:36 PM:
--------------------------------------------------------------------
[~amashenkov], [~Yuriy_Shuliha] hi!
I investigated this issue. I found some issues there:
# Text queries ignore pageSize set in a query.
# Lucene returns non-sorted result from every node, so we can't use MergeSort
like SQL queries do that.
So, I think, that number of pages to load can be limited with different
approach that described in the ticket description. We can move check of limit
to the `next()` call instead of `enqueue()`. After `next()` returns NULL
iterator automatically closes and send cancel requests to all nodes.
Please, check this approach, in this PR:
[https://github.com/apache/ignite/pull/9086/files]. There are some new tests
that cover page loading.
Also, I've prepared a draft PR
([https://github.com/apache/ignite/pull/9081/files]) that introduces the
MergeSort reducer. Also it loads pages more carefully, one by one from every
node. It'll be used in index queries, but for TextQueries it's useless due to
our lucene index doesn't return sorted result from every node, so we need to
decide what to do with this issue. So I move this to different ticket
IGNITE-14703.
I think that we should follow both of this approaches but sequentially. Firstly
fix useless page loadings, then provide sorting for text queries. WDYT?
was (Author: timonin.maksim):
[~amashenkov], [~Yuriy_Shuliha] hi!
I investigated this issue. I found some issues there:
# Text queries ignore pageSize set in a query.
# Lucene returns non-sorted result from every node, so we can't use MergeSort
like SQL queries do that.
So, I think, that number of pages to load can be limited with different
approach that described in the ticket description. We can move check of limit
to the `next()` call instead of `enqueue()`. After `next()` returns NULL
iterator automatically closes and send cancel requests to all nodes.
Please, check this approach, in this PR:
[https://github.com/apache/ignite/pull/9086/files]. There are some new tests
that cover page loading.
Also, I've prepared a draft PR
([https://github.com/apache/ignite/pull/9081/files|https://github.com/apache/ignite/pull/9081/files])
that introduces the MergeSort reducer. Also it loads pages more carefully, one
by one from every node. It'll be used in index queries, but for TextQueries
it's useless due to our lucene index doesn't return sorted result from every
node, so we need to decide what to do with this issue.
I think that we should follow both of this approaches but sequentially. Firstly
fix useless page loadings, then provide sorting for text queries. WDYT?
> Create controllable paged query requests / responses for TextQuery similar to
> current SQL result processing
> -----------------------------------------------------------------------------------------------------------
>
> Key: IGNITE-12291
> URL: https://issues.apache.org/jira/browse/IGNITE-12291
> Project: Ignite
> Issue Type: Improvement
> Reporter: Yuriy Shuliha
> Assignee: Maksim Timonin
> Priority: Major
>
> For now query initiator node sends 1 _GridCacheQueryRequest_ and can get
> multiple _GridCacheQueryResponse_-s per remote.
> TextQuery processing finishes when all remotes send _GridCacheQueryResponse_
> with 'finished' flag.
> _TextQuery_ processing should be reworked like it was done for SQL queries.
> Ignite has _GridQueryNextPageRequest \ Response_ classes for SQL result
> processing.
> Similar processing should be implemented for _TextQueries_:
> *GridTextQueryNextPageRequest*
> *GridTextQueryNextPageResponse*
> Proper _TextQuery_ response processing should be implemented in
> _GridCacheQueryFutureAdapter._
> This will allow us to add correct sorting and apply limit correctly on reduce.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
