[jira] [Commented] (IGNITE-12401) Some Text Queries return repeated results
[ https://issues.apache.org/jira/browse/IGNITE-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17374607#comment-17374607 ] Atri Sharma commented on IGNITE-12401: -- [~Pavlukhin] Thanks, just sent an email :) > Some Text Queries return repeated results > - > > Key: IGNITE-12401 > URL: https://issues.apache.org/jira/browse/IGNITE-12401 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.8 >Reporter: Ilya Kasnacheev >Assignee: Atri Sharma >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > It came to my attention while checking for Range Queries support that we > don't actually check that found query results are the correct ones. > We were checking that we got some results, but not whether they were expected. > And voila, it turns out that Range Queries examples, as well as some other > test cases, will readily fail when run with such checks! A query will return > same value repeatedly, e.g. range query will return the "1" record twice, and > limited text query will return "14" record twice. > It didn't really occur on non-range queries before the introduction of limits. > I think we should not ship broken limit queries. Maybe also fix range > queries, if it's hard let's @Ignore them for now. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12401) Some Text Queries return repeated results
[ https://issues.apache.org/jira/browse/IGNITE-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17374603#comment-17374603 ] Ivan Pavlukhin commented on IGNITE-12401: - [~atri], by mistake I assigned this ticket to myself. Unnasigned now. Feel free to assign it to yourself. Also I suggest to start (or resume) a discussion thread on d...@ignite.apache.org. The ticket looks abandoned and it is worth refreshing text queries track with the community. > Some Text Queries return repeated results > - > > Key: IGNITE-12401 > URL: https://issues.apache.org/jira/browse/IGNITE-12401 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.8 >Reporter: Ilya Kasnacheev >Assignee: Atri Sharma >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > It came to my attention while checking for Range Queries support that we > don't actually check that found query results are the correct ones. > We were checking that we got some results, but not whether they were expected. > And voila, it turns out that Range Queries examples, as well as some other > test cases, will readily fail when run with such checks! A query will return > same value repeatedly, e.g. range query will return the "1" record twice, and > limited text query will return "14" record twice. > It didn't really occur on non-range queries before the introduction of limits. > I think we should not ship broken limit queries. Maybe also fix range > queries, if it's hard let's @Ignore them for now. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12401) Some Text Queries return repeated results
[ https://issues.apache.org/jira/browse/IGNITE-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17374585#comment-17374585 ] Atri Sharma commented on IGNITE-12401: -- [~Pavlukhin] [~ilyak] Can you please assign this ticket to me? I would want to add it to the text queries refactoring work that I am focusing on. Maybe we can see this ticket and figure out if there is a way to avoid working on this ticket right now and move forward with text queries? > Some Text Queries return repeated results > - > > Key: IGNITE-12401 > URL: https://issues.apache.org/jira/browse/IGNITE-12401 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.8 >Reporter: Ilya Kasnacheev >Assignee: Ivan Pavlukhin >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > It came to my attention while checking for Range Queries support that we > don't actually check that found query results are the correct ones. > We were checking that we got some results, but not whether they were expected. > And voila, it turns out that Range Queries examples, as well as some other > test cases, will readily fail when run with such checks! A query will return > same value repeatedly, e.g. range query will return the "1" record twice, and > limited text query will return "14" record twice. > It didn't really occur on non-range queries before the introduction of limits. > I think we should not ship broken limit queries. Maybe also fix range > queries, if it's hard let's @Ignore them for now. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12401) Some Text Queries return repeated results
[ https://issues.apache.org/jira/browse/IGNITE-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17374579#comment-17374579 ] Ivan Pavlukhin commented on IGNITE-12401: - Sorry, I cannot help here. So far from context =( > Some Text Queries return repeated results > - > > Key: IGNITE-12401 > URL: https://issues.apache.org/jira/browse/IGNITE-12401 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.8 >Reporter: Ilya Kasnacheev >Assignee: Yuriy Shuliha >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > It came to my attention while checking for Range Queries support that we > don't actually check that found query results are the correct ones. > We were checking that we got some results, but not whether they were expected. > And voila, it turns out that Range Queries examples, as well as some other > test cases, will readily fail when run with such checks! A query will return > same value repeatedly, e.g. range query will return the "1" record twice, and > limited text query will return "14" record twice. > It didn't really occur on non-range queries before the introduction of limits. > I think we should not ship broken limit queries. Maybe also fix range > queries, if it's hard let's @Ignore them for now. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12401) Some Text Queries return repeated results
[ https://issues.apache.org/jira/browse/IGNITE-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370634#comment-17370634 ] Ilya Kasnacheev commented on IGNITE-12401: -- Unfortunately, I'm not sure since I don't know rebalance process in-depth. Maybe [~Pavlukhin] can help? > Some Text Queries return repeated results > - > > Key: IGNITE-12401 > URL: https://issues.apache.org/jira/browse/IGNITE-12401 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.8 >Reporter: Ilya Kasnacheev >Assignee: Yuriy Shuliha >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > It came to my attention while checking for Range Queries support that we > don't actually check that found query results are the correct ones. > We were checking that we got some results, but not whether they were expected. > And voila, it turns out that Range Queries examples, as well as some other > test cases, will readily fail when run with such checks! A query will return > same value repeatedly, e.g. range query will return the "1" record twice, and > limited text query will return "14" record twice. > It didn't really occur on non-range queries before the introduction of limits. > I think we should not ship broken limit queries. Maybe also fix range > queries, if it's hard let's @Ignore them for now. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12401) Some Text Queries return repeated results
[ https://issues.apache.org/jira/browse/IGNITE-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17370494#comment-17370494 ] Atri Sharma commented on IGNITE-12401: -- [~ilyak] What do we need to fix this one, please? > Some Text Queries return repeated results > - > > Key: IGNITE-12401 > URL: https://issues.apache.org/jira/browse/IGNITE-12401 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.8 >Reporter: Ilya Kasnacheev >Assignee: Yuriy Shuliha >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > It came to my attention while checking for Range Queries support that we > don't actually check that found query results are the correct ones. > We were checking that we got some results, but not whether they were expected. > And voila, it turns out that Range Queries examples, as well as some other > test cases, will readily fail when run with such checks! A query will return > same value repeatedly, e.g. range query will return the "1" record twice, and > limited text query will return "14" record twice. > It didn't really occur on non-range queries before the introduction of limits. > I think we should not ship broken limit queries. Maybe also fix range > queries, if it's hard let's @Ignore them for now. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12401) Some Text Queries return repeated results
[ https://issues.apache.org/jira/browse/IGNITE-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17367151#comment-17367151 ] Ilya Kasnacheev commented on IGNITE-12401: -- I don't think that we have fixed it. > Some Text Queries return repeated results > - > > Key: IGNITE-12401 > URL: https://issues.apache.org/jira/browse/IGNITE-12401 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.8 >Reporter: Ilya Kasnacheev >Assignee: Yuriy Shuliha >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > It came to my attention while checking for Range Queries support that we > don't actually check that found query results are the correct ones. > We were checking that we got some results, but not whether they were expected. > And voila, it turns out that Range Queries examples, as well as some other > test cases, will readily fail when run with such checks! A query will return > same value repeatedly, e.g. range query will return the "1" record twice, and > limited text query will return "14" record twice. > It didn't really occur on non-range queries before the introduction of limits. > I think we should not ship broken limit queries. Maybe also fix range > queries, if it's hard let's @Ignore them for now. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12401) Some Text Queries return repeated results
[ https://issues.apache.org/jira/browse/IGNITE-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17366452#comment-17366452 ] Atri Sharma commented on IGNITE-12401: -- [~Pavlukhin] [~ilyak] Did we ever fix MOVING partitions? If not, I would like to help on this ticket, please > Some Text Queries return repeated results > - > > Key: IGNITE-12401 > URL: https://issues.apache.org/jira/browse/IGNITE-12401 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.8 >Reporter: Ilya Kasnacheev >Assignee: Yuriy Shuliha >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > It came to my attention while checking for Range Queries support that we > don't actually check that found query results are the correct ones. > We were checking that we got some results, but not whether they were expected. > And voila, it turns out that Range Queries examples, as well as some other > test cases, will readily fail when run with such checks! A query will return > same value repeatedly, e.g. range query will return the "1" record twice, and > limited text query will return "14" record twice. > It didn't really occur on non-range queries before the introduction of limits. > I think we should not ship broken limit queries. Maybe also fix range > queries, if it's hard let's @Ignore them for now. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12401) Some Text Queries return repeated results
[ https://issues.apache.org/jira/browse/IGNITE-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16991506#comment-16991506 ] Ilya Kasnacheev commented on IGNITE-12401: -- OK, I can fix range query once we do anything to MOVING partitions. This sounds pretty bad. > Some Text Queries return repeated results > - > > Key: IGNITE-12401 > URL: https://issues.apache.org/jira/browse/IGNITE-12401 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.8 >Reporter: Ilya Kasnacheev >Assignee: Yuriy Shuliha >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > It came to my attention while checking for Range Queries support that we > don't actually check that found query results are the correct ones. > We were checking that we got some results, but not whether they were expected. > And voila, it turns out that Range Queries examples, as well as some other > test cases, will readily fail when run with such checks! A query will return > same value repeatedly, e.g. range query will return the "1" record twice, and > limited text query will return "14" record twice. > It didn't really occur on non-range queries before the introduction of limits. > I think we should not ship broken limit queries. Maybe also fix range > queries, if it's hard let's @Ignore them for now. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12401) Some Text Queries return repeated results
[ https://issues.apache.org/jira/browse/IGNITE-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16986938#comment-16986938 ] Ivan Pavlukhin commented on IGNITE-12401: - [~Yuriy_Shuliha], [~ilyak], I dig into the problem and found that the cause of the test failure is improper mapping of text queries on topology when there are MOVING partitions. The scenario in test is as follows: 1. Start 2 nodes. 2. Fill cache with data. 3. Execute text query and check results. And the tricky thing here is that between steps 1 and 2 there are MOVING partitions! Really misleading, because it seems that there should be no data movement as cluster starts without data, but the reality is different. As a result text query loads some entries twice (from both nodes). So, we identified that _mapping_ of text queries on topology works incorrectly. Consequently incorrect results on topologies with MOVING partitions. > Some Text Queries return repeated results > - > > Key: IGNITE-12401 > URL: https://issues.apache.org/jira/browse/IGNITE-12401 > Project: Ignite > Issue Type: Bug > Components: cache >Affects Versions: 2.8 >Reporter: Ilya Kasnacheev >Assignee: Yuriy Shuliha >Priority: Critical > Time Spent: 10m > Remaining Estimate: 0h > > It came to my attention while checking for Range Queries support that we > don't actually check that found query results are the correct ones. > We were checking that we got some results, but not whether they were expected. > And voila, it turns out that Range Queries examples, as well as some other > test cases, will readily fail when run with such checks! A query will return > same value repeatedly, e.g. range query will return the "1" record twice, and > limited text query will return "14" record twice. > It didn't really occur on non-range queries before the introduction of limits. > I think we should not ship broken limit queries. Maybe also fix range > queries, if it's hard let's @Ignore them for now. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12401) Some Text Queries return repeated results
[
https://issues.apache.org/jira/browse/IGNITE-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16985221#comment-16985221
]
Yuriy Shuliha commented on IGNITE-12401:
-
After investigations, there're 2 causes of tests failure found:
*1. Related to Range tests only*
{{testTextQueryWithRange()}} uses range {{"[10 TO 20}" }}In this case the
Lucene response "unexpectedly" will also contain a key "2", but this key is not
among expected keys, because they are selected by predicate {{x ->
String.valueOf(x).startsWith("1")}} .
The clue is that here Lucene compares range boundaries as Strings. Where "10"
is less then "2" and "20" is less then "3".
[https://docs.oracle.com/javase/7/docs/api/java/lang/String.html#compareTo(java.lang.String)]
More detailed explanation:
For {{v IN range[A TO B]}}
Lucene uses rather the next logic with string comparison (black-box tested by
me):
{{(A.compareTo(v) <= inclusive ? 0 : -1) && (B.compareTo(v) >= inclusive ? 0 :
1)}}
{{=== A range 10 includes === === B range 20 excludes ===}}
{{"10".compareTo("2") == -1 "20".compareTo("2") == 1 // passed, false
positive}}
{{"10".compareTo("15") == -5 "20".compareTo("15") == 1 // passed}}
{{"10".compareTo("25") == -1 "20".compareTo("20") == 0 // passed}}
{{"10".compareTo("3") == -2 "20".compareTo("3") == -1 // not passed}}
*2. Distributed Query produces responses from two nodes that partially contain
the same keys*
When checking the text query response, I noticed that GridLuceneIndex.query()
was executed on 2 grids/nodes per single request. This is expected for
distributed query. But on the other hand is cache mode is PARTITIONED, they
should not contain overlapping keys.
But here Lucene indexes on different nodes definitely contain data from the
same partition.
During final join with limit, key duplicates appears in response, that fails
assertion where unique values are expected.
Backups are set to 0 for current cache. Does it mean that entities are
incorrectly distributed over partitions/nodes?
This part needs extra investigations.
Still no direct links found with this issue and {{limits}} introduced earlier.
CC [~amashenkov], [~Pavlukhin]
> Some Text Queries return repeated results
> -
>
> Key: IGNITE-12401
> URL: https://issues.apache.org/jira/browse/IGNITE-12401
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.8
>Reporter: Ilya Kasnacheev
>Assignee: Yuriy Shuliha
>Priority: Critical
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It came to my attention while checking for Range Queries support that we
> don't actually check that found query results are the correct ones.
> We were checking that we got some results, but not whether they were expected.
> And voila, it turns out that Range Queries examples, as well as some other
> test cases, will readily fail when run with such checks! A query will return
> same value repeatedly, e.g. range query will return the "1" record twice, and
> limited text query will return "14" record twice.
> It didn't really occur on non-range queries before the introduction of limits.
> I think we should not ship broken limit queries. Maybe also fix range
> queries, if it's hard let's @Ignore them for now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12401) Some Text Queries return repeated results
[
https://issues.apache.org/jira/browse/IGNITE-12401?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16984525#comment-16984525
]
Ignite TC Bot commented on IGNITE-12401:
{panel:title=Branch: [pull/7083/head] Base: [master] : Possible Blockers
(6)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Queries 1{color} [[tests
6|https://ci.ignite.apache.org/viewLog.html?buildId=4799634]]
* IgniteBinaryCacheQueryTestSuite:
GridCacheFullTextQuerySelfTest.testTextQueryWithRange - History for base branch
is absent.
* IgniteBinaryCacheQueryTestSuite:
GridCacheFullTextQuerySelfTest.testTextQueryWithConditionAndRange - History for
base branch is absent.
* IgniteBinaryCacheQueryTestSuite:
GridCacheFullTextQuerySelfTest.testTextQueryWithField - Test has low fail rate
in base branch 0,0% and is not flaky
* IgniteBinaryCacheQueryTestSuite: GridCacheFullTextQuerySelfTest.testTextQuery
- Test has low fail rate in base branch 0,0% and is not flaky
* IgniteBinaryCacheQueryTestSuite:
GridCacheFullTextQuerySelfTest.testTextQueryLimitedConcurrent - Test has low
fail rate in base branch 0,0% and is not flaky
* IgniteBinaryCacheQueryTestSuite:
GridCacheFullTextQuerySelfTest.testTextQueryWithFieldLimited - Test has low
fail rate in base branch 0,0% and is not flaky
{panel}
[TeamCity *--> Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4799658&buildTypeId=IgniteTests24Java8_RunAll]
> Some Text Queries return repeated results
> -
>
> Key: IGNITE-12401
> URL: https://issues.apache.org/jira/browse/IGNITE-12401
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Affects Versions: 2.8
>Reporter: Ilya Kasnacheev
>Priority: Critical
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It came to my attention while checking for Range Queries support that we
> don't actually check that found query results are the correct ones.
> We were checking that we got some results, but not whether they were expected.
> And voila, it turns out that Range Queries examples, as well as some other
> test cases, will readily fail when run with such checks! A query will return
> same value repeatedly, e.g. range query will return the "1" record twice, and
> limited text query will return "14" record twice.
> It didn't really occur on non-range queries before the introduction of limits.
> I think we should not ship broken limit queries. Maybe also fix range
> queries, if it's hard let's @Ignore them for now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
