[jira] [Commented] (ASTERIXDB-2372) Providing a float value predicate to an integer primary index does not work as expected.
[
https://issues.apache.org/jira/browse/ASTERIXDB-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16464468#comment-16464468
]
Dmitry Lychagin commented on ASTERIXDB-2372:
For non-enforced index we should continue using cast-lax function as we're
doing right now.
> Providing a float value predicate to an integer primary index does not work
> as expected.
>
>
> Key: ASTERIXDB-2372
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2372
> Project: Apache AsterixDB
> Issue Type: Bug
>Reporter: Taewoo Kim
>Assignee: Taewoo Kim
>Priority: Critical
>
> If we have an integer primary index and feed a float value predicate that is
> not an integer such as 1.3, the search result is not correct.
>
> The DDL and DML
> {code:java}
> drop dataverse test if exists;
> create dataverse test;
> use test;
> create type MyRecord as closed {
> id: int64
> };
> create dataset MyData(MyRecord) primary key id;
> insert into MyData({"id":1});
> insert into MyData({"id":2});
> select * from MyData where id = 1.3;{code}
>
> The result should be empty. But, it returns 1 and 2 as the result.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ASTERIXDB-2372) Providing a float value predicate to an integer primary index does not work as expected.
[
https://issues.apache.org/jira/browse/ASTERIXDB-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463198#comment-16463198
]
Xikui Wang commented on ASTERIXDB-2372:
---
I'm trying to work on this problem and have an issue with the non-force index.
In the case where we have a non-enforced index with different types, let's say
the record is double and the index is int64, return NULL will cause an empty
result. An example is a query in the test case `btree-index-04`. Any thoughts
on this?
> Providing a float value predicate to an integer primary index does not work
> as expected.
>
>
> Key: ASTERIXDB-2372
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2372
> Project: Apache AsterixDB
> Issue Type: Bug
>Reporter: Taewoo Kim
>Assignee: Taewoo Kim
>Priority: Critical
>
> If we have an integer primary index and feed a float value predicate that is
> not an integer such as 1.3, the search result is not correct.
>
> The DDL and DML
> {code:java}
> drop dataverse test if exists;
> create dataverse test;
> use test;
> create type MyRecord as closed {
> id: int64
> };
> create dataset MyData(MyRecord) primary key id;
> insert into MyData({"id":1});
> insert into MyData({"id":2});
> select * from MyData where id = 1.3;{code}
>
> The result should be empty. But, it returns 1 and 2 as the result.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ASTERIXDB-2372) Providing a float value predicate to an integer primary index does not work as expected.
[
https://issues.apache.org/jira/browse/ASTERIXDB-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455587#comment-16455587
]
Dmitry Lychagin commented on ASTERIXDB-2372:
May be we should slightly reconsider the floor/ceil approach when it comes to
equality comparison, since we also want to support cases when the probe type is
not known at compile time.
For equality comparison, can we first convert it to the indexed type then
figure out the range and inclusiveness? (we somewhat do it with floor/ceil, but
let me explain further)
That conversion should return NULL if some information will be lost, like in
case #2 above. The inclusiveness option would still be maintained (<=, >=), but
it would not matter because the Btree-search runtime should be modified to
return empty result if one of the range values is NULL (if it doesn't do that
already).
So the rewrite for equality would conceptually be
id = X (let's say 'id' is closed field of type int32 or indexed with enforced
index as int32)
into
id >= convert(X, int32) and id <= convert(X, int32)
'convert' returns NULL in cases where there's information loss, for example
- non-integral floating point value (1.3), or
- int64 which is out of range for int32.
In these cases we know that no indexed value (int32) can possibly match the
probe value therefore the result would be empty, so we don't even need to
physically scan the index
With this approach X does not have to be a constant, but could be any
expression for which the type might not even be known at compile time.
We should figure out rules for other comparison operators. There the conversion
to the indexed type will be permitted even if there's an information loss.
> Providing a float value predicate to an integer primary index does not work
> as expected.
>
>
> Key: ASTERIXDB-2372
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2372
> Project: Apache AsterixDB
> Issue Type: Bug
>Reporter: Taewoo Kim
>Assignee: Taewoo Kim
>Priority: Critical
>
> If we have an integer primary index and feed a float value predicate that is
> not an integer such as 1.3, the search result is not correct.
>
> The DDL and DML
> {code:java}
> drop dataverse test if exists;
> create dataverse test;
> use test;
> create type MyRecord as closed {
> id: int64
> };
> create dataset MyData(MyRecord) primary key id;
> insert into MyData({"id":1});
> insert into MyData({"id":2});
> select * from MyData where id = 1.3;{code}
>
> The result should be empty. But, it returns 1 and 2 as the result.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ASTERIXDB-2372) Providing a float value predicate to an integer primary index does not work as expected.
[
https://issues.apache.org/jira/browse/ASTERIXDB-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454858#comment-16454858
]
Taewoo Kim commented on ASTERIXDB-2372:
---
id = 1.0 is a valid predicate for an equality search on an integer index. So,
id >= 1 and id <=1 will return one result (id = 1).
But, id = 1.3 is not a valid predicate. However, id >=1 and id <=2 return two
results (id =1 and id = 2). Thus, we need to remove the inclusiveness option.
So, in short, if ceil( x ) = floor ( x ), we maintain the current framework (id
>= floor( x ) and id <= ceil ( x )). If not, we remove the inclusiveness option.
> Providing a float value predicate to an integer primary index does not work
> as expected.
>
>
> Key: ASTERIXDB-2372
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2372
> Project: Apache AsterixDB
> Issue Type: Bug
>Reporter: Taewoo Kim
>Assignee: Taewoo Kim
>Priority: Critical
>
> If we have an integer primary index and feed a float value predicate that is
> not an integer such as 1.3, the search result is not correct.
>
> The DDL and DML
> {code:java}
> drop dataverse test if exists;
> create dataverse test;
> use test;
> create type MyRecord as closed {
> id: int64
> };
> create dataset MyData(MyRecord) primary key id;
> insert into MyData({"id":1});
> insert into MyData({"id":2});
> select * from MyData where id = 1.3;{code}
>
> The result should be empty. But, it returns 1 and 2 as the result.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ASTERIXDB-2372) Providing a float value predicate to an integer primary index does not work as expected.
[
https://issues.apache.org/jira/browse/ASTERIXDB-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454852#comment-16454852
]
Michael J. Carey commented on ASTERIXDB-2372:
-
I don't understand (see below) but have to do some other stuff this afternoon
unfortunately. Here's my confusion - in my world:
{noformat}
>>> math.floor(1)
1
>>> math.ceil(1)
1
{noformat}
Thus I don't understand the Case 1 description.
> Providing a float value predicate to an integer primary index does not work
> as expected.
>
>
> Key: ASTERIXDB-2372
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2372
> Project: Apache AsterixDB
> Issue Type: Bug
>Reporter: Taewoo Kim
>Assignee: Taewoo Kim
>Priority: Critical
>
> If we have an integer primary index and feed a float value predicate that is
> not an integer such as 1.3, the search result is not correct.
>
> The DDL and DML
> {code:java}
> drop dataverse test if exists;
> create dataverse test;
> use test;
> create type MyRecord as closed {
> id: int64
> };
> create dataset MyData(MyRecord) primary key id;
> insert into MyData({"id":1});
> insert into MyData({"id":2});
> select * from MyData where id = 1.3;{code}
>
> The result should be empty. But, it returns 1 and 2 as the result.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ASTERIXDB-2372) Providing a float value predicate to an integer primary index does not work as expected.
[
https://issues.apache.org/jira/browse/ASTERIXDB-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454837#comment-16454837
]
Taewoo Kim commented on ASTERIXDB-2372:
---
It's case #1 - we maintain the inclusiveness option. (id >=1 and id <= 1)
> Providing a float value predicate to an integer primary index does not work
> as expected.
>
>
> Key: ASTERIXDB-2372
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2372
> Project: Apache AsterixDB
> Issue Type: Bug
>Reporter: Taewoo Kim
>Assignee: Taewoo Kim
>Priority: Critical
>
> If we have an integer primary index and feed a float value predicate that is
> not an integer such as 1.3, the search result is not correct.
>
> The DDL and DML
> {code:java}
> drop dataverse test if exists;
> create dataverse test;
> use test;
> create type MyRecord as closed {
> id: int64
> };
> create dataset MyData(MyRecord) primary key id;
> insert into MyData({"id":1});
> insert into MyData({"id":2});
> select * from MyData where id = 1.3;{code}
>
> The result should be empty. But, it returns 1 and 2 as the result.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ASTERIXDB-2372) Providing a float value predicate to an integer primary index does not work as expected.
[
https://issues.apache.org/jira/browse/ASTERIXDB-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454832#comment-16454832
]
Xikui Wang commented on ASTERIXDB-2372:
---
Is "select x.id = 1.0" covered in this case?
> Providing a float value predicate to an integer primary index does not work
> as expected.
>
>
> Key: ASTERIXDB-2372
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2372
> Project: Apache AsterixDB
> Issue Type: Bug
>Reporter: Taewoo Kim
>Assignee: Taewoo Kim
>Priority: Critical
>
> If we have an integer primary index and feed a float value predicate that is
> not an integer such as 1.3, the search result is not correct.
>
> The DDL and DML
> {code:java}
> drop dataverse test if exists;
> create dataverse test;
> use test;
> create type MyRecord as closed {
> id: int64
> };
> create dataset MyData(MyRecord) primary key id;
> insert into MyData({"id":1});
> insert into MyData({"id":2});
> select * from MyData where id = 1.3;{code}
>
> The result should be empty. But, it returns 1 and 2 as the result.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (ASTERIXDB-2372) Providing a float value predicate to an integer primary index does not work as expected.
[
https://issues.apache.org/jira/browse/ASTERIXDB-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454783#comment-16454783
]
Taewoo Kim commented on ASTERIXDB-2372:
---
Fact 1. Btree-search is a range search. We need to provide a low value (L) and
a high value (H) and their inclusiveness. For example, a genral B-tree search
query is "id > 1 and id <= 5". In this case, the low value is 1 and the high
value is 5. Also, the low value should not be included in the result and the
high value should be included. For an equality-condition search such as "id =
1" is translated as "id >=1 and id <= 1" to get only one value.
Fact 2. For the equality-condition search, if a float or double value is
provided to an integer index, we transform it to two values - floor(x) and
ceil(x). The intuition behind this is that the search only makes sense when
floor(x) is equal to ceil(x). This is only true when a float or double value
only contains the integral part (e.g., 1.0, 8.0). For other values, ceil(x) is
not equal to floor(x).
The cause: the cause is that "id = 1.3" will be translated as "id >=1 and id
<=2" because floor(1.3) = 1 and ceil(1.3) = 2. So, two tuples are returned
since AsterixDB removed the SELECT operator that checks "id = 1.3" by providing
that condition to an index-search.
The solution is removing the inclusiveness option when a float value is
translated as two different values when applying ceil() and floor().
Case #1:
query: id = 1.0 (floor(1) = 1 and ceil(1) = 2)
internal: id >=1 and id <= 1 : maintained the inclusiveness
Case #2:
query: id = 1.3 (floor(1.3) = 1 and ceil(1.3) = 2)
internal: id > 1 and id < 2 : removed the inclusiveness
> Providing a float value predicate to an integer primary index does not work
> as expected.
>
>
> Key: ASTERIXDB-2372
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-2372
> Project: Apache AsterixDB
> Issue Type: Bug
>Reporter: Taewoo Kim
>Assignee: Taewoo Kim
>Priority: Critical
>
> If we have an integer primary index and feed a float value predicate that is
> not an integer such as 1.3, the search result is not correct.
>
> The DDL and DML
> {code:java}
> drop dataverse test if exists;
> create dataverse test;
> use test;
> create type MyRecord as closed {
> id: int64
> };
> create dataset MyData(MyRecord) primary key id;
> insert into MyData({"id":1});
> insert into MyData({"id":2});
> select * from MyData where id = 1.3;{code}
>
> The result should be empty. But, it returns 1 and 2 as the result.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
