[jira] [Commented] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205456#comment-16205456
]
Hadoop QA commented on PHOENIX-4283:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12892327/PHOENIX-4283-v1.patch
against master branch at commit ff8037ef103f73258115fc766bbe89c430a0.
ATTACHMENT ID: 12892327
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+String createQuery="CREATE TABLE "+tableName+" (a BIGINT NOT
NULL,c BIGINT NOT NULL CONSTRAINT PK PRIMARY KEY (a, c))";
+String updateQuery="UPSERT INTO "+tableName+"(a,c)
VALUES(444, 555)";
+String query="SELECT a FROM (SELECT a, c FROM "+tableName+" GROUP
BY a, c) GROUP BY a, c";
{color:red}-1 core tests{color}. The patch failed these unit tests:
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1543//testReport/
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1543//console
This message is automatically generated.
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4283-v3.patch, PHOENIX-4283.patch
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205446#comment-16205446
]
Ethan Wang edited comment on PHOENIX-4283 at 10/16/17 5:30 AM:
---
Thanks [~jamestaylor] I should've be more careful.
I will apply the patch when CI finishes.
was (Author: aertoria):
Thanks [~jamestaylor] I should've be more careful.
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4283-v3.patch, PHOENIX-4283.patch
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Wang updated PHOENIX-4283:
Attachment: (was: PHOENIX-4283-v2.patch)
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4283-v3.patch, PHOENIX-4283.patch
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Wang updated PHOENIX-4283:
Attachment: PHOENIX-4283-v3.patch
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4283-v3.patch, PHOENIX-4283.patch
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205446#comment-16205446
]
Ethan Wang commented on PHOENIX-4283:
-
Thanks [~jamestaylor] I should've be more careful.
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4283-v2.patch, PHOENIX-4283.patch
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205443#comment-16205443
]
James Taylor edited comment on PHOENIX-4283 at 10/16/17 5:22 AM:
-
One last nit: swap the argument order here so that the expected value is first
or the assertion message if it were to fail would be confusing.
{code}
assertEquals(rs.getLong(1),444L);
{code}
Then make sure your new test passes and commit to 4.12-HBase-0.98,
4.x-HBase-0.98, and master branches.
was (Author: jamestaylor):
One last nit: swap the argument order here so that the expected value is first
or the assertion message if it were to fail would be confusing.
Then commit to 4.12-HBase-0.98, 4.x-HBase-0.98, and master branches.
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4283-v2.patch, PHOENIX-4283.patch
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205443#comment-16205443
]
James Taylor commented on PHOENIX-4283:
---
One last nit: swap the argument order here so that the expected value is first
or the assertion message if it were to fail would be confusing.
Then commit to 4.12-HBase-0.98, 4.x-HBase-0.98, and master branches.
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4283-v2.patch, PHOENIX-4283.patch
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205438#comment-16205438
]
Ethan Wang edited comment on PHOENIX-4283 at 10/16/17 5:18 AM:
---
[~jamestaylor] thanks!
was (Author: aertoria):
[~jamestaylor] +1
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4283-v2.patch, PHOENIX-4283.patch
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205438#comment-16205438
]
Ethan Wang commented on PHOENIX-4283:
-
[~jamestaylor] +1
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4283-v2.patch, PHOENIX-4283.patch
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Wang updated PHOENIX-4283:
Attachment: PHOENIX-4283-v2.patch
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4283-v2.patch, PHOENIX-4283.patch
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Wang updated PHOENIX-4283:
Attachment: (was: PHOENIX-4283-v1.patch)
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4283.patch
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205433#comment-16205433
]
James Taylor commented on PHOENIX-4283:
---
Couple more minor nits:
- Get rid of the loop and just do this instead (since there should just be a
single row):
{code}
assertTrue(rs.next());
assertEquals(44L, rs.getLong(1));
assertFalse(rs.next());
{code}
- Use getLong instead of getString as shown above
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4283-v1.patch, PHOENIX-4283.patch
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Wang updated PHOENIX-4283:
Attachment: PHOENIX-4283-v1.patch
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4283-v1.patch, PHOENIX-4283.patch
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205420#comment-16205420
]
Ethan Wang commented on PHOENIX-4283:
-
[~jamestaylor] Make sense. I go prepare another patch. Thanks
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4283.patch
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205413#comment-16205413
]
James Taylor commented on PHOENIX-4283:
---
Thanks for the quick turnaround on this, [~aertoria]. The test failure does not
look related. One minor nit: add your new tests to AggregateIT instead of
GroupByIT, since the latter has parameterized tests based on adding indexes to
columns outside of the ones you're using (thus the same test would run multiple
times which just increases test run time).
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4283.patch
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205408#comment-16205408
]
Hadoop QA commented on PHOENIX-4283:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12892318/PHOENIX-4283.patch
against master branch at commit ff8037ef103f73258115fc766bbe89c430a0.
ATTACHMENT ID: 12892318
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+" where ORGANIZATION_ID = ? and ENTITY_ID = ? GROUP
BY X_LONG, A_FLOAT) GROUP BY X_LONG, A_FLOAT ";
+String createQuery="CREATE TABLE "+tableName+" (a BIGINT NOT
NULL,c BIGINT NOT NULL CONSTRAINT PK PRIMARY KEY (a, c))";
+String updateQuery="UPSERT INTO "+tableName+"(a,c)
VALUES(444, 555)";
+String query="SELECT a FROM (SELECT a, c FROM "+tableName+" GROUP
BY a, c) GROUP BY a, c";
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AlterMultiTenantTableWithViewsIT
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1542//testReport/
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1542//console
This message is automatically generated.
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4283.patch
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ethan Wang updated PHOENIX-4283:
Attachment: PHOENIX-4283.patch
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
> Attachments: PHOENIX-4283.patch
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205289#comment-16205289
]
James Taylor commented on PHOENIX-4283:
---
Yes, good find. We should only be executing this code if
actualType.isCoercible(TIMESTAMP).
{code}
// Decrease size of TIMESTAMP to size of LONG and continue coerce
if (ptr.getLength() > getByteSize()) {
ptr.set(ptr.get(), ptr.getOffset(), getByteSize());
}
{code}
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205268#comment-16205268
]
Ethan Wang edited comment on PHOENIX-4283 at 10/15/17 7:46 PM:
---
Yes, when cut off happens, the "actualType" is Decimal, "this" is BigInt. The
"actualType" is pass from RowKeyColumnExpression.fromType, which is Decimal in
the nested groupby case. So, as comparison, in RowKeyColumnExpression,
logic : type.coerceBytes(ptr, fromType);
*nested*: BIGINT.coerceBytes(ptr, DECIMAL);
*normal*: BIGINT.coerceBytes(ptr, BIGINT);
I think a issue may be that, in PLong, coerceBytes() is overriding *regardless*
the "actualType", before passing into super.coerceBytes(). Therefore, the cut
off get executed always.
{code} @Override
public void coerceBytes(ImmutableBytesWritable ptr, Object object,
PDataType actualType,
Integer maxLength, Integer scale, SortOrder actualModifier, Integer
desiredMaxLength, Integer desiredScale,
SortOrder expectedModifier) {
// Decrease size of TIMESTAMP to size of LONG and continue coerce
if (ptr.getLength() > getByteSize()) {
ptr.set(ptr.get(), ptr.getOffset(), getByteSize());
}
super.coerceBytes(ptr, object, actualType, maxLength, scale,
actualModifier, desiredMaxLength,
desiredScale, expectedModifier);
}
{code}
was (Author: aertoria):
Yes, when cut off happens, the "actualType" is Decimal, "this" is BigInt. The
"actualType" is pass from RowKeyColumnExpression.fromType, which is Decimal in
nested groupby. So, as comparison, in RowKeyColumnExpression,
logic : type.coerceBytes(ptr, fromType);
*nested*: BIGINT.coerceBytes(ptr, DECIMAL);
*normal*: BIGINT.coerceBytes(ptr, BIGINT);
I think the issue may be that, in PLong, coerceBytes() is override *regardless*
the "actualType", before passing into super.coerceBytes(). Therefore, the cut
off get executed always.
{code} @Override
public void coerceBytes(ImmutableBytesWritable ptr, Object object,
PDataType actualType,
Integer maxLength, Integer scale, SortOrder actualModifier, Integer
desiredMaxLength, Integer desiredScale,
SortOrder expectedModifier) {
// Decrease size of TIMESTAMP to size of LONG and continue coerce
if (ptr.getLength() > getByteSize()) {
ptr.set(ptr.get(), ptr.getOffset(), getByteSize());
}
super.coerceBytes(ptr, object, actualType, maxLength, scale,
actualModifier, desiredMaxLength,
desiredScale, expectedModifier);
}
{code}
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205268#comment-16205268
]
Ethan Wang commented on PHOENIX-4283:
-
Yes, when cut off happens, the "actualType" is Decimal, "this" is BigInt. The
"actualType" is pass from RowKeyColumnExpression.fromType, which is Decimal in
nested groupby. So, as comparison, in RowKeyColumnExpression,
logic : type.coerceBytes(ptr, fromType);
*nested*: BIGINT.coerceBytes(ptr, DECIMAL);
*normal*: BIGINT.coerceBytes(ptr, BIGINT);
I think the issue may be that, in PLong, coerceBytes() is override *regardless*
the "actualType", before passing into super.coerceBytes(). Therefore, the cut
off get executed always.
{code} @Override
public void coerceBytes(ImmutableBytesWritable ptr, Object object,
PDataType actualType,
Integer maxLength, Integer scale, SortOrder actualModifier, Integer
desiredMaxLength, Integer desiredScale,
SortOrder expectedModifier) {
// Decrease size of TIMESTAMP to size of LONG and continue coerce
if (ptr.getLength() > getByteSize()) {
ptr.set(ptr.get(), ptr.getOffset(), getByteSize());
}
super.coerceBytes(ptr, object, actualType, maxLength, scale,
actualModifier, desiredMaxLength,
desiredScale, expectedModifier);
}
{code}
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205199#comment-16205199
]
James Taylor edited comment on PHOENIX-4283 at 10/15/17 5:00 PM:
-
bq. a byte array operation mistakenly cutting a serialized DECIMAL
ImmutableBytesWritable from 11B to 8B, dropping 3B in the tail, which
responsible for 6 last digits data loss. This cut-off happened at
PLong.coerceBytes() when it try to "Decrease size of TIMESTAMP to size of LONG
and continue coerce".
Need to figure out why it’s decreasing size here. Sounds like Phoenix thinks
the underlying type is a long instead of a decimal (based on the truncation to
8 bytes, the size of a long). Does it have decimal as the actualType in the
method? I think if you compare the non nested coercions (which are correct) to
the nested coercions, you may be able to spot this particular issue.
was (Author: jamestaylor):
bq. a byte array operation mistakenly cutting a serialized DECIMAL
ImmutableBytesWritable from 11B to 8B, dropping 3B in the tail, which
responsible for 6 last digits data loss. This cut-off happened at
PLong.coerceBytes() when it try to "Decrease size of TIMESTAMP to size of LONG
and continue coerce".
Need to figure out why it’s decreasing size here. Sounds like Phoenix thinks
the underlying type is a long instead of a decimal (based on the truncation to
8 bytes, the size of a long). Does it have decimal as the actualType in the
method?
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205199#comment-16205199
]
James Taylor commented on PHOENIX-4283:
---
bq. a byte array operation mistakenly cutting a serialized DECIMAL
ImmutableBytesWritable from 11B to 8B, dropping 3B in the tail, which
responsible for 6 last digits data loss. This cut-off happened at
PLong.coerceBytes() when it try to "Decrease size of TIMESTAMP to size of LONG
and continue coerce".
Need to figure out why it’s decreasing size here. Sounds like Phoenix thinks
the underlying type is a long instead of a decimal (based on the truncation to
8 bytes, the size of a long). Does it have decimal as the actualType in the
method?
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4283) Group By statement truncating BIGINTs
[
https://issues.apache.org/jira/browse/PHOENIX-4283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205073#comment-16205073
]
Ethan Wang commented on PHOENIX-4283:
-
Thanks [~jamestaylor] for the direction. It turns out, as you directed, between
rs.next() and rs.getValue(), a byte array operation mistakenly cutting a
serialized DECIMAL ImmutableBytesWritable from 11B to 8B, dropping 3B in the
tail, which responsible for 6 last digits data loss. This cut-off happened at
PLong.coerceBytes() when it try to "Decrease size of TIMESTAMP to size of LONG
and continue coerce".
As for details, here is what happened (correct me). In the case of a nested
group by, at very beginning the subquery (select A, C from table grouby A, C)
as a whole is returned from server as a tuple. Here as "A" is a
CoercedExpression, it first gets converted it to its child,
ProjectedColumnExpression. This converts A from BIGINT to DECIMAL which is then
serialized into a byte array("so that it is binary comparable"). That's when
the 8B BigDecimal becomes a 11B byte array.
This byte array gets passed along to the moment when user calls rs.getValue,
"A" as a RowKeyColoumnExpression gets evaluated, that's where this serialized
DECIMAL is converted back to BIGINT. during this converting, PLong overrides
coerceBytes() and injected a cut-off there. In this cut-off, anything that is
larger than Bytes.SIZEOF_LONG (which is 8B) will be reduce to first 8B.
Therefore the last 3B is lost. So in summary:
Server -> BIGINT 8B -> BIGDECIMAL byte[] 11B -> BIGINT 8B
Let me know your thoughts [~jamestaylor] then I'd like to discuss a thought for
a patch.
> Group By statement truncating BIGINTs
> -
>
> Key: PHOENIX-4283
> URL: https://issues.apache.org/jira/browse/PHOENIX-4283
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.11.0
>Reporter: Steven Sadowski
>Assignee: Ethan Wang
> Fix For: 4.12.1
>
>
> *Versions:*
> Phoenix 4.11.0
> HBase: 1.3.1
> (Amazon EMR: 5.8.0)
> *Steps to reproduce:*
> 1. From the `sqlline-thin.py` client setup the following table:
> {code:sql}
> CREATE TABLE test_table (
> a BIGINT NOT NULL,
> c BIGINT NOT NULL
> CONSTRAINT PK PRIMARY KEY (a, c)
> );
> UPSERT INTO test_table(a,c) VALUES(444, 555);
> SELECT a FROM (SELECT a, c FROM test_table GROUP BY a, c) GROUP BY a, c;
> {code}
> *Expected Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 444 |
> +--+
> {code}
> *Actual Result:*
> {code:sql}
> +--+
> | A |
> +--+
> | 400 |
> +--+
> {code}
> *Comments:*
> Having the two Group By statements together seems to truncate the last 6 or
> so digits of the final result. Removing the outer (or either) group by will
> produce the correct result.
> Please fix the Group by statement to not truncate the outer result's value.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
