[jira] [Updated] (HIVE-12788) Setting hive.optimize.union.remove to TRUE will break UNION ALL with aggregate functions
[
https://issues.apache.org/jira/browse/HIVE-12788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-12788:
---
Fix Version/s: (was: 2.1.0)
(was: 1.3.0)
> Setting hive.optimize.union.remove to TRUE will break UNION ALL with
> aggregate functions
>
>
> Key: HIVE-12788
> URL: https://issues.apache.org/jira/browse/HIVE-12788
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.1
>Reporter: Eric Lin
>Assignee: Chaoyu Tang
> Fix For: 2.0.0
>
> Attachments: HIVE-12788-branch-1.patch, HIVE-12788.1.patch,
> HIVE-12788.patch
>
>
> See the test case below:
> {code}
> 0: jdbc:hive2://localhost:1/default> create table test (a int);
> 0: jdbc:hive2://localhost:1/default> insert overwrite table test values
> (1);
> 0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove=true;
> No rows affected (0.01 seconds)
> 0: jdbc:hive2://localhost:1/default> set
> hive.mapred.supports.subdirectories=true;
> No rows affected (0.007 seconds)
> 0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
> SELECT COUNT(1) FROM test;
> +--+--+
> | _u1._c0 |
> +--+--+
> +--+--+
> {code}
> UNION ALL without COUNT function will work as expected:
> {code}
> 0: jdbc:hive2://localhost:1/default> select * from test UNION ALL SELECT
> * FROM test;
> ++--+
> | _u1.a |
> ++--+
> | 1 |
> | 1 |
> ++--+
> {code}
> Run the same query without setting hive.mapred.supports.subdirectories and
> hive.optimize.union.remove to true will give correct result:
> {code}
> 0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove;
> +---+--+
> |set|
> +---+--+
> | hive.optimize.union.remove=false |
> +---+--+
> 0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
> SELECT COUNT(1) FROM test;
> +--+--+
> | _u1._c0 |
> +--+--+
> | 1|
> | 1|
> +--+--+
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12788) Setting hive.optimize.union.remove to TRUE will break UNION ALL with aggregate functions
[
https://issues.apache.org/jira/browse/HIVE-12788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chaoyu Tang updated HIVE-12788:
---
Attachment: HIVE-12788-branch-1.patch
> Setting hive.optimize.union.remove to TRUE will break UNION ALL with
> aggregate functions
>
>
> Key: HIVE-12788
> URL: https://issues.apache.org/jira/browse/HIVE-12788
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.1
>Reporter: Eric Lin
>Assignee: Chaoyu Tang
> Attachments: HIVE-12788-branch-1.patch, HIVE-12788.1.patch,
> HIVE-12788.patch
>
>
> See the test case below:
> {code}
> 0: jdbc:hive2://localhost:1/default> create table test (a int);
> 0: jdbc:hive2://localhost:1/default> insert overwrite table test values
> (1);
> 0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove=true;
> No rows affected (0.01 seconds)
> 0: jdbc:hive2://localhost:1/default> set
> hive.mapred.supports.subdirectories=true;
> No rows affected (0.007 seconds)
> 0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
> SELECT COUNT(1) FROM test;
> +--+--+
> | _u1._c0 |
> +--+--+
> +--+--+
> {code}
> UNION ALL without COUNT function will work as expected:
> {code}
> 0: jdbc:hive2://localhost:1/default> select * from test UNION ALL SELECT
> * FROM test;
> ++--+
> | _u1.a |
> ++--+
> | 1 |
> | 1 |
> ++--+
> {code}
> Run the same query without setting hive.mapred.supports.subdirectories and
> hive.optimize.union.remove to true will give correct result:
> {code}
> 0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove;
> +---+--+
> |set|
> +---+--+
> | hive.optimize.union.remove=false |
> +---+--+
> 0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
> SELECT COUNT(1) FROM test;
> +--+--+
> | _u1._c0 |
> +--+--+
> | 1|
> | 1|
> +--+--+
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12788) Setting hive.optimize.union.remove to TRUE will break UNION ALL with aggregate functions
[
https://issues.apache.org/jira/browse/HIVE-12788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chaoyu Tang updated HIVE-12788:
---
Attachment: HIVE-12788.1.patch
Fix the failed tests.
> Setting hive.optimize.union.remove to TRUE will break UNION ALL with
> aggregate functions
>
>
> Key: HIVE-12788
> URL: https://issues.apache.org/jira/browse/HIVE-12788
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.1
>Reporter: Eric Lin
>Assignee: Chaoyu Tang
> Attachments: HIVE-12788.1.patch, HIVE-12788.patch
>
>
> See the test case below:
> {code}
> 0: jdbc:hive2://localhost:1/default> create table test (a int);
> 0: jdbc:hive2://localhost:1/default> insert overwrite table test values
> (1);
> 0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove=true;
> No rows affected (0.01 seconds)
> 0: jdbc:hive2://localhost:1/default> set
> hive.mapred.supports.subdirectories=true;
> No rows affected (0.007 seconds)
> 0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
> SELECT COUNT(1) FROM test;
> +--+--+
> | _u1._c0 |
> +--+--+
> +--+--+
> {code}
> UNION ALL without COUNT function will work as expected:
> {code}
> 0: jdbc:hive2://localhost:1/default> select * from test UNION ALL SELECT
> * FROM test;
> ++--+
> | _u1.a |
> ++--+
> | 1 |
> | 1 |
> ++--+
> {code}
> Run the same query without setting hive.mapred.supports.subdirectories and
> hive.optimize.union.remove to true will give correct result:
> {code}
> 0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove;
> +---+--+
> |set|
> +---+--+
> | hive.optimize.union.remove=false |
> +---+--+
> 0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
> SELECT COUNT(1) FROM test;
> +--+--+
> | _u1._c0 |
> +--+--+
> | 1|
> | 1|
> +--+--+
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12788) Setting hive.optimize.union.remove to TRUE will break UNION ALL with aggregate functions
[
https://issues.apache.org/jira/browse/HIVE-12788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chaoyu Tang updated HIVE-12788:
---
Attachment: HIVE-12788.patch
The patch adds StatsOptimizator support to union with aggregate function.
> Setting hive.optimize.union.remove to TRUE will break UNION ALL with
> aggregate functions
>
>
> Key: HIVE-12788
> URL: https://issues.apache.org/jira/browse/HIVE-12788
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.1
>Reporter: Eric Lin
>Assignee: Chaoyu Tang
> Attachments: HIVE-12788.patch
>
>
> See the test case below:
> {code}
> 0: jdbc:hive2://localhost:1/default> create table test (a int);
> 0: jdbc:hive2://localhost:1/default> insert overwrite table test values
> (1);
> 0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove=true;
> No rows affected (0.01 seconds)
> 0: jdbc:hive2://localhost:1/default> set
> hive.mapred.supports.subdirectories=true;
> No rows affected (0.007 seconds)
> 0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
> SELECT COUNT(1) FROM test;
> +--+--+
> | _u1._c0 |
> +--+--+
> +--+--+
> {code}
> UNION ALL without COUNT function will work as expected:
> {code}
> 0: jdbc:hive2://localhost:1/default> select * from test UNION ALL SELECT
> * FROM test;
> ++--+
> | _u1.a |
> ++--+
> | 1 |
> | 1 |
> ++--+
> {code}
> Run the same query without setting hive.mapred.supports.subdirectories and
> hive.optimize.union.remove to true will give correct result:
> {code}
> 0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove;
> +---+--+
> |set|
> +---+--+
> | hive.optimize.union.remove=false |
> +---+--+
> 0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
> SELECT COUNT(1) FROM test;
> +--+--+
> | _u1._c0 |
> +--+--+
> | 1|
> | 1|
> +--+--+
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12788) Setting hive.optimize.union.remove to TRUE will break UNION ALL with aggregate functions
[
https://issues.apache.org/jira/browse/HIVE-12788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chaoyu Tang updated HIVE-12788:
---
Attachment: (was: HIVE-12788.patch)
> Setting hive.optimize.union.remove to TRUE will break UNION ALL with
> aggregate functions
>
>
> Key: HIVE-12788
> URL: https://issues.apache.org/jira/browse/HIVE-12788
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.1
>Reporter: Eric Lin
>Assignee: Chaoyu Tang
> Attachments: HIVE-12788.patch
>
>
> See the test case below:
> {code}
> 0: jdbc:hive2://localhost:1/default> create table test (a int);
> 0: jdbc:hive2://localhost:1/default> insert overwrite table test values
> (1);
> 0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove=true;
> No rows affected (0.01 seconds)
> 0: jdbc:hive2://localhost:1/default> set
> hive.mapred.supports.subdirectories=true;
> No rows affected (0.007 seconds)
> 0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
> SELECT COUNT(1) FROM test;
> +--+--+
> | _u1._c0 |
> +--+--+
> +--+--+
> {code}
> UNION ALL without COUNT function will work as expected:
> {code}
> 0: jdbc:hive2://localhost:1/default> select * from test UNION ALL SELECT
> * FROM test;
> ++--+
> | _u1.a |
> ++--+
> | 1 |
> | 1 |
> ++--+
> {code}
> Run the same query without setting hive.mapred.supports.subdirectories and
> hive.optimize.union.remove to true will give correct result:
> {code}
> 0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove;
> +---+--+
> |set|
> +---+--+
> | hive.optimize.union.remove=false |
> +---+--+
> 0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
> SELECT COUNT(1) FROM test;
> +--+--+
> | _u1._c0 |
> +--+--+
> | 1|
> | 1|
> +--+--+
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12788) Setting hive.optimize.union.remove to TRUE will break UNION ALL with aggregate functions
[
https://issues.apache.org/jira/browse/HIVE-12788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chaoyu Tang updated HIVE-12788:
---
Attachment: HIVE-12788.patch
> Setting hive.optimize.union.remove to TRUE will break UNION ALL with
> aggregate functions
>
>
> Key: HIVE-12788
> URL: https://issues.apache.org/jira/browse/HIVE-12788
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.1
>Reporter: Eric Lin
>Assignee: Chaoyu Tang
> Attachments: HIVE-12788.patch
>
>
> See the test case below:
> {code}
> 0: jdbc:hive2://localhost:1/default> create table test (a int);
> 0: jdbc:hive2://localhost:1/default> insert overwrite table test values
> (1);
> 0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove=true;
> No rows affected (0.01 seconds)
> 0: jdbc:hive2://localhost:1/default> set
> hive.mapred.supports.subdirectories=true;
> No rows affected (0.007 seconds)
> 0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
> SELECT COUNT(1) FROM test;
> +--+--+
> | _u1._c0 |
> +--+--+
> +--+--+
> {code}
> UNION ALL without COUNT function will work as expected:
> {code}
> 0: jdbc:hive2://localhost:1/default> select * from test UNION ALL SELECT
> * FROM test;
> ++--+
> | _u1.a |
> ++--+
> | 1 |
> | 1 |
> ++--+
> {code}
> Run the same query without setting hive.mapred.supports.subdirectories and
> hive.optimize.union.remove to true will give correct result:
> {code}
> 0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove;
> +---+--+
> |set|
> +---+--+
> | hive.optimize.union.remove=false |
> +---+--+
> 0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
> SELECT COUNT(1) FROM test;
> +--+--+
> | _u1._c0 |
> +--+--+
> | 1|
> | 1|
> +--+--+
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12788) Setting hive.optimize.union.remove to TRUE will break UNION ALL with aggregate functions
[
https://issues.apache.org/jira/browse/HIVE-12788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Lin updated HIVE-12788:
Description:
See the test case below:
{code}
0: jdbc:hive2://localhost:1/default> create table test (a int);
0: jdbc:hive2://localhost:1/default> insert overwrite table test values (1);
0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove=true;
No rows affected (0.01 seconds)
0: jdbc:hive2://localhost:1/default> set
hive.mapred.supports.subdirectories=true;
No rows affected (0.007 seconds)
0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
SELECT COUNT(1) FROM test;
+--+--+
| _u1._c0 |
+--+--+
+--+--+
{code}
Run the same query without setting hive.mapred.supports.subdirectories and
hive.optimize.union.remove to true will give correct result:
{code}
0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
SELECT COUNT(1) FROM test;
+--+--+
| _u1._c0 |
+--+--+
| 1|
| 1|
+--+--+
{code}
UNION ALL without COUNT function will work as expected:
{code}
0: jdbc:hive2://localhost:1/default> select * from test UNION ALL SELECT *
FROM test;
++--+
| _u1.a |
++--+
| 1 |
| 1 |
++--+
{code}
was:
See the test case below:
{code}
0: jdbc:hive2://localhost:1/default> create table test (a int);
0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove=true;
No rows affected (0.01 seconds)
0: jdbc:hive2://localhost:1/default> set
hive.mapred.supports.subdirectories=true;
No rows affected (0.007 seconds)
0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
SELECT COUNT(1) FROM test;
+--+--+
| _u1._c0 |
+--+--+
+--+--+
{code}
Run the same query without setting hive.mapred.supports.subdirectories and
hive.optimize.union.remove to true will give correct result:
{code}
0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
SELECT COUNT(1) FROM test;
+--+--+
| _u1._c0 |
+--+--+
| 1|
| 1|
+--+--+
{code}
UNION ALL without COUNT function will work as expected:
{code}
0: jdbc:hive2://localhost:1/default> select * from test UNION ALL SELECT *
FROM test;
++--+
| _u1.a |
++--+
| 1 |
| 1 |
++--+
{code}
> Setting hive.optimize.union.remove to TRUE will break UNION ALL with
> aggregate functions
>
>
> Key: HIVE-12788
> URL: https://issues.apache.org/jira/browse/HIVE-12788
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.1
>Reporter: Eric Lin
>
> See the test case below:
> {code}
> 0: jdbc:hive2://localhost:1/default> create table test (a int);
> 0: jdbc:hive2://localhost:1/default> insert overwrite table test values
> (1);
> 0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove=true;
> No rows affected (0.01 seconds)
> 0: jdbc:hive2://localhost:1/default> set
> hive.mapred.supports.subdirectories=true;
> No rows affected (0.007 seconds)
> 0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
> SELECT COUNT(1) FROM test;
> +--+--+
> | _u1._c0 |
> +--+--+
> +--+--+
> {code}
> Run the same query without setting hive.mapred.supports.subdirectories and
> hive.optimize.union.remove to true will give correct result:
> {code}
> 0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
> SELECT COUNT(1) FROM test;
> +--+--+
> | _u1._c0 |
> +--+--+
> | 1|
> | 1|
> +--+--+
> {code}
> UNION ALL without COUNT function will work as expected:
> {code}
> 0: jdbc:hive2://localhost:1/default> select * from test UNION ALL SELECT
> * FROM test;
> ++--+
> | _u1.a |
> ++--+
> | 1 |
> | 1 |
> ++--+
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-12788) Setting hive.optimize.union.remove to TRUE will break UNION ALL with aggregate functions
[
https://issues.apache.org/jira/browse/HIVE-12788?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Lin updated HIVE-12788:
Description:
See the test case below:
{code}
0: jdbc:hive2://localhost:1/default> create table test (a int);
0: jdbc:hive2://localhost:1/default> insert overwrite table test values (1);
0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove=true;
No rows affected (0.01 seconds)
0: jdbc:hive2://localhost:1/default> set
hive.mapred.supports.subdirectories=true;
No rows affected (0.007 seconds)
0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
SELECT COUNT(1) FROM test;
+--+--+
| _u1._c0 |
+--+--+
+--+--+
{code}
UNION ALL without COUNT function will work as expected:
{code}
0: jdbc:hive2://localhost:1/default> select * from test UNION ALL SELECT *
FROM test;
++--+
| _u1.a |
++--+
| 1 |
| 1 |
++--+
{code}
Run the same query without setting hive.mapred.supports.subdirectories and
hive.optimize.union.remove to true will give correct result:
{code}
0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove;
+---+--+
|set|
+---+--+
| hive.optimize.union.remove=false |
+---+--+
0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
SELECT COUNT(1) FROM test;
+--+--+
| _u1._c0 |
+--+--+
| 1|
| 1|
+--+--+
{code}
was:
See the test case below:
{code}
0: jdbc:hive2://localhost:1/default> create table test (a int);
0: jdbc:hive2://localhost:1/default> insert overwrite table test values (1);
0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove=true;
No rows affected (0.01 seconds)
0: jdbc:hive2://localhost:1/default> set
hive.mapred.supports.subdirectories=true;
No rows affected (0.007 seconds)
0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
SELECT COUNT(1) FROM test;
+--+--+
| _u1._c0 |
+--+--+
+--+--+
{code}
Run the same query without setting hive.mapred.supports.subdirectories and
hive.optimize.union.remove to true will give correct result:
{code}
0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
SELECT COUNT(1) FROM test;
+--+--+
| _u1._c0 |
+--+--+
| 1|
| 1|
+--+--+
{code}
UNION ALL without COUNT function will work as expected:
{code}
0: jdbc:hive2://localhost:1/default> select * from test UNION ALL SELECT *
FROM test;
++--+
| _u1.a |
++--+
| 1 |
| 1 |
++--+
{code}
> Setting hive.optimize.union.remove to TRUE will break UNION ALL with
> aggregate functions
>
>
> Key: HIVE-12788
> URL: https://issues.apache.org/jira/browse/HIVE-12788
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 1.1.1
>Reporter: Eric Lin
>
> See the test case below:
> {code}
> 0: jdbc:hive2://localhost:1/default> create table test (a int);
> 0: jdbc:hive2://localhost:1/default> insert overwrite table test values
> (1);
> 0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove=true;
> No rows affected (0.01 seconds)
> 0: jdbc:hive2://localhost:1/default> set
> hive.mapred.supports.subdirectories=true;
> No rows affected (0.007 seconds)
> 0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
> SELECT COUNT(1) FROM test;
> +--+--+
> | _u1._c0 |
> +--+--+
> +--+--+
> {code}
> UNION ALL without COUNT function will work as expected:
> {code}
> 0: jdbc:hive2://localhost:1/default> select * from test UNION ALL SELECT
> * FROM test;
> ++--+
> | _u1.a |
> ++--+
> | 1 |
> | 1 |
> ++--+
> {code}
> Run the same query without setting hive.mapred.supports.subdirectories and
> hive.optimize.union.remove to true will give correct result:
> {code}
> 0: jdbc:hive2://localhost:1/default> set hive.optimize.union.remove;
> +---+--+
> |set|
> +---+--+
> | hive.optimize.union.remove=false |
> +---+--+
> 0: jdbc:hive2://localhost:1/default> SELECT COUNT(1) FROM test UNION ALL
> SELECT COUNT(1) FROM test;
> +--+--+
> | _u1._c0 |
> +--+--+
> | 1|
> | 1|
> +--+--+
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
