[jira] [Updated] (HIVE-3674) test case TestParse broken after recent checkin
[ https://issues.apache.org/jira/browse/HIVE-3674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3674: - Resolution: Fixed Fix Version/s: 0.10.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk. Thanks Sambavi! test case TestParse broken after recent checkin --- Key: HIVE-3674 URL: https://issues.apache.org/jira/browse/HIVE-3674 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.9.0 Reporter: Sambavi Muthukrishnan Assignee: Sambavi Muthukrishnan Fix For: 0.10.0 Attachments: TestParseFix.1.patch The below test cases fail after running svn up on my clean checkout. org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby1 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby2 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby3 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby4 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby5 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby6 The build on Nov 2 shows this issue as well. https://builds.apache.org/job/Hive-trunk-h0.21/1770/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3674) test case TestParse broken after recent checkin
[ https://issues.apache.org/jira/browse/HIVE-3674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3674: - Component/s: Tests test case TestParse broken after recent checkin --- Key: HIVE-3674 URL: https://issues.apache.org/jira/browse/HIVE-3674 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.9.0 Reporter: Sambavi Muthukrishnan Assignee: Sambavi Muthukrishnan Fix For: 0.10.0 Attachments: TestParseFix.1.patch The below test cases fail after running svn up on my clean checkout. org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby1 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby2 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby3 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby4 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby5 org.apache.hadoop.hive.ql.parse.TestParse.testParse_groupby6 The build on Nov 2 shows this issue as well. https://builds.apache.org/job/Hive-trunk-h0.21/1770/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3657) errors in testParse due to HIVE-3570
[ https://issues.apache.org/jira/browse/HIVE-3657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3657: - Resolution: Duplicate Status: Resolved (was: Patch Available) errors in testParse due to HIVE-3570 Key: HIVE-3657 URL: https://issues.apache.org/jira/browse/HIVE-3657 Project: Hive Issue Type: Bug Components: Tests Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3657.1.patch For some reason, I missed the test failures before committing. My bad, updating the new test files. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1977) DESCRIBE TABLE syntax doesn't support specifying a database qualified table name
[
https://issues.apache.org/jira/browse/HIVE-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-1977:
-
Status: Patch Available (was: Open)
DESCRIBE TABLE syntax doesn't support specifying a database qualified table
name
Key: HIVE-1977
URL: https://issues.apache.org/jira/browse/HIVE-1977
Project: Hive
Issue Type: Bug
Components: Database/Schema, Query Processor, SQL
Reporter: Carl Steinbach
Assignee: Zhenxiao Luo
Attachments: HIVE-1977.1.patch.txt, HIVE-1977.2.patch.txt,
HIVE-1977.3.patch.txt, HIVE-1977.4.patch.txt, HIVE-1977.5.patch.txt,
HIVE-1977.6.patch.txt
The syntax for DESCRIBE is broken. It should be:
{code}
DESCRIBE [EXTENDED] [database DOT]table [column]
{code}
but is actually
{code}
DESCRIBE [EXTENDED] table[DOT col_name]
{code}
Ref: http://dev.mysql.com/doc/refman/5.0/en/describe.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1977) DESCRIBE TABLE syntax doesn't support specifying a database qualified table name
[
https://issues.apache.org/jira/browse/HIVE-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-1977:
-
Resolution: Fixed
Fix Version/s: 0.10.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks Zhenxiao!
DESCRIBE TABLE syntax doesn't support specifying a database qualified table
name
Key: HIVE-1977
URL: https://issues.apache.org/jira/browse/HIVE-1977
Project: Hive
Issue Type: Bug
Components: Database/Schema, Query Processor, SQL
Reporter: Carl Steinbach
Assignee: Zhenxiao Luo
Fix For: 0.10.0
Attachments: HIVE-1977.1.patch.txt, HIVE-1977.2.patch.txt,
HIVE-1977.3.patch.txt, HIVE-1977.4.patch.txt, HIVE-1977.5.patch.txt,
HIVE-1977.6.patch.txt
The syntax for DESCRIBE is broken. It should be:
{code}
DESCRIBE [EXTENDED] [database DOT]table [column]
{code}
but is actually
{code}
DESCRIBE [EXTENDED] table[DOT col_name]
{code}
Ref: http://dev.mysql.com/doc/refman/5.0/en/describe.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1362) Column level scalar valued statistics
[ https://issues.apache.org/jira/browse/HIVE-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1362: - Summary: Column level scalar valued statistics (was: column level statistics) Column level scalar valued statistics - Key: HIVE-1362 URL: https://issues.apache.org/jira/browse/HIVE-1362 Project: Hive Issue Type: Sub-task Components: Statistics Reporter: Ning Zhang Assignee: Shreepadma Venugopalan Attachments: HIVE-1362.10.patch.txt, HIVE-1362.1.patch.txt, HIVE-1362.2.patch.txt, HIVE-1362.3.patch.txt, HIVE-1362.4.patch.txt, HIVE-1362.5.patch.txt, HIVE-1362.6.patch.txt, HIVE-1362.7.patch.txt, HIVE-1362.8.patch.txt, HIVE-1362.9.patch.txt, HIVE-1362.D6339.1.patch, HIVE-1362_gen-thrift.10.patch.txt, HIVE-1362-gen_thrift.1.patch.txt, HIVE-1362-gen_thrift.2.patch.txt, HIVE-1362-gen_thrift.3.patch.txt, HIVE-1362-gen_thrift.4.patch.txt, HIVE-1362-gen_thrift.5.patch.txt, HIVE-1362-gen_thrift.6.patch.txt, HIVE-1362_gen-thrift.7.patch.txt, HIVE-1362_gen-thrift.8.patch.txt, HIVE-1362_gen-thrift.9.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3517) Provide README file for Oracle backed Metastore
[ https://issues.apache.org/jira/browse/HIVE-3517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3517: - Assignee: Shreepadma Venugopalan Provide README file for Oracle backed Metastore --- Key: HIVE-3517 URL: https://issues.apache.org/jira/browse/HIVE-3517 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach Assignee: Shreepadma Venugopalan We currently provide README files for Derby, MySQL, and Derby. We should also provide a README file for Oracle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1362) Column level scalar valued statistics on Tables and Partitions
[ https://issues.apache.org/jira/browse/HIVE-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1362: - Resolution: Fixed Fix Version/s: 0.10.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk. Thanks Shreepadma! Column level scalar valued statistics on Tables and Partitions -- Key: HIVE-1362 URL: https://issues.apache.org/jira/browse/HIVE-1362 Project: Hive Issue Type: Sub-task Components: Statistics Reporter: Ning Zhang Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Attachments: HIVE-1362.10.patch.txt, HIVE-1362.1.patch.txt, HIVE-1362.2.patch.txt, HIVE-1362.3.patch.txt, HIVE-1362.4.patch.txt, HIVE-1362.5.patch.txt, HIVE-1362.6.patch.txt, HIVE-1362.7.patch.txt, HIVE-1362.8.patch.txt, HIVE-1362.9.patch.txt, HIVE-1362.D6339.1.patch, HIVE-1362_gen-thrift.10.patch.txt, HIVE-1362-gen_thrift.1.patch.txt, HIVE-1362-gen_thrift.2.patch.txt, HIVE-1362-gen_thrift.3.patch.txt, HIVE-1362-gen_thrift.4.patch.txt, HIVE-1362-gen_thrift.5.patch.txt, HIVE-1362-gen_thrift.6.patch.txt, HIVE-1362_gen-thrift.7.patch.txt, HIVE-1362_gen-thrift.8.patch.txt, HIVE-1362_gen-thrift.9.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3678) Add metastore upgrade scripts for column stats schema changes
[ https://issues.apache.org/jira/browse/HIVE-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3678: - Component/s: (was: Database/Schema) Metastore Summary: Add metastore upgrade scripts for column stats schema changes (was: Add upgrade script for column stats schema changes) Add metastore upgrade scripts for column stats schema changes - Key: HIVE-3678 URL: https://issues.apache.org/jira/browse/HIVE-3678 Project: Hive Issue Type: Bug Components: Metastore Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Fix For: 0.10.0 Add upgrade script for column statistics schema changes for Postgres/MySQL/Oracle/Derby -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3676) INSERT INTO regression caused by HIVE-3465
Carl Steinbach created HIVE-3676: Summary: INSERT INTO regression caused by HIVE-3465 Key: HIVE-3676 URL: https://issues.apache.org/jira/browse/HIVE-3676 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3676) INSERT INTO regression caused by HIVE-3465
[
https://issues.apache.org/jira/browse/HIVE-3676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13491258#comment-13491258
]
Carl Steinbach commented on HIVE-3676:
--
Prior to HIVE-3465 the following set of statements produced the expected result:
{noformat}
CREATE DATABASE db2;
USE db2;
CREATE TABLE result(col1 STRING);
INSERT OVERWRITE TABLE result SELECT 'db2_insert1' FROM default.src LIMIT 1;
INSERT INTO TABLE result SELECT 'db2_insert2' FROM default.src LIMIT 1;
SELECT * FROM result;
db_insert1
db_insert2
{noformat}
While the following set of statements produced inaccurate results:
{noformat}
CREATE DATABASE db1;
CREATE TABLE db1.result(col1 STRING);
INSERT OVERWRITE TABLE db1.result SELECT 'db1_insert1' FROM src LIMIT 1;
INSERT INTO TABLE db1.result SELECT 'db1_insert2' FROM src LIMIT 1;
SELECT * FROM db1.result;
db1_insert2
{noformat}
After HIVE-3465 the first set of statements produces inaccurate results, while
the second set of statements now behaves as expected.
INSERT INTO regression caused by HIVE-3465
--
Key: HIVE-3676
URL: https://issues.apache.org/jira/browse/HIVE-3676
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Carl Steinbach
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3563) Drop database cascade fails when there are indexes on any tables
[ https://issues.apache.org/jira/browse/HIVE-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3563: - Resolution: Fixed Fix Version/s: 0.10.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk. Thanks Prasad! Drop database cascade fails when there are indexes on any tables Key: HIVE-3563 URL: https://issues.apache.org/jira/browse/HIVE-3563 Project: Hive Issue Type: Bug Components: Indexing, Metastore Affects Versions: 0.9.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Fix For: 0.10.0 Attachments: HIVE-3563-3.patch, HIVE-3563-4.patch, HIVE-3563-6.patch, HIVE-3563-7.patch Drop database cascade fails if any of the table has index. create database db2; use db2; create table tab1 (id int, name string); create index idx1 on table tab1(id) as 'COMPACT' with DEFERRED REBUILD; drop database db2 cascade; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1977) DESCRIBE TABLE syntax doesn't support specifying a database qualified table name
[
https://issues.apache.org/jira/browse/HIVE-1977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13489684#comment-13489684
]
Carl Steinbach commented on HIVE-1977:
--
+1. Will commit if tests pass.
DESCRIBE TABLE syntax doesn't support specifying a database qualified table
name
Key: HIVE-1977
URL: https://issues.apache.org/jira/browse/HIVE-1977
Project: Hive
Issue Type: Bug
Components: Database/Schema, Query Processor, SQL
Reporter: Carl Steinbach
Assignee: Zhenxiao Luo
Attachments: HIVE-1977.1.patch.txt, HIVE-1977.2.patch.txt,
HIVE-1977.3.patch.txt, HIVE-1977.4.patch.txt, HIVE-1977.5.patch.txt,
HIVE-1977.6.patch.txt
The syntax for DESCRIBE is broken. It should be:
{code}
DESCRIBE [EXTENDED] [database DOT]table [column]
{code}
but is actually
{code}
DESCRIBE [EXTENDED] table[DOT col_name]
{code}
Ref: http://dev.mysql.com/doc/refman/5.0/en/describe.html
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3563) Drop database cascade fails when there are indexes on any tables
[ https://issues.apache.org/jira/browse/HIVE-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3563: - Status: Patch Available (was: Open) Drop database cascade fails when there are indexes on any tables Key: HIVE-3563 URL: https://issues.apache.org/jira/browse/HIVE-3563 Project: Hive Issue Type: Bug Components: Indexing, Metastore Affects Versions: 0.9.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-3563-3.patch, HIVE-3563-4.patch, HIVE-3563-6.patch, HIVE-3563-7.patch Drop database cascade fails if any of the table has index. create database db2; use db2; create table tab1 (id int, name string); create index idx1 on table tab1(id) as 'COMPACT' with DEFERRED REBUILD; drop database db2 cascade; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3563) Drop database cascade fails when there are indexes on any tables
[ https://issues.apache.org/jira/browse/HIVE-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13488508#comment-13488508 ] Carl Steinbach commented on HIVE-3563: -- +1. Will commit if tests pass. Drop database cascade fails when there are indexes on any tables Key: HIVE-3563 URL: https://issues.apache.org/jira/browse/HIVE-3563 Project: Hive Issue Type: Bug Components: Indexing, Metastore Affects Versions: 0.9.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-3563-3.patch, HIVE-3563-4.patch, HIVE-3563-6.patch, HIVE-3563-7.patch Drop database cascade fails if any of the table has index. create database db2; use db2; create table tab1 (id int, name string); create index idx1 on table tab1(id) as 'COMPACT' with DEFERRED REBUILD; drop database db2 cascade; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2715) Upgrade Thrift dependency to 0.9.0
[
https://issues.apache.org/jira/browse/HIVE-2715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-2715:
-
Status: Open (was: Patch Available)
@Ashutosh: After applying the patch I deleted the contents of
*/src/gen/thrift/*, and then tried running the thriftif target again, and
noticed that the following files are not replaced:
{noformat}
D
metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Constants.java
D metastore/src/gen/thrift/gen-php/ThriftHiveMetastore.php
D metastore/src/gen/thrift/gen-php/hive_metastore/ThriftHiveMetastore.php
D metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_constants.php
D metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php
D metastore/src/gen/thrift/gen-php/hive_metastore_constants.php
D metastore/src/gen/thrift/gen-php/hive_metastore_types.php
D ql/src/gen/thrift/gen-php/queryplan/queryplan_types.php
D serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde/Constants.java
D
serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/MegaStruct.java
D
serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/MiniStruct.java
D
serde/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/serde2/thrift/test/MyEnum.java
D serde/src/gen/thrift/gen-php/serde/serde_constants.php
D serde/src/gen/thrift/gen-php/serde/serde_types.php
D service/src/gen/thrift/gen-php/hive_service/ThriftHive.php
D service/src/gen/thrift/gen-php/hive_service/hive_service_types.php
{noformat}
The missing Constants.java files appear to be caused by THRIFT-1090.
I also think it would be a good idea to modify the thriftif Ant target to
automatically clean the old generated artifacts before running the code
generator. The HiveServer2 patch actually includes this modification, so maybe
it would be worthwhile copying it from there and including it in this patch.
Upgrade Thrift dependency to 0.9.0
--
Key: HIVE-2715
URL: https://issues.apache.org/jira/browse/HIVE-2715
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.9.0, 0.8.0
Reporter: Mithun Radhakrishnan
Assignee: Ashutosh Chauhan
Attachments: hive-2715.patch
I work on HCatalog (0.2). Recently, we ran into HCat_server running out of
memory every few days, and it boiled down to a bug in thrift, (THRIFT-1468,
recently fixed).
HCat-0.2-branch depends on Hive-0.8, which in turn depends on thrift-0.5.0.
(The bug also exists on 0.7.0.)
May I please enquire if Hive can't depend on a more current version of
thrift? (Does it break the metastore?) I'm afraid I'm not privy to the
reasoning behind Hive's dependency on a slightly dated thrift-lib.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3563) Drop database cascade fails when there are indexes on any tables
[
https://issues.apache.org/jira/browse/HIVE-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-3563:
-
Status: Open (was: Patch Available)
The gen-test hbase-handler ant target fails:
{noformat}
BUILD FAILED
/Users/carl/Work/repos/hive-test/build.xml:306: The following error occurred
while executing this line:
/Users/carl/Work/repos/hive-test/build.xml:119: The following error occurred
while executing this line:
/Users/carl/Work/repos/hive-test/hbase-handler/build.xml:76:
java.lang.NullPointerException
at java.util.Arrays$ArrayList.init(Arrays.java:3357)
at java.util.Arrays.asList(Arrays.java:3343)
at
org.apache.hadoop.hive.ant.QTestGenTask.execute(QTestGenTask.java:288)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:390)
at org.apache.tools.ant.Target.performTasks(Target.java:411)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
at
org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
at org.apache.tools.ant.taskdefs.SubAnt.execute(SubAnt.java:302)
at org.apache.tools.ant.taskdefs.SubAnt.execute(SubAnt.java:221)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.taskdefs.Sequential.execute(Sequential.java:68)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at
org.apache.tools.ant.taskdefs.MacroInstance.execute(MacroInstance.java:398)
at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
at org.apache.tools.ant.Task.perform(Task.java:348)
at org.apache.tools.ant.Target.execute(Target.java:390)
at org.apache.tools.ant.Target.performTasks(Target.java:411)
at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
at org.apache.tools.ant.Project.executeTarget(Project.java:1366)
at
org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
at org.apache.tools.ant.Main.runBuild(Main.java:801)
at org.apache.tools.ant.Main.startAnt(Main.java:218)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)
Total time: 1 minute 11 seconds
{noformat}
If running all of the tests is too onerous then please at least run one small
test in order to trigger all of the gen-test targets. Thanks.
Drop database cascade fails when there are indexes on any tables
Key: HIVE-3563
URL: https://issues.apache.org/jira/browse/HIVE-3563
Project: Hive
Issue Type: Bug
Components: Indexing, Metastore
Affects Versions: 0.9.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
Attachments: HIVE-3563-3.patch, HIVE-3563-4.patch, HIVE-3563-6.patch
Drop database cascade fails if any of the table has index.
create database db2;
use db2;
create table tab1 (id int, name string);
create index
[jira] [Commented] (HIVE-1362) column level statistics
[ https://issues.apache.org/jira/browse/HIVE-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13487345#comment-13487345 ] Carl Steinbach commented on HIVE-1362: -- @Namit: I created a review request on phabricator: https://reviews.facebook.net/D6339 @Shreepadma: I had to correct one lint error in order to get phabricator to accept this patch (non ASCII character in the GenericUDAF). Phabricator also flagged a bunch of lines that exceed the length limit, and several files that don't end with a newline. Please fix these issues and upload a new patch to JIRA. Thanks. column level statistics --- Key: HIVE-1362 URL: https://issues.apache.org/jira/browse/HIVE-1362 Project: Hive Issue Type: Sub-task Components: Statistics Reporter: Ning Zhang Assignee: Shreepadma Venugopalan Attachments: HIVE-1362.1.patch.txt, HIVE-1362.2.patch.txt, HIVE-1362.3.patch.txt, HIVE-1362.4.patch.txt, HIVE-1362.5.patch.txt, HIVE-1362.6.patch.txt, HIVE-1362.7.patch.txt, HIVE-1362.D6339.1.patch, HIVE-1362-gen_thrift.1.patch.txt, HIVE-1362-gen_thrift.2.patch.txt, HIVE-1362-gen_thrift.3.patch.txt, HIVE-1362-gen_thrift.4.patch.txt, HIVE-1362-gen_thrift.5.patch.txt, HIVE-1362-gen_thrift.6.patch.txt, HIVE-1362_gen-thrift.7.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-1362) column level statistics
[ https://issues.apache.org/jira/browse/HIVE-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13486223#comment-13486223 ] Carl Steinbach commented on HIVE-1362: -- +1. Will commit if tests pass. @Namit: Let me know if you need more time to review this. If so I'll hold off until you're ready. Thanks. column level statistics --- Key: HIVE-1362 URL: https://issues.apache.org/jira/browse/HIVE-1362 Project: Hive Issue Type: Sub-task Components: Statistics Reporter: Ning Zhang Assignee: Shreepadma Venugopalan Attachments: HIVE-1362.1.patch.txt, HIVE-1362.2.patch.txt, HIVE-1362.3.patch.txt, HIVE-1362.4.patch.txt, HIVE-1362.5.patch.txt, HIVE-1362-gen_thrift.1.patch.txt, HIVE-1362-gen_thrift.2.patch.txt, HIVE-1362-gen_thrift.3.patch.txt, HIVE-1362-gen_thrift.4.patch.txt, HIVE-1362-gen_thrift.5.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2723) should throw Ambiguous column reference key Exception in particular join condition
[ https://issues.apache.org/jira/browse/HIVE-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2723: - Description:(was: This Bug can be Repeated as following : create table test(key string, value string); create table test1(key string, value string); 1: Correct! select t.key from (select a.key, b.key from (select * from src ) a right outer join (select * from src1) b on (a.key = b.key)) t; FAILED: Error in semantic analysis: Ambiguous column reference key 2: Uncorrect!! Should throw Exception as above too! select t.key --Is this a.key or b.key ? It's ambiduous! from (select a.\*, b.\* from (select * from src ) a right outer join (select * from src1) b on (a.value = b.value)) t; Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks not specified. Defaulting to jobconf value of: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer=number In order to limit the maximum number of reducers: set hive.exec.reducers.max=number In order to set a constant number of reducers: set mapred.reduce.tasks=number Starting Job = job_201201170959_0004, Tracking URL = http://zongren-VirtualBox:50030/jobdetails.jsp?jobid=job_201201170959_0004 Kill Command = /home/zongren/workspace/hadoop-adh/bin/hadoop job -Dmapred.job.tracker=zongren-VirtualBox:9001 -kill job_201201170959_0004 Hadoop job information for Stage-1: number of mappers: 2; number of reducers: 1 2012-01-17 11:02:47,507 Stage-1 map = 0%, reduce = 0% 2012-01-17 11:02:55,002 Stage-1 map = 100%, reduce = 0% 2012-01-17 11:03:04,240 Stage-1 map = 100%, reduce = 33% 2012-01-17 11:03:05,258 Stage-1 map = 100%, reduce = 100% Ended Job = job_201201170959_0004 MapReduce Jobs Launched: Job 0: Map: 2 Reduce: 1 HDFS Read: 669 HDFS Write: 216 SUCESS Total MapReduce CPU Time Spent: 0 msec OK ) should throw Ambiguous column reference key Exception in particular join condition -- Key: HIVE-2723 URL: https://issues.apache.org/jira/browse/HIVE-2723 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.9.0 Environment: Linux zongren-VirtualBox 3.0.0-14-generic #23-Ubuntu SMP Mon Nov 21 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux java version 1.6.0_25 hadoop-0.20.2-cdh3u0 hive-0.7.0-cdh3u0 Reporter: caofangkun Assignee: Navis Priority: Minor Labels: exception-handling, query, queryparser Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2723.D1275.1.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2723.D1275.2.patch, HIVE-2723.D1275.3.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3563) Drop database cascade fails when there are indexes on any tables
[ https://issues.apache.org/jira/browse/HIVE-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13484311#comment-13484311 ] Carl Steinbach commented on HIVE-3563: -- +1. Will commit if tests pass. Drop database cascade fails when there are indexes on any tables Key: HIVE-3563 URL: https://issues.apache.org/jira/browse/HIVE-3563 Project: Hive Issue Type: Bug Components: Indexing, Metastore Affects Versions: 0.9.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-3563-3.patch Drop database cascade fails if any of the table has index. create database db2; use db2; create table tab1 (id int, name string); create index idx1 on table tab1(id) as 'COMPACT' with DEFERRED REBUILD; drop database db2 cascade; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3596) Regression - HiveConf static variable causes issues in long running JVM instances with /tmp/ data
[
https://issues.apache.org/jira/browse/HIVE-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-3596:
-
Status: Open (was: Patch Available)
Looks like the problem is still there after applying the patch:
{noformat}
hive set system:java.io.tmpdir;
set system:java.io.tmpdir;
system:java.io.tmpdir=/var/folders/6d/bmvmcxh938q6htnxtyg9jk5rgn/T/
hive SHOW TABLES;
...
hive !ls ${system:java.io.tmpdir};
...
hive-default-187929306310164734.xml
hive-default-433053476043176173.xml
hive-default-3093481938548070336.xml
hive-default-159172276671008947.xml
...
hive !rm ${system:java.io.tmpdir}/hive-default*;
hive SHOW TABLES;
Failed with exception java.io.IOException:java.lang.RuntimeException:
java.io.FileNotFoundException:
/var/folders/6d/bmvmcxh938q6htnxtyg9jk5rgn/T/hive-default-187929306310164734.xml
(No such file or directory)
Time taken: 0.081 seconds
{noformat}
It's also worth pointing out that previously we created a single copy of
hive-default that was used by all HiveConf instances, but with this patch in
place we create a new hive-default file for each HiveConf instance. Roughly
speaking this means that we end up creating at least one new hive-default file
for each statement that is executed.
Regression - HiveConf static variable causes issues in long running JVM
instances with /tmp/ data
-
Key: HIVE-3596
URL: https://issues.apache.org/jira/browse/HIVE-3596
Project: Hive
Issue Type: Bug
Components: Configuration
Affects Versions: 0.9.0, 0.8.1, 0.8.0
Reporter: Chris McConnell
Assignee: Chris McConnell
Fix For: 0.10.0, 0.9.0, 0.8.1
Attachments: HIVE-3596.patch
With Hive 0.8.x, HiveConf was changed to utilize the private, static member
confVarURL which points to /tmp/hive-user-tmp_number.xml for job
configuration settings.
During long running JVMs, such as a Beeswax server, which creates multiple
HiveConf objects over time this variable does not properly get updated
between jobs and can cause job failure if the OS cleans /tmp/ during a cron
job.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3563) Drop database cascade fails when there are indexes on any tables
[ https://issues.apache.org/jira/browse/HIVE-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13481984#comment-13481984 ] Carl Steinbach commented on HIVE-3563: -- @Prasad: Is this ready for review? Is HIVE-3563-2.patch the most recent version of this patch? Thanks. Drop database cascade fails when there are indexes on any tables Key: HIVE-3563 URL: https://issues.apache.org/jira/browse/HIVE-3563 Project: Hive Issue Type: Bug Components: Indexing, Metastore Affects Versions: 0.9.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-3563-1.patch, HIVE-3563-2.patch Drop database cascade fails if any of the table has index. create database db2; use db2; create table tab1 (id int, name string); create index idx1 on table tab1(id) as 'COMPACT' with DEFERRED REBUILD; drop database db2 cascade; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3608) Support appends (INSERT INTO) for bucket tables
Carl Steinbach created HIVE-3608: Summary: Support appends (INSERT INTO) for bucket tables Key: HIVE-3608 URL: https://issues.apache.org/jira/browse/HIVE-3608 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Carl Steinbach Support for appending (INSERT INTO) non-bucketed tables was added in HIVE-306. We should extend this to support bucketed tables as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3608) Support appends (INSERT INTO) for bucketed tables
[ https://issues.apache.org/jira/browse/HIVE-3608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3608: - Summary: Support appends (INSERT INTO) for bucketed tables (was: Support appends (INSERT INTO) for bucket tables) Support appends (INSERT INTO) for bucketed tables - Key: HIVE-3608 URL: https://issues.apache.org/jira/browse/HIVE-3608 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Carl Steinbach Support for appending (INSERT INTO) non-bucketed tables was added in HIVE-306. We should extend this to support bucketed tables as well. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3609) Nested sub select queries with duplicate column aliases should throw an error instead of truncating columns
[
https://issues.apache.org/jira/browse/HIVE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13482087#comment-13482087
]
Carl Steinbach commented on HIVE-3609:
--
This makes sense:
{noformat}
hive SELECT * FROM src LIMIT 1;
OK
key value
238 val_238
Time taken: 0.061 seconds
hive SELECT * FROM src a JOIN src b ON (a.key = b.key) LIMIT 1;
OK
key value key value
0 val_0 0 val_0
{noformat}
But this doesn't:
{noformat}
hive SELECT * FROM (SELECT * FROM src a JOIN src b ON (a.key = b.key) LIMIT 1)
c;
OK
key value
0 val_0
{noformat}
Note that the third query produces the following SQL exception when run in
MySQL:
{noformat}
mysql SELECT * FROM (SELECT * FROM src a JOIN src b ON (a.key=b.key) LIMIT 1)
as c;
ERROR 1060 (42S21): Duplicate column name 'key'
{noformat}
I think Hive should throw an exception in this situation too.
Nested sub select queries with duplicate column aliases should throw an error
instead of truncating columns
---
Key: HIVE-3609
URL: https://issues.apache.org/jira/browse/HIVE-3609
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Carl Steinbach
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HIVE-3609) Nested sub select queries with duplicate column aliases should throw an error instead of truncating columns
[ https://issues.apache.org/jira/browse/HIVE-3609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach resolved HIVE-3609. -- Resolution: Duplicate Nested sub select queries with duplicate column aliases should throw an error instead of truncating columns --- Key: HIVE-3609 URL: https://issues.apache.org/jira/browse/HIVE-3609 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2723) should throw Ambiguous column reference key Exception in particular join condition
[ https://issues.apache.org/jira/browse/HIVE-2723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2723: - Status: Open (was: Patch Available) @Navis: Looks like this patch almost applies cleanly except for one small reject in SemanticAnalyzer.java. Would you be willing to rebase this patch and submit it for review? Thanks. should throw Ambiguous column reference key Exception in particular join condition -- Key: HIVE-2723 URL: https://issues.apache.org/jira/browse/HIVE-2723 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.9.0 Environment: Linux zongren-VirtualBox 3.0.0-14-generic #23-Ubuntu SMP Mon Nov 21 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux java version 1.6.0_25 hadoop-0.20.2-cdh3u0 hive-0.7.0-cdh3u0 Reporter: caofangkun Assignee: Navis Priority: Minor Labels: exception-handling, query, queryparser Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2723.D1275.1.patch, ASF.LICENSE.NOT.GRANTED--HIVE-2723.D1275.2.patch This Bug can be Repeated as following : create table test(key string, value string); create table test1(key string, value string); 1: Correct! select t.key from (select a.key, b.key from (select * from src ) a right outer join (select * from src1) b on (a.key = b.key)) t; FAILED: Error in semantic analysis: Ambiguous column reference key 2: Uncorrect!! Should throw Exception as above too! select t.key --Is this a.key or b.key ? It's ambiduous! from (select a.\*, b.\* from (select * from src ) a right outer join (select * from src1) b on (a.value = b.value)) t; Total MapReduce jobs = 1 Launching Job 1 out of 1 Number of reduce tasks not specified. Defaulting to jobconf value of: 1 In order to change the average load for a reducer (in bytes): set hive.exec.reducers.bytes.per.reducer=number In order to limit the maximum number of reducers: set hive.exec.reducers.max=number In order to set a constant number of reducers: set mapred.reduce.tasks=number Starting Job = job_201201170959_0004, Tracking URL = http://zongren-VirtualBox:50030/jobdetails.jsp?jobid=job_201201170959_0004 Kill Command = /home/zongren/workspace/hadoop-adh/bin/hadoop job -Dmapred.job.tracker=zongren-VirtualBox:9001 -kill job_201201170959_0004 Hadoop job information for Stage-1: number of mappers: 2; number of reducers: 1 2012-01-17 11:02:47,507 Stage-1 map = 0%, reduce = 0% 2012-01-17 11:02:55,002 Stage-1 map = 100%, reduce = 0% 2012-01-17 11:03:04,240 Stage-1 map = 100%, reduce = 33% 2012-01-17 11:03:05,258 Stage-1 map = 100%, reduce = 100% Ended Job = job_201201170959_0004 MapReduce Jobs Launched: Job 0: Map: 2 Reduce: 1 HDFS Read: 669 HDFS Write: 216 SUCESS Total MapReduce CPU Time Spent: 0 msec OK -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3590) TCP KeepAlive and connection timeout for the HiveServer
[
https://issues.apache.org/jira/browse/HIVE-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-3590:
-
Resolution: Fixed
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks Esteban!
TCP KeepAlive and connection timeout for the HiveServer
---
Key: HIVE-3590
URL: https://issues.apache.org/jira/browse/HIVE-3590
Project: Hive
Issue Type: Improvement
Components: Server Infrastructure
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Labels: patch
Fix For: 0.10.0
Attachments: HIVE-3590.1.patch, HIVE-3590.2.patch
Simple implementation of TCP KeepAlive and socket timeout for the HiveServer
in order to avoid half open connections or connection drops during long
running queries that use HiveServer.
Two new settings have been added in order to control this functionality.
{code}
property
namehive.server.read.socket.timeout/name
value10/value
descriptionTimeout for the HiveServer to close the connection if no
response from the client in N seconds, defaults to 10 seconds./description
/property
property
namehive.server.tcp.keepalive/name
valuetrue/value
descriptionWhether to enable TCP keepalive for the HiveServer. Keepalive
will prevent accumulation of half-open connections./description
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3596) Regression - HiveConf static variable causes issues in long running JVM instances with /tmp/ data
[ https://issues.apache.org/jira/browse/HIVE-3596?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3596: - Assignee: Chris McConnell Regression - HiveConf static variable causes issues in long running JVM instances with /tmp/ data - Key: HIVE-3596 URL: https://issues.apache.org/jira/browse/HIVE-3596 Project: Hive Issue Type: Bug Components: Configuration Affects Versions: 0.8.0, 0.8.1, 0.9.0 Reporter: Chris McConnell Assignee: Chris McConnell Fix For: 0.8.1, 0.9.0, 0.10.0 Attachments: HIVE-3596.patch With Hive 0.8.x, HiveConf was changed to utilize the private, static member confVarURL which points to /tmp/hive-user-tmp_number.xml for job configuration settings. During long running JVMs, such as a Beeswax server, which creates multiple HiveConf objects over time this variable does not properly get updated between jobs and can cause job failure if the OS cleans /tmp/ during a cron job. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3525) Avro Maps with Nullable Values fail with NPE
[ https://issues.apache.org/jira/browse/HIVE-3525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3525: - Assignee: Sean Busbey Avro Maps with Nullable Values fail with NPE Key: HIVE-3525 URL: https://issues.apache.org/jira/browse/HIVE-3525 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Reporter: Sean Busbey Assignee: Sean Busbey Fix For: 0.10.0 Attachments: HIVE-3525.1.patch.txt, HIVE-3525.2.patch.txt When working against current trunk@1393794, using a backing Avro schema that has a Map field with nullable values causes a NPE on deserialization when the map contains a null value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3525) Avro Maps with Nullable Values fail with NPE
[ https://issues.apache.org/jira/browse/HIVE-3525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3525: - Resolution: Fixed Fix Version/s: 0.10.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk. Thanks Sean! Avro Maps with Nullable Values fail with NPE Key: HIVE-3525 URL: https://issues.apache.org/jira/browse/HIVE-3525 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Reporter: Sean Busbey Assignee: Sean Busbey Fix For: 0.10.0 Attachments: HIVE-3525.1.patch.txt, HIVE-3525.2.patch.txt When working against current trunk@1393794, using a backing Avro schema that has a Map field with nullable values causes a NPE on deserialization when the map contains a null value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3535) derby metastore upgrade script throw errors when updating from 0.7 to 0.8
[ https://issues.apache.org/jira/browse/HIVE-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3535: - Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk. Thanks Zhenxiao! derby metastore upgrade script throw errors when updating from 0.7 to 0.8 - Key: HIVE-3535 URL: https://issues.apache.org/jira/browse/HIVE-3535 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.9.0 Reporter: Zhenxiao Luo Assignee: Zhenxiao Luo Fix For: 0.10.0 Attachments: HIVE-3535.1.patch.txt $DERBY_HOME/bin/ij upgrade.sql ij version 10.4 ij CONNECT 'jdbc:derby:/var/lib/hive/metastore/metastore_db'; ij RUN '/usr/lib/hive/scripts/metastore/upgrade/derby/upgrade-0.7.0-to-0.8.0.derby.sql'; ij -- Upgrade MetaStore schema from 0.7.0 to 0.8.0 RUN '008-HIVE-2246.derby.sql'; ij /* * Creates the following tables: * - CDS * - COLUMNS_V2 * The new columns table is called COLUMNS_V2 * because many columns are removed, and the schema is changed. * It'd take too long to migrate and keep the same table. */ CREATE TABLE CDS ( CD_ID bigint NOT NULL, PRIMARY KEY (CD_ID) ); 0 rows inserted/updated/deleted ij CREATE TABLE COLUMNS_V2 ( CD_ID bigint NOT NULL, COMMENT varchar(4000), COLUMN_NAME varchar(128) NOT NULL, TYPE_NAME varchar(4000), INTEGER_IDX INTEGER NOT NULL, PRIMARY KEY (CD_ID, COLUMN_NAME) ); 0 rows inserted/updated/deleted ij ALTER TABLE COLUMNS_V2 ADD CONSTRAINT COLUMNS_V2_FK1 FOREIGN KEY (CD_ID) REFERENCES CDS (CD_ID) ON DELETE NO ACTION ON UPDATE NO ACTION ; 0 rows inserted/updated/deleted ij /* Alter the SDS table to: * - add the column CD_ID * - add a foreign key on CD_ID * - create an index on CD_ID */ ALTER TABLE SDS ADD COLUMN CD_ID bigint ; 0 rows inserted/updated/deleted ij ALTER TABLE SDS ADD CONSTRAINT SDS_FK2 FOREIGN KEY (CD_ID) REFERENCES CDS (CD_ID) ; 0 rows inserted/updated/deleted ij /* * Migrate the TBLS table * Add entries into CDS. * Populate the CD_ID field in SDS for tables * Add entires to COLUMNS_V2 based on this table's sd's columns */ /* In the migration, there is a 1:1 mapping between CD_ID and SD_ID * for tables. For speed, just let CD_ID = SD_ID for tables */ INSERT INTO CDS (CD_ID) SELECT t.SD_ID FROM TBLS t WHERE t.SD_ID IS NOT NULL ORDER BY t.SD_ID; ERROR 42X01: Syntax error: Encountered ORDER at line 13, column 54. ij UPDATE SDS SET CD_ID = SD_ID WHERE SD_ID in (SELECT t.SD_ID FROM TBLS t WHERE t.SD_ID IS NOT NULL ORDER BY t.SD_ID); ERROR 42X01: Syntax error: Encountered ORDER at line 4, column 55. ij INSERT INTO COLUMNS_V2 (CD_ID, COMMENT, COLUMN_NAME, TYPE_NAME, INTEGER_IDX) SELECT c.SD_ID, c.COMMENT, c.COLUMN_NAME, c.TYPE_NAME, c.INTEGER_IDX FROM COLUMNS c JOIN TBLS t ON t.SD_ID = c.SD_ID ; ERROR 23503: INSERT on table 'COLUMNS_V2' caused a violation of foreign key constraint 'COLUMNS_V2_FK1' for key (1). The statement has been rolled back. ij /* * Migrate the partitions. * Update the partitions' SDS to use the parent tables' CD_ID BEGIN * Derby does not allow joins in update statements, * so we have to make a temporary tableh */ DECLARE GLOBAL TEMPORARY TABLE TMP_TBL ( SD_ID bigint not null, CD_ID bigint not null ) ON COMMIT PRESERVE ROWS NOT LOGGED; 0 rows inserted/updated/deleted ij INSERT INTO SESSION.TMP_TBL SELECT p.SD_ID, sds.CD_ID FROM PARTITIONS p JOIN TBLS t ON t.TBL_ID = p.TBL_ID JOIN SDS sds on t.SD_ID = sds.SD_ID WHERE p.SD_ID IS NOT NULL; ERROR 23502: Column 'CD_ID' cannot accept a NULL value. ij UPDATE SDS sd SET sd.CD_ID = (SELECT tt.CD_ID FROM SESSION.TMP_TBL tt WHERE tt.SD_ID = sd.SD_ID) WHERE sd.SD_ID IN (SELECT SD_ID FROM SESSION.TMP_TBL); 0 rows inserted/updated/deleted WARNING 02000: No row was found for FETCH, UPDATE or DELETE; or the result of a query is an empty table. ij /* * Migrate IDXS */ INSERT INTO CDS (CD_ID) SELECT i.SD_ID FROM IDXS i WHERE i.SD_ID IS NOT NULL ORDER BY i.SD_ID; ERROR 42X01: Syntax error: Encountered ORDER at line 5, column 54. ij UPDATE SDS SET CD_ID = SD_ID WHERE SD_ID in (SELECT i.SD_ID FROM IDXS i WHERE i.SD_ID IS NOT NULL ORDER BY i.SD_ID); ERROR 42X01: Syntax error: Encountered ORDER at line 4, column 55. ij INSERT INTO COLUMNS_V2 (CD_ID, COMMENT, COLUMN_NAME, TYPE_NAME, INTEGER_IDX) SELECT c.SD_ID, c.COMMENT, c.COLUMN_NAME, c.TYPE_NAME, c.INTEGER_IDX FROM COLUMNS c JOIN IDXS i ON i.SD_ID = c.SD_ID ; ERROR 42X05: Table/View 'IDXS' does not exist. ij /* * rename the old COLUMNS table */ RENAME TABLE COLUMNS TO
[jira] [Commented] (HIVE-2935) Implement HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477665#comment-13477665 ] Carl Steinbach commented on HIVE-2935: -- We're not planning to combine these services. Rather, we're planning to run these services in parallel in the same JVM, but each service will operate on its own unique port. If you want to use the HS2 API and the metastore API at the same time you will need to create two separate connections and use two separate clients. Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: HiveServer2 Attachments: beelinepositive.tar.gz, HIVE-2935.1.notest.patch.txt, HIVE-2935.2.notest.patch.txt, HIVE-2935.2.nothrift.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3589) describe command not giving the details when called as describe dbname.tablename
[ https://issues.apache.org/jira/browse/HIVE-3589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3589: - Component/s: (was: CLI) Query Processor Metastore describe command not giving the details when called as describe dbname.tablename Key: HIVE-3589 URL: https://issues.apache.org/jira/browse/HIVE-3589 Project: Hive Issue Type: Bug Components: Metastore, Query Processor Affects Versions: 0.8.1 Reporter: Sujesh Chirackkal Priority: Minor describe command not giving the details when called as describe dbname.tablename. Throwing the error Table dbname not found. Ex: hive -e describe masterdb.table1 will throw error Table masterdb not found -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3590) TCP KeepAlive and connection timeout for the HiveServer
[
https://issues.apache.org/jira/browse/HIVE-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-3590:
-
Status: Open (was: Patch Available)
@Esteban: I left comments for you on reviewboard. Tnanks.
TCP KeepAlive and connection timeout for the HiveServer
---
Key: HIVE-3590
URL: https://issues.apache.org/jira/browse/HIVE-3590
Project: Hive
Issue Type: Improvement
Components: Server Infrastructure
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Labels: patch
Fix For: 0.10.0
Attachments: HIVE-3590.1.patch
Simple implementation of TCP KeepAlive and socket timeout for the HiveServer
in order to avoid half open connections or connection drops during long
running queries that use HiveServer.
Two new settings have been added in order to control this functionality.
{code}
property
namehive.server.read.socket.timeout/name
value10/value
descriptionTimeout for the HiveServer to close the connection if no
response from the client in N seconds, defaults is 10 seconds./description
/property
property
namehive.server.tcp.keepalive/name
valuetrue/value
descriptionWhether to enable TCP keepalive for the HiveServer. Keepalive
will prevent accumulation of half-open connections./description
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3590) TCP KeepAlive and connection timeout for the HiveServer
[
https://issues.apache.org/jira/browse/HIVE-3590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13477735#comment-13477735
]
Carl Steinbach commented on HIVE-3590:
--
+1. Will commit in a day if tests pass and no one requests more time for review.
TCP KeepAlive and connection timeout for the HiveServer
---
Key: HIVE-3590
URL: https://issues.apache.org/jira/browse/HIVE-3590
Project: Hive
Issue Type: Improvement
Components: Server Infrastructure
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Priority: Critical
Labels: patch
Fix For: 0.10.0
Attachments: HIVE-3590.1.patch, HIVE-3590.2.patch
Simple implementation of TCP KeepAlive and socket timeout for the HiveServer
in order to avoid half open connections or connection drops during long
running queries that use HiveServer.
Two new settings have been added in order to control this functionality.
{code}
property
namehive.server.read.socket.timeout/name
value10/value
descriptionTimeout for the HiveServer to close the connection if no
response from the client in N seconds, defaults to 10 seconds./description
/property
property
namehive.server.tcp.keepalive/name
valuetrue/value
descriptionWhether to enable TCP keepalive for the HiveServer. Keepalive
will prevent accumulation of half-open connections./description
/property
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3525) Avro Maps with Nullable Values fail with NPE
[ https://issues.apache.org/jira/browse/HIVE-3525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478416#comment-13478416 ] Carl Steinbach commented on HIVE-3525: -- @Jakob: thanks for reviewing this! @Sean: +1. Will commit if tests pass. Avro Maps with Nullable Values fail with NPE Key: HIVE-3525 URL: https://issues.apache.org/jira/browse/HIVE-3525 Project: Hive Issue Type: Bug Components: Serializers/Deserializers Reporter: Sean Busbey Attachments: HIVE-3525.1.patch.txt, HIVE-3525.2.patch.txt When working against current trunk@1393794, using a backing Avro schema that has a Map field with nullable values causes a NPE on deserialization when the map contains a null value. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3563) Drop database cascade fails when there are indexes on any tables
[ https://issues.apache.org/jira/browse/HIVE-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3563: - Status: Open (was: Patch Available) @Prasad: I left some more comments on reviewboard. Thanks. Drop database cascade fails when there are indexes on any tables Key: HIVE-3563 URL: https://issues.apache.org/jira/browse/HIVE-3563 Project: Hive Issue Type: Bug Components: Indexing, Metastore Affects Versions: 0.9.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-3563-1.patch, HIVE-3563-2.patch Drop database cascade fails if any of the table has index. create database db2; use db2; create table tab1 (id int, name string); create index idx1 on table tab1(id) as 'COMPACT' with DEFERRED REBUILD; drop database db2 cascade; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3535) derby metastore upgrade script throw errors when updating from 0.7 to 0.8
[ https://issues.apache.org/jira/browse/HIVE-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478559#comment-13478559 ] Carl Steinbach commented on HIVE-3535: -- +1. Will commit if tests pass. Thanks. derby metastore upgrade script throw errors when updating from 0.7 to 0.8 - Key: HIVE-3535 URL: https://issues.apache.org/jira/browse/HIVE-3535 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.9.0 Reporter: Zhenxiao Luo Assignee: Zhenxiao Luo Fix For: 0.10.0 Attachments: HIVE-3535.1.patch.txt $DERBY_HOME/bin/ij upgrade.sql ij version 10.4 ij CONNECT 'jdbc:derby:/var/lib/hive/metastore/metastore_db'; ij RUN '/usr/lib/hive/scripts/metastore/upgrade/derby/upgrade-0.7.0-to-0.8.0.derby.sql'; ij -- Upgrade MetaStore schema from 0.7.0 to 0.8.0 RUN '008-HIVE-2246.derby.sql'; ij /* * Creates the following tables: * - CDS * - COLUMNS_V2 * The new columns table is called COLUMNS_V2 * because many columns are removed, and the schema is changed. * It'd take too long to migrate and keep the same table. */ CREATE TABLE CDS ( CD_ID bigint NOT NULL, PRIMARY KEY (CD_ID) ); 0 rows inserted/updated/deleted ij CREATE TABLE COLUMNS_V2 ( CD_ID bigint NOT NULL, COMMENT varchar(4000), COLUMN_NAME varchar(128) NOT NULL, TYPE_NAME varchar(4000), INTEGER_IDX INTEGER NOT NULL, PRIMARY KEY (CD_ID, COLUMN_NAME) ); 0 rows inserted/updated/deleted ij ALTER TABLE COLUMNS_V2 ADD CONSTRAINT COLUMNS_V2_FK1 FOREIGN KEY (CD_ID) REFERENCES CDS (CD_ID) ON DELETE NO ACTION ON UPDATE NO ACTION ; 0 rows inserted/updated/deleted ij /* Alter the SDS table to: * - add the column CD_ID * - add a foreign key on CD_ID * - create an index on CD_ID */ ALTER TABLE SDS ADD COLUMN CD_ID bigint ; 0 rows inserted/updated/deleted ij ALTER TABLE SDS ADD CONSTRAINT SDS_FK2 FOREIGN KEY (CD_ID) REFERENCES CDS (CD_ID) ; 0 rows inserted/updated/deleted ij /* * Migrate the TBLS table * Add entries into CDS. * Populate the CD_ID field in SDS for tables * Add entires to COLUMNS_V2 based on this table's sd's columns */ /* In the migration, there is a 1:1 mapping between CD_ID and SD_ID * for tables. For speed, just let CD_ID = SD_ID for tables */ INSERT INTO CDS (CD_ID) SELECT t.SD_ID FROM TBLS t WHERE t.SD_ID IS NOT NULL ORDER BY t.SD_ID; ERROR 42X01: Syntax error: Encountered ORDER at line 13, column 54. ij UPDATE SDS SET CD_ID = SD_ID WHERE SD_ID in (SELECT t.SD_ID FROM TBLS t WHERE t.SD_ID IS NOT NULL ORDER BY t.SD_ID); ERROR 42X01: Syntax error: Encountered ORDER at line 4, column 55. ij INSERT INTO COLUMNS_V2 (CD_ID, COMMENT, COLUMN_NAME, TYPE_NAME, INTEGER_IDX) SELECT c.SD_ID, c.COMMENT, c.COLUMN_NAME, c.TYPE_NAME, c.INTEGER_IDX FROM COLUMNS c JOIN TBLS t ON t.SD_ID = c.SD_ID ; ERROR 23503: INSERT on table 'COLUMNS_V2' caused a violation of foreign key constraint 'COLUMNS_V2_FK1' for key (1). The statement has been rolled back. ij /* * Migrate the partitions. * Update the partitions' SDS to use the parent tables' CD_ID BEGIN * Derby does not allow joins in update statements, * so we have to make a temporary tableh */ DECLARE GLOBAL TEMPORARY TABLE TMP_TBL ( SD_ID bigint not null, CD_ID bigint not null ) ON COMMIT PRESERVE ROWS NOT LOGGED; 0 rows inserted/updated/deleted ij INSERT INTO SESSION.TMP_TBL SELECT p.SD_ID, sds.CD_ID FROM PARTITIONS p JOIN TBLS t ON t.TBL_ID = p.TBL_ID JOIN SDS sds on t.SD_ID = sds.SD_ID WHERE p.SD_ID IS NOT NULL; ERROR 23502: Column 'CD_ID' cannot accept a NULL value. ij UPDATE SDS sd SET sd.CD_ID = (SELECT tt.CD_ID FROM SESSION.TMP_TBL tt WHERE tt.SD_ID = sd.SD_ID) WHERE sd.SD_ID IN (SELECT SD_ID FROM SESSION.TMP_TBL); 0 rows inserted/updated/deleted WARNING 02000: No row was found for FETCH, UPDATE or DELETE; or the result of a query is an empty table. ij /* * Migrate IDXS */ INSERT INTO CDS (CD_ID) SELECT i.SD_ID FROM IDXS i WHERE i.SD_ID IS NOT NULL ORDER BY i.SD_ID; ERROR 42X01: Syntax error: Encountered ORDER at line 5, column 54. ij UPDATE SDS SET CD_ID = SD_ID WHERE SD_ID in (SELECT i.SD_ID FROM IDXS i WHERE i.SD_ID IS NOT NULL ORDER BY i.SD_ID); ERROR 42X01: Syntax error: Encountered ORDER at line 4, column 55. ij INSERT INTO COLUMNS_V2 (CD_ID, COMMENT, COLUMN_NAME, TYPE_NAME, INTEGER_IDX) SELECT c.SD_ID, c.COMMENT, c.COLUMN_NAME, c.TYPE_NAME, c.INTEGER_IDX FROM COLUMNS c JOIN IDXS i ON i.SD_ID = c.SD_ID ; ERROR 42X05: Table/View 'IDXS' does not exist. ij /* * rename the old COLUMNS table */ RENAME TABLE COLUMNS TO COLUMNS_OLD; 0 rows inserted/updated/deleted ij
[jira] [Commented] (HIVE-2715) Upgrade Thrift dependency to 0.9.0
[ https://issues.apache.org/jira/browse/HIVE-2715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478584#comment-13478584 ] Carl Steinbach commented on HIVE-2715: -- Thrift 0.9.0 has been released. We should Hive's Thrift dependency. Upgrade Thrift dependency to 0.9.0 -- Key: HIVE-2715 URL: https://issues.apache.org/jira/browse/HIVE-2715 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.8.0 Reporter: Mithun Radhakrishnan I work on HCatalog (0.2). Recently, we ran into HCat_server running out of memory every few days, and it boiled down to a bug in thrift, (THRIFT-1468, recently fixed). HCat-0.2-branch depends on Hive-0.8, which in turn depends on thrift-0.5.0. (The bug also exists on 0.7.0.) May I please enquire if Hive can't depend on a more current version of thrift? (Does it break the metastore?) I'm afraid I'm not privy to the reasoning behind Hive's dependency on a slightly dated thrift-lib. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (HIVE-2715) Upgrade Thrift dependency to 0.9.0
[ https://issues.apache.org/jira/browse/HIVE-2715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13478584#comment-13478584 ] Carl Steinbach edited comment on HIVE-2715 at 10/18/12 1:46 AM: Thrift 0.9.0 has been released. We should update Hive's Thrift dependency to 0.9.0. was (Author: cwsteinbach): Thrift 0.9.0 has been released. We should Hive's Thrift dependency. Upgrade Thrift dependency to 0.9.0 -- Key: HIVE-2715 URL: https://issues.apache.org/jira/browse/HIVE-2715 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.8.0 Reporter: Mithun Radhakrishnan I work on HCatalog (0.2). Recently, we ran into HCat_server running out of memory every few days, and it boiled down to a bug in thrift, (THRIFT-1468, recently fixed). HCat-0.2-branch depends on Hive-0.8, which in turn depends on thrift-0.5.0. (The bug also exists on 0.7.0.) May I please enquire if Hive can't depend on a more current version of thrift? (Does it break the metastore?) I'm afraid I'm not privy to the reasoning behind Hive's dependency on a slightly dated thrift-lib. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2539) Enable passing username/password via JDBC
[ https://issues.apache.org/jira/browse/HIVE-2539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2539: - Status: Open (was: Patch Available) Please post a review request on reviewboard or phabricator. Thanks. Enable passing username/password via JDBC - Key: HIVE-2539 URL: https://issues.apache.org/jira/browse/HIVE-2539 Project: Hive Issue Type: Improvement Components: JDBC Affects Versions: 0.7.1 Reporter: Sriram Krishnan Assignee: chunqing xie Labels: patch Attachments: HIVE-2539.PATCH Changing the username and/or the password seems to have no effect (also confirmed here: https://cwiki.apache.org/Hive/hivejdbcinterface.html). Connection con = DriverManager.getConnection(jdbc:hive://localhost:1/default, , ); Would be beneficial to pass the username/password via JDBC - and also for the server to honor the username password being passed (may be dependent of that being fixed first). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3576) Regression: ALTER TABLE DROP IF EXISTS PARTITION throws a SemanticException if Partition is not found
[
https://issues.apache.org/jira/browse/HIVE-3576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-3576:
-
Component/s: Metastore
Regression: ALTER TABLE DROP IF EXISTS PARTITION throws a SemanticException
if Partition is not found
-
Key: HIVE-3576
URL: https://issues.apache.org/jira/browse/HIVE-3576
Project: Hive
Issue Type: Bug
Components: Metastore, Query Processor
Affects Versions: 0.9.0
Reporter: Harsh J
Doing a simple {{ALTER TABLE testtable DROP IF EXISTS
PARTITION(dt=NONEXISTENTPARTITION)}} fails with a SemanticException of the
10006 kind (INVALID_PARTITION).
This does not respect the {{hive.exec.drop.ignorenonexistent}} condition
either, since there are no if-check-wraps around this area, when fetching
partitions from the store.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3437) 0.23 compatibility: fix unit tests when building against 0.23
[ https://issues.apache.org/jira/browse/HIVE-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13476023#comment-13476023 ] Carl Steinbach commented on HIVE-3437: -- @Chris: I left some comments on phabricator. Please let me know what you think. Thanks. 0.23 compatibility: fix unit tests when building against 0.23 - Key: HIVE-3437 URL: https://issues.apache.org/jira/browse/HIVE-3437 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.9.0, 0.10.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.9.0, 0.10.0 Attachments: HIVE-3437-0.9-1.patch, HIVE-3437-0.9-2.patch, HIVE-3437-0.9-3.patch, HIVE-3437-0.9-4.patch, HIVE-3437-0.9.patch, HIVE-3437-trunk-1.patch, HIVE-3437-trunk-2.patch, HIVE-3437-trunk-3.patch, HIVE-3437-trunk-4.patch, HIVE-3437-trunk.patch Many unit tests fail as a result of building the code against hadoop 0.23. Initial focus will be to fix 0.9. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2935) Implement HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13476464#comment-13476464 ] Carl Steinbach commented on HIVE-2935: -- @Alan: I think this patch is in pretty good shape right now. Since this work is almsot completely decoupled from the rest of Hive I think the best option would be to commit it directly to trunk and address any bugs in separate tickets. However, in an earlier email to the PMC mailing list you argued that the size of this patch precludes the possibility of reviewing it, so I am in the process of breaking it into smaller pieces (as detailed above) and will start posting those pieces for review over the next couple of days. I don't think committing this to a separate branch will make the code any easier to review, and there are no established conventions within this project concerning development work on feature branches. I would prefer that we not make this patch the testcase for a new policy. Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: HiveServer2 Attachments: beelinepositive.tar.gz, HIVE-2935.1.notest.patch.txt, HIVE-2935.2.notest.patch.txt, HIVE-2935.2.nothrift.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2935) Implement HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13476476#comment-13476476 ] Carl Steinbach commented on HIVE-2935: -- bq. Alan and I have been talking about various approaches to JDBC / ODBC. We'll work to get something on the list in the next few weeks. We've got a RESTful API to HCat and Hive execution prototyped (with help from a bunch of folks). It seems like we can put a thin client together for ODBC JDBC based on these APIs that will be more consistent with future plans for adding web services to YARN and current work on HCat. @Alan: Are you guys still working on this? If so are there any design docs I can look at? The RESTful HCat API for Hive looks like it's batch oriented. I don't think it's going to be possible to build an ODBC or JDBC driver on top of that API without adding explicit support for sessions and statement oriented execute/fetch calls. Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: HiveServer2 Attachments: beelinepositive.tar.gz, HIVE-2935.1.notest.patch.txt, HIVE-2935.2.notest.patch.txt, HIVE-2935.2.nothrift.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2935) Implement HiveServer2
[
https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13476561#comment-13476561
]
Carl Steinbach commented on HIVE-2935:
--
{quote}
1. metadata such as numRows and rawDataSize get reported as 0 .
eg -
alter_merge_stats
bucketmapjoin2.q
stats18.q - desc formatted stats_part partition query - missing some stats
records in output
union22 - stats records missing
{quote}
We attempted to make the diff masking logic in QFileClient far more selective
about what it elides than the current logic found in QTestUtil. For example,
QTestUtil currently masks the entire output of the DESCRIBE EXTENDED command.
One of the unintended consequences of this effort is that we appear to have
unmasked some non-deterministic behavior in the output of these commands. For
example, I just ran alter_merge_stats.q and observed that it fails because the
order of several table parameters in the output of DESCRIBE EXTENDED is
reversed. The other tests you listed pass when I run them, but I'm guessing
that they would eventually fail for the same reason. I will file a subtask to
cover fixing this.
Implement HiveServer2
-
Key: HIVE-2935
URL: https://issues.apache.org/jira/browse/HIVE-2935
Project: Hive
Issue Type: New Feature
Components: Server Infrastructure
Reporter: Carl Steinbach
Assignee: Carl Steinbach
Labels: HiveServer2
Attachments: beelinepositive.tar.gz, HIVE-2935.1.notest.patch.txt,
HIVE-2935.2.notest.patch.txt, HIVE-2935.2.nothrift.patch.txt
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2935) Implement HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13476568#comment-13476568 ] Carl Steinbach commented on HIVE-2935: -- @Alan: bq. If we start posting patches on top of the existing patches we'll have a mess. Please don't do this. The convention in this project (as well as every other Apache project that I am familiar with) is to post review comments and give the original author time to respond. So far I have been busy splitting this patch into smaller pieces in order to satisfy your previous request. However, I'm more than willing to post a review request for the patch as it stands if that would expedite the review process. It's up to you. Please let me know how you would like proceed. Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: HiveServer2 Attachments: beelinepositive.tar.gz, HIVE-2935.1.notest.patch.txt, HIVE-2935.2.notest.patch.txt, HIVE-2935.2.nothrift.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2935) Implement HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13476583#comment-13476583 ] Carl Steinbach commented on HIVE-2935: -- @Thejas: Thanks for the report about running the tests concurrently. I will try reproducing this on my end tonight and see if I get similar results. bq. 1. metadata such as numRows and rawDataSize get reported as 0 . I think there's a good chance that the table stats subsystem is not thread-safe. I'm inclined to disable these tests and address this issue as a followup. Please let me know if you disagree with this approach. bq. 2. DDL command failed Haven't seen this before. Let me see if I can reproduce it. Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: HiveServer2 Attachments: beelinepositive.tar.gz, HIVE-2935.1.notest.patch.txt, HIVE-2935.2.notest.patch.txt, HIVE-2935.2.nothrift.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3583) 'parameters' component of DESCRIBE EXTENDED output is non-deterministic
[
https://issues.apache.org/jira/browse/HIVE-3583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13476592#comment-13476592
]
Carl Steinbach commented on HIVE-3583:
--
This issue was first observed while running TestBeeLineDriver. It is missed by
TestCliDriver due to over-aggressive diff-masking in QTestUtil.
The 'totalSize' and 'numRows' subcomponents of the 'parameters' field are
frequently reversed:
parameters:{numPartitions=1, numFiles=3, transient_lastDdlTime=!!UNIXTIME!!,
totalSize=636, numRows=0, rawDataSize=0}
vs:
parameters:{numPartitions=1, numFiles=3, transient_lastDdlTime=!!UNIXTIME!!,
numRows=0, totalSize=636, rawDataSize=0}
The easiest way to fix this is sort the parameters alphabetically by key name.
'parameters' component of DESCRIBE EXTENDED output is non-deterministic
---
Key: HIVE-3583
URL: https://issues.apache.org/jira/browse/HIVE-3583
Project: Hive
Issue Type: Sub-task
Components: Query Processor
Reporter: Carl Steinbach
Assignee: Carl Steinbach
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3276) optimize union sub-queries
[ https://issues.apache.org/jira/browse/HIVE-3276?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13473935#comment-13473935 ] Carl Steinbach commented on HIVE-3276: -- @Namit: I added two comments on phabricator. I'm looking at this pretty late so feel free to ignore them. optimize union sub-queries -- Key: HIVE-3276 URL: https://issues.apache.org/jira/browse/HIVE-3276 Project: Hive Issue Type: Bug Reporter: Namit Jain Assignee: Namit Jain Attachments: hive.3276.10.patch, hive.3276.11.patch, HIVE-3276.1.patch, hive.3276.2.patch, hive.3276.3.patch, hive.3276.4.patch, hive.3276.5.patch, hive.3276.6.patch, hive.3276.7.patch, hive.3276.8.patch, hive.3276.9.patch It might be a good idea to optimize simple union queries containing map-reduce jobs in at least one of the sub-qeuries. For eg: a query like: insert overwrite table T1 partition P1 select * from ( subq1 union all subq2 ) u; today creates 3 map-reduce jobs, one for subq1, another for subq2 and the final one for the union. It might be a good idea to optimize this. Instead of creating the union task, it might be simpler to create a move task (or something like a move task), where the outputs of the two sub-queries will be moved to the final directory. This can easily extend to more than 2 sub-queries in the union. This is very useful if there is a select * followed by filesink after the union. This can be independently useful, and also be used to optimize the skewed joins https://cwiki.apache.org/Hive/skewed-join-optimization.html. If there is a select, filter between the union and the filesink, the select and the filter can be moved before the union, and the follow-up job can still be removed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3213) ODBC API enhancements
[ https://issues.apache.org/jira/browse/HIVE-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13473969#comment-13473969 ] Carl Steinbach commented on HIVE-3213: -- @Prasad: Please attach the patch, provide a link to the review request, and change the status to 'patch available' when you're ready to have this reviewed. Thanks. ODBC API enhancements - Key: HIVE-3213 URL: https://issues.apache.org/jira/browse/HIVE-3213 Project: Hive Issue Type: Sub-task Components: ODBC Affects Versions: 0.10.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Fix For: 0.10.0 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1915) authorization on database level is broken.
[ https://issues.apache.org/jira/browse/HIVE-1915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1915: - Component/s: Authorization authorization on database level is broken. -- Key: HIVE-1915 URL: https://issues.apache.org/jira/browse/HIVE-1915 Project: Hive Issue Type: Bug Components: Authorization, Metastore, Security Reporter: He Yongqiang Assignee: He Yongqiang Fix For: 0.7.0 Attachments: HIVE-1915.1.patch, HIVE-1915-2.patch, HIVE-1915-3.patch CREATE DATABASE IF NOT EXISTS test_db COMMENT 'Hive test database'; SHOW DATABASES; grant `drop` on DATABASE test_db to user hive_test_user; grant `select` on DATABASE test_db to user hive_test_user; show grant user hive_test_user on DATABASE test_db; DROP DATABASE IF EXISTS test_db; will fail. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3563) Drop database cascade fails when there are indexes on any tables
[ https://issues.apache.org/jira/browse/HIVE-3563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3563: - Status: Open (was: Patch Available) @Prasad: Please see my comments on RB. Thanks. Drop database cascade fails when there are indexes on any tables Key: HIVE-3563 URL: https://issues.apache.org/jira/browse/HIVE-3563 Project: Hive Issue Type: Bug Components: Indexing, Metastore Affects Versions: 0.9.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-3563-1.patch Drop database cascade fails if any of the table has index. create database db2; use db2; create table tab1 (id int, name string); create index idx1 on table tab1(id) as 'COMPACT' with DEFERRED REBUILD; drop database db2 cascade; -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3554) Hive List Bucketing - Query logic
[ https://issues.apache.org/jira/browse/HIVE-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13473740#comment-13473740 ] Carl Steinbach commented on HIVE-3554: -- @Tim: I added some comments on phabricator. Also, can you please link this ticket to the other list bucketing tickets? Thanks. Hive List Bucketing - Query logic - Key: HIVE-3554 URL: https://issues.apache.org/jira/browse/HIVE-3554 Project: Hive Issue Type: New Feature Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-3554.patch.1, HIVE-3554.patch.2 This is part of efforts for list bucketing feature: https://cwiki.apache.org/Hive/listbucketing.html This patch includes: 1. Query logic: hive chooses right sub-directory instead of partition directory. 2. alter table grammar which is required to support query logic This patch doesn't include list bucketing DML. Main reasons: 1. risk. w/o DML, this patch won't impact any existing hive regression features since no touch on any data manipulation so that very low risk. 2. manageability. w/ DML, patch is getting bigger and hard to review. Removing DML, it's easy to review. We still disable hive feature by default since DML is not in yet. DML will be in follow-up patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3554) Hive List Bucketing - Query logic
[ https://issues.apache.org/jira/browse/HIVE-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3554: - Status: Open (was: Patch Available) Hive List Bucketing - Query logic - Key: HIVE-3554 URL: https://issues.apache.org/jira/browse/HIVE-3554 Project: Hive Issue Type: New Feature Reporter: Gang Tim Liu Assignee: Gang Tim Liu Attachments: HIVE-3554.patch.1, HIVE-3554.patch.2 This is part of efforts for list bucketing feature: https://cwiki.apache.org/Hive/listbucketing.html This patch includes: 1. Query logic: hive chooses right sub-directory instead of partition directory. 2. alter table grammar which is required to support query logic This patch doesn't include list bucketing DML. Main reasons: 1. risk. w/o DML, this patch won't impact any existing hive regression features since no touch on any data manipulation so that very low risk. 2. manageability. w/ DML, patch is getting bigger and hard to review. Removing DML, it's easy to review. We still disable hive feature by default since DML is not in yet. DML will be in follow-up patch. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3545) Add BeeLine source to Hive repository and package in a separate submodule
Carl Steinbach created HIVE-3545: Summary: Add BeeLine source to Hive repository and package in a separate submodule Key: HIVE-3545 URL: https://issues.apache.org/jira/browse/HIVE-3545 Project: Hive Issue Type: Sub-task Components: CLI Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3546) Implement TestBeeLineDriver
Carl Steinbach created HIVE-3546: Summary: Implement TestBeeLineDriver Key: HIVE-3546 URL: https://issues.apache.org/jira/browse/HIVE-3546 Project: Hive Issue Type: Sub-task Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3547) thriftif Ant target should clean old Thrift generated code before generating new code
Carl Steinbach created HIVE-3547: Summary: thriftif Ant target should clean old Thrift generated code before generating new code Key: HIVE-3547 URL: https://issues.apache.org/jira/browse/HIVE-3547 Project: Hive Issue Type: Sub-task Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3548) Implement core HiveServer2 components
Carl Steinbach created HIVE-3548: Summary: Implement core HiveServer2 components Key: HIVE-3548 URL: https://issues.apache.org/jira/browse/HIVE-3548 Project: Hive Issue Type: Sub-task Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3549) Add option to run JobClient in process instead of creating a bin/hadoop subprocess
Carl Steinbach created HIVE-3549: Summary: Add option to run JobClient in process instead of creating a bin/hadoop subprocess Key: HIVE-3549 URL: https://issues.apache.org/jira/browse/HIVE-3549 Project: Hive Issue Type: Sub-task Components: Query Processor Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3449) Speed up ant builds with the ant uptodate task
[ https://issues.apache.org/jira/browse/HIVE-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13471144#comment-13471144 ] Carl Steinbach commented on HIVE-3449: -- @Harsh: You can also do this: % ant clean package -Divy.checkmodified=false Speed up ant builds with the ant uptodate task -- Key: HIVE-3449 URL: https://issues.apache.org/jira/browse/HIVE-3449 Project: Hive Issue Type: Improvement Components: Build Infrastructure Affects Versions: 0.9.0 Reporter: Swarnim Kulkarni Given that the hive build is an enormously lng build (~ 6hrs), it might be very helpful if there are some checkpointing capabilities available to be able to resume a build from failed point and not have to restart everything on a single test failure. One possible way to do this would be to use the ant uptodate task to check set of target files to be more up-to-date than given set of source files and execute a target only if that is true. By default this capability could be off but can be activated with the -Dresume=true argument. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2935) Implement HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470623#comment-13470623 ] Carl Steinbach commented on HIVE-2935: -- @Namit: The only change we made to the Driver class was to wrap a monitor lock around the compile() call in order to serialize access to the compilation phase. I can split this out into a separate patch if you think that would helpful. Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: HiveServer2 Attachments: beelinepositive.tar.gz, HIVE-2935.1.notest.patch.txt, HIVE-2935.2.notest.patch.txt, HIVE-2935.2.nothrift.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2935) Implement HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470641#comment-13470641 ] Carl Steinbach commented on HIVE-2935: -- bq. What do you think about fork beeline as a separate project inside hive. Suchs as hive-beeline. Because a majority of this patch looks to be beeline with some subtle tweeks. In the current version of the patch the BeeLine code is included in the hive-cli package. I think this makes sense since BeeLine is a CLI. On the other hand, if we added a new package for beeline we would be able to avoid adding dependencies on the other Hive JARs that the current CLI mandates we include. Providing this separation will probably be beneficial in the long term so I'll start making the change and will submit this in another ticket. Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: HiveServer2 Attachments: beelinepositive.tar.gz, HIVE-2935.1.notest.patch.txt, HIVE-2935.2.notest.patch.txt, HIVE-2935.2.nothrift.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2935) Implement HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13470649#comment-13470649 ] Carl Steinbach commented on HIVE-2935: -- bq. To follow on Edward's comment, I don't understand why beeline is in the patch. Is it integral to HiveServer2? People need a way to interact with HiveServer2. We could have spent time modifying the existing CLI to work with HS2, but we decided against this approach because a) the HiveCLI has a lot of bugs, and b) we risked introducing new bugs in the process of modifying the CLI to work with both HS1 and HS2. We included BeeLine in this patch because most of the test coverage we have provided for HiveServer2 depends on the new TestBeeLineDriver, which in turn depends on BeeLine. Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: HiveServer2 Attachments: beelinepositive.tar.gz, HIVE-2935.1.notest.patch.txt, HIVE-2935.2.notest.patch.txt, HIVE-2935.2.nothrift.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-1362) column level statistics
[ https://issues.apache.org/jira/browse/HIVE-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-1362: - Status: Open (was: Patch Available) @Shreepadma: More comments on RB. Thanks. column level statistics --- Key: HIVE-1362 URL: https://issues.apache.org/jira/browse/HIVE-1362 Project: Hive Issue Type: Sub-task Components: Statistics Reporter: Ning Zhang Assignee: Shreepadma Venugopalan Attachments: HIVE-1362.1.patch.txt, HIVE-1362.2.patch.txt, HIVE-1362.3.patch.txt, HIVE-1362.4.patch.txt, HIVE-1362-gen_thrift.1.patch.txt, HIVE-1362-gen_thrift.2.patch.txt, HIVE-1362-gen_thrift.3.patch.txt, HIVE-1362-gen_thrift.4.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3501) Track table and keys used in joins and group bys for logging
[ https://issues.apache.org/jira/browse/HIVE-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3501: - Resolution: Fixed Fix Version/s: 0.10.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk. Thanks Sambavi! Track table and keys used in joins and group bys for logging Key: HIVE-3501 URL: https://issues.apache.org/jira/browse/HIVE-3501 Project: Hive Issue Type: Task Components: Query Processor Affects Versions: 0.10.0 Reporter: Sambavi Muthukrishnan Assignee: Sambavi Muthukrishnan Priority: Minor Fix For: 0.10.0 Attachments: table_access_keys.1.patch, table_access_keys.2.patch, table_access_keys.3.patch, table_access_keys.4.patch, table_access_keys.5.patch Original Estimate: 96h Remaining Estimate: 96h For all operators that could benefit from bucketing, it will be useful to keep track of and log the table names and key column names in order for the operator to be converted to the bucketed version. This task is to track this information for joins and group bys when the keys can be directly mapped back to table scans and columns on that table. This information will be tracked on the QueryPlan object so it is available to any pre/post execution hooks for logging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2240) Set hive.security.authorization.createtable.owner.grants to ALL by default
[ https://issues.apache.org/jira/browse/HIVE-2240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2240: - Component/s: Authorization Set hive.security.authorization.createtable.owner.grants to ALL by default -- Key: HIVE-2240 URL: https://issues.apache.org/jira/browse/HIVE-2240 Project: Hive Issue Type: Bug Components: Authorization, Configuration, Security Reporter: Carl Steinbach The default value is currently 'null', which means that users will not have the ability to access tables that they created. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2818) Create table checks the current database privilege
[
https://issues.apache.org/jira/browse/HIVE-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-2818:
-
Component/s: Security
Create table checks the current database privilege
--
Key: HIVE-2818
URL: https://issues.apache.org/jira/browse/HIVE-2818
Project: Hive
Issue Type: Bug
Components: Authorization, Security
Affects Versions: 0.7.1
Reporter: Benyi Wang
Hive seems check the current database to determine the privilege of a
statement when you use fully qualified name like 'database.table'
{code}
hive set hive.security.authorization.enabled=true;
hive create database test_db;
hive grant all on database test_db to user test_user;
hive revoke all on database default from test_user;
hive use default;
hive create table test_db.new_table (id int);
Authorization failed:No privilege 'Create' found for outputs {
database:default}. Use show grant to get more details.
hive use test_db;
hive create table test_db.new_table (id int);
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2538) Database level authorization can be circumvented by first using a database that one has ALL privileges to
[
https://issues.apache.org/jira/browse/HIVE-2538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Carl Steinbach updated HIVE-2538:
-
Component/s: Authorization
Database level authorization can be circumvented by first using a database
that one has ALL privileges to
---
Key: HIVE-2538
URL: https://issues.apache.org/jira/browse/HIVE-2538
Project: Hive
Issue Type: Bug
Components: Authorization, Security
Affects Versions: 0.7.1
Reporter: Sriram Krishnan
You can replicate this bug by recreating my setup, which I describe below:
I have our default database set up to only allow SELECT for user
skrishnan. But user skrishnan has ALL privileges on database skrishnan.
The following works correctly (i.e user shouldn't be able to create a table
in the default database):
hive use default;
OK
Time taken: 0.043 seconds
hive create table skrishnan_test(i int);
Authorization failed:No privilege 'Create' found for outputs {
database:default}. Use show grant to get more details. (*Correct Behavior*)
However, user skrishnan can indeed create tables in the default database by
doing this:
hive use skrishnan;
OK
Time taken: 0.038 seconds
hive create table *default.skrishnan_test(i int);*
OK
Time taken: 0.34 seconds (*Incorrect behavior*)
That means that the database level authorization is basically circumvented by
first using a database that a user has all privileges to. And then using the
fully qualified table name (db\_name.table\_name) for a database that a user
doesn't have permissions to.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2094) CREATE and DROP DATABASE doesn't check user permission for doing it
[ https://issues.apache.org/jira/browse/HIVE-2094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2094: - Component/s: Authorization CREATE and DROP DATABASE doesn't check user permission for doing it --- Key: HIVE-2094 URL: https://issues.apache.org/jira/browse/HIVE-2094 Project: Hive Issue Type: Bug Components: Authorization, Metastore, Security Reporter: Siying Dong Assignee: He Yongqiang We need to make sure only users with system permission to do it. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3532) ALTER TABLE SET LOCATION can be used to circumvent authorization checks
Carl Steinbach created HIVE-3532: Summary: ALTER TABLE SET LOCATION can be used to circumvent authorization checks Key: HIVE-3532 URL: https://issues.apache.org/jira/browse/HIVE-3532 Project: Hive Issue Type: Bug Components: Authorization, Security Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2935) Implement HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2935: - Attachment: HIVE-2935.2.nothrift.patch.txt Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: HiveServer2 Attachments: beelinepositive.tar.gz, HIVE-2935.1.notest.patch.txt, HIVE-2935.2.notest.patch.txt, HIVE-2935.2.nothrift.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2935) Implement HiveServer2
[
https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13469526#comment-13469526
]
Carl Steinbach commented on HIVE-2935:
--
The attached patches contain a complete, working version of HiveServer2. Before
going into details about the contents of the patch I want to first quickly
review how to apply the patch and try out the new server:
1) Download and apply HIVE-2935.2.nothrift.patch.txt
2) Run the Thrift code generator (make sure $THRIFT_HOME points to thrift
version 0.7.0)
{{% ant thriftif -Dthrift.home=$THRIFT_HOME}}
3) Optionally download and unpack the beeline test outputs in the
ql/src/test/results directory.
4) Build Hive:
{{% ant clean package}}
5) Start HiveServer2
{{% hiveserver2}}
6) From another window start the beeline CLI and connect to HiveServer2:
{noformat}
% beeline
Hive version 0.10.0-SNAPSHOT by Apache
beeline !connect jdbc:hive2://localhost:1 scott tiger
org.apache.hive.jdbc.HiveDriver
!connect jdbc:hive2://localhost:1 scott tiger
org.apache.hive.jdbc.HiveDriver
Connecting to jdbc:hive2://localhost:1
Connected to: Hive (version 0.10.0)
Driver: Hive (version 0.10.0-SNAPSHOT)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:1 show tables;
show tables;
+---+
| tab_name |
+---+
| primitives|
| src |
| src1 |
| src_json |
| src_sequencefile |
| src_thrift|
| srcbucket |
| srcbucket2|
| srcpart |
+---+
9 rows selected (1.079 seconds)
{noformat}
If you downloaded the test outputs you can also try running the new
BeeLineTestDriver:
% ant test -Dtestcase=TestBeeLineDriver -Dtest.concurrency.num.threads=10
Patch contents:
* {{cli/src/java/org/apache/hive/cli/beeline/*.java}}
** This a JDBC CLI for Hive based on the SQLLine CLI. An earlier version of
BeeLine which had a dependency on SQLLine was previously added in HIVE-3100.
However, while working on HS2 we discovered bugs in SQLLine that we needed to
fix, and also needed to build in some extensions in order to support the
BeeLine test driver. Adding the code directly to Hive seemed like the best
option since the upstream project is no longer actively maintained.
* {{cli/src/java/org/apache/hive/cli/beeline/util/QFileClient.java}}
** This is a beeline test client used by TestBeeLineDriver. This class also
provides an improved version of the output masking functionality currently
located in QTestUtil.
* {{common/src/java/org/apache/hive/common/util/*.java}}
** Utility classes borrowed from Hadoop.
* {{data/files/types/primitives/*}}
** Data files for a 'primitives' table that contains all Hive primitive types
along with NULLs.
* {{data/scripts/q_test_*.sql}}
** Test initialization scripts that are used to create and initialize all of
the tables that are referenced by CliDriver tests. Called from QFileClient.
* {{jdbc/src/java/org/apache/hive/jdbc/*}}
** The HS2 JDBC driver.
* {{service/if/cli_service.thrift}}
** The CliService Thrift IDL file.
* {{service/src/java/org/apache/hive/service/*}}
** Service infrastructure classes borrowed from o.a.hadoop.yarn.service
* {{service/src/java/org/apache/hive/service/auth/*}}
** Kerberos/LDAP/SASL auth code for HS2
* {{service/src/java/org/apache/hive/service/cli/*}}
** Implementation classes for CLIService. These classes form the core of
HiveServer2.
* {{service/src/java/org/apache/hive/service/server/HiveServer2.java}}
** HiveServer2 class. At the moment it just starts the CLIService, but we plan
to extend it with other pluggable services in the future.
* {{testutils/junit/*}
** Utility classes for running concurrent JUnit tests. Most of this code was
borrowed from tempus-fugit.
This code was a collaborative effort between me and my colleague Prasad
Mujumdar. Prasad is wholly responsible for the new JDBC driver and
authorization code. We worked together on the other parts.
We plan to break this monolithic patch up into several smaller patches in order
to make the review process easier. Here's our initial plan for how to do this:
# Review/commit the beeline CLI (note that this can be used with the existing
JDBC driver and HiveServer1)
# Review/commit the BeeLine test driver and new test outputs
# Review/commit the HiveServer2 core component.
# Review/commit the HiveServer2 JDBC driver.
We would appreciate receiving feedback from the Hive committers about whether
or not this plan makes sense.
Thanks!
Implement HiveServer2
-
Key: HIVE-2935
URL: https://issues.apache.org/jira/browse/HIVE-2935
Project: Hive
Issue Type: New Feature
Components: Server Infrastructure
Reporter: Carl Steinbach
Assignee: Carl Steinbach
Labels:
[jira] [Commented] (HIVE-3531) Simple lock manager for dedicated hive server
[ https://issues.apache.org/jira/browse/HIVE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13469552#comment-13469552 ] Carl Steinbach commented on HIVE-3531: -- @Navis: This is awesome. We were planning to write something like this for HiveServer2, but you beat us to it :) I'll try to review this in the next day or so. Thanks! Simple lock manager for dedicated hive server - Key: HIVE-3531 URL: https://issues.apache.org/jira/browse/HIVE-3531 Project: Hive Issue Type: Improvement Components: Server Infrastructure Reporter: Navis Assignee: Navis Priority: Trivial Attachments: HIVE-3531.D5871.1.patch In many cases, we uses hive server as a sole proxy for executing all the queries. For that, current default lock manager based on zookeeper seemed a little heavy. Simple in-memory lock manager could be enough. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3537) release locks at the end of move tasks
[ https://issues.apache.org/jira/browse/HIVE-3537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3537: - Component/s: Locking release locks at the end of move tasks -- Key: HIVE-3537 URL: https://issues.apache.org/jira/browse/HIVE-3537 Project: Hive Issue Type: Bug Components: Locking, Query Processor Reporter: Namit Jain Assignee: Namit Jain Look at HIVE-3106 for details. In order to make sure that concurrency is not an issue for multi-table inserts, the current option is to introduce a dependency task, which thereby delays the creation of all partitions. It would be desirable to release the locks for the outputs as soon as the move task is completed. That way, for multi-table inserts, the concurrency can be enabled without delaying any table. Currently, the movetask contains a input/output, but they do not seem to be populated correctly. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3036) hive should support BigDecimal datatype
[ https://issues.apache.org/jira/browse/HIVE-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3036: - Component/s: Types hive should support BigDecimal datatype --- Key: HIVE-3036 URL: https://issues.apache.org/jira/browse/HIVE-3036 Project: Hive Issue Type: New Feature Components: Query Processor, Types Affects Versions: 0.7.1, 0.8.0, 0.8.1 Reporter: Anurag Tangri Fix For: 0.10.0 hive has support for big int but people have use cases where they need decimal precision to a big value. Values in question are like decimal(x,y). for eg. decimal of form (17,6) which cannot be represented by float/double. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3501) Track table and keys used in joins and group bys for logging
[ https://issues.apache.org/jira/browse/HIVE-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13467523#comment-13467523 ] Carl Steinbach commented on HIVE-3501: -- +1. Will commit if tests pass. Track table and keys used in joins and group bys for logging Key: HIVE-3501 URL: https://issues.apache.org/jira/browse/HIVE-3501 Project: Hive Issue Type: Task Components: Query Processor Affects Versions: 0.10.0 Reporter: Sambavi Muthukrishnan Assignee: Sambavi Muthukrishnan Priority: Minor Attachments: table_access_keys.1.patch, table_access_keys.2.patch, table_access_keys.3.patch, table_access_keys.4.patch, table_access_keys.5.patch Original Estimate: 96h Remaining Estimate: 96h For all operators that could benefit from bucketing, it will be useful to keep track of and log the table names and key column names in order for the operator to be converted to the bucketed version. This task is to track this information for joins and group bys when the keys can be directly mapped back to table scans and columns on that table. This information will be tracked on the QueryPlan object so it is available to any pre/post execution hooks for logging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2935) Implement HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2935: - Attachment: HIVE-2935.1.notest.patch.txt HIVE-2935.1.notest.patch.txt: patch w/o new qfile test outputs. Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: HiveServer2 Attachments: HIVE-2935.1.notest.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2935) Implement HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2935: - Attachment: HIVE-2935.2.notest.patch.txt Second patch excludes some test outputs missed in the first patch. Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: HiveServer2 Attachments: HIVE-2935.1.notest.patch.txt, HIVE-2935.2.notest.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3501) Track table and keys used in joins and group bys for logging
[ https://issues.apache.org/jira/browse/HIVE-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13467931#comment-13467931 ] Carl Steinbach commented on HIVE-3501: -- @Sambavi: Please change the status of this ticket to patch submitted. Thanks. Track table and keys used in joins and group bys for logging Key: HIVE-3501 URL: https://issues.apache.org/jira/browse/HIVE-3501 Project: Hive Issue Type: Task Components: Query Processor Affects Versions: 0.10.0 Reporter: Sambavi Muthukrishnan Assignee: Sambavi Muthukrishnan Priority: Minor Attachments: table_access_keys.1.patch, table_access_keys.2.patch, table_access_keys.3.patch, table_access_keys.4.patch, table_access_keys.5.patch Original Estimate: 96h Remaining Estimate: 96h For all operators that could benefit from bucketing, it will be useful to keep track of and log the table names and key column names in order for the operator to be converted to the bucketed version. This task is to track this information for joins and group bys when the keys can be directly mapped back to table scans and columns on that table. This information will be tracked on the QueryPlan object so it is available to any pre/post execution hooks for logging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2935) Implement HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-2935: - Attachment: beelinepositive.tar.gz Uploading the new test outputs separately since the combined patch exceeds the 10MB size limit. Untar this file in the ql/src/test/results directory. Implement HiveServer2 - Key: HIVE-2935 URL: https://issues.apache.org/jira/browse/HIVE-2935 Project: Hive Issue Type: New Feature Components: Server Infrastructure Reporter: Carl Steinbach Assignee: Carl Steinbach Labels: HiveServer2 Attachments: beelinepositive.tar.gz, HIVE-2935.1.notest.patch.txt, HIVE-2935.2.notest.patch.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3501) Track table and keys used in joins and group bys for logging
[ https://issues.apache.org/jira/browse/HIVE-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13468145#comment-13468145 ] Carl Steinbach commented on HIVE-3501: -- @Sambavi: I'm a little behind schedule on this. Just kicked off the test run a minute ago. If everything goes ok I should have this committed later tonight. Sorry for the delay. Track table and keys used in joins and group bys for logging Key: HIVE-3501 URL: https://issues.apache.org/jira/browse/HIVE-3501 Project: Hive Issue Type: Task Components: Query Processor Affects Versions: 0.10.0 Reporter: Sambavi Muthukrishnan Assignee: Sambavi Muthukrishnan Priority: Minor Attachments: table_access_keys.1.patch, table_access_keys.2.patch, table_access_keys.3.patch, table_access_keys.4.patch, table_access_keys.5.patch Original Estimate: 96h Remaining Estimate: 96h For all operators that could benefit from bucketing, it will be useful to keep track of and log the table names and key column names in order for the operator to be converted to the bucketed version. This task is to track this information for joins and group bys when the keys can be directly mapped back to table scans and columns on that table. This information will be tracked on the QueryPlan object so it is available to any pre/post execution hooks for logging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3523) Hive info logging is broken
[ https://issues.apache.org/jira/browse/HIVE-3523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3523: - Component/s: Logging Hive info logging is broken --- Key: HIVE-3523 URL: https://issues.apache.org/jira/browse/HIVE-3523 Project: Hive Issue Type: Bug Components: Logging Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Carl Steinbach Hive Info logging is broken on trunk. hive -hiveconf hive.root.logger=INFO,console doesn't print the output of LOG.info statements to the console. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-3523) Hive info logging is broken
[ https://issues.apache.org/jira/browse/HIVE-3523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reassigned HIVE-3523: Assignee: Carl Steinbach Hive info logging is broken --- Key: HIVE-3523 URL: https://issues.apache.org/jira/browse/HIVE-3523 Project: Hive Issue Type: Bug Components: Logging Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Carl Steinbach Hive Info logging is broken on trunk. hive -hiveconf hive.root.logger=INFO,console doesn't print the output of LOG.info statements to the console. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3523) Hive info logging is broken
[
https://issues.apache.org/jira/browse/HIVE-3523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13468241#comment-13468241
]
Carl Steinbach commented on HIVE-3523:
--
This bug is caused by HIVE-3505 which modified the hive-log4j.properties file.
Before HIVE-3505 the hive-log4j.properties file looked like this:
{noformat}
hive.root.logger=WARN,DRFA
hive.log.dir=/tmp/${user.name}
...
# Logging Threshold
log4j.threshhold=WARN
{noformat}
And after HIVE-3505 it looks like this:
{noformat}
hive.log.threshold=WARN
hive.root.logger=${hive.log.threshold},DRFA
...
# Logging Threshold
log4j.threshold=${hive.log.threshold}
{noformat}
One not so obvious change is that we corrected a spelling mistake, changing
log4j.thresshold to log4j.threshold. The fact that log4j.threshold was
previously misspelled meant that log4j had been using the default threshold
value ALL, which is equivalent to no threshold at all. HIVE-3505 fixed the
spelling mistake, which caused log4j to start using the new threshold value
WARN, which explains why INFO level messages are getting filtered out even when
hive.root.logger is set to INFO,console.
It's possible to work around this problem right now by setting both
hive.log.threshold and hive.root.logger. For example:
hive -hiveconf hive.log.threshold=INFO -hiveconf hive.root.logger=INFO,console
Hive info logging is broken
---
Key: HIVE-3523
URL: https://issues.apache.org/jira/browse/HIVE-3523
Project: Hive
Issue Type: Bug
Components: Logging
Affects Versions: 0.10.0
Reporter: Shreepadma Venugopalan
Assignee: Carl Steinbach
Hive Info logging is broken on trunk. hive -hiveconf
hive.root.logger=INFO,console doesn't print the output of LOG.info statements
to the console.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3523) Hive info logging is broken
[ https://issues.apache.org/jira/browse/HIVE-3523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3523: - Attachment: HIVE-3523.1.patch.txt Hive info logging is broken --- Key: HIVE-3523 URL: https://issues.apache.org/jira/browse/HIVE-3523 Project: Hive Issue Type: Bug Components: Logging Affects Versions: 0.10.0 Reporter: Shreepadma Venugopalan Assignee: Carl Steinbach Attachments: HIVE-3523.1.patch.txt Hive Info logging is broken on trunk. hive -hiveconf hive.root.logger=INFO,console doesn't print the output of LOG.info statements to the console. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3517) Provide README file for Oracle backed Metastore
Carl Steinbach created HIVE-3517: Summary: Provide README file for Oracle backed Metastore Key: HIVE-3517 URL: https://issues.apache.org/jira/browse/HIVE-3517 Project: Hive Issue Type: Bug Components: Metastore Reporter: Carl Steinbach We currently provide README files for Derby, MySQL, and Derby. We should also provide a README file for Oracle. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3437) 0.23 compatibility: fix unit tests when building against 0.23
[ https://issues.apache.org/jira/browse/HIVE-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13467171#comment-13467171 ] Carl Steinbach commented on HIVE-3437: -- @Chris: I left some more comments on phabricator. Thanks. 0.23 compatibility: fix unit tests when building against 0.23 - Key: HIVE-3437 URL: https://issues.apache.org/jira/browse/HIVE-3437 Project: Hive Issue Type: Bug Components: Tests Affects Versions: 0.9.0, 0.10.0 Reporter: Chris Drome Assignee: Chris Drome Fix For: 0.9.0, 0.10.0 Attachments: HIVE-3437-0.9-1.patch, HIVE-3437-0.9-2.patch, HIVE-3437-0.9.patch, HIVE-3437-trunk-1.patch, HIVE-3437-trunk-2.patch, HIVE-3437-trunk.patch Many unit tests fail as a result of building the code against hadoop 0.23. Initial focus will be to fix 0.9. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3433) Implement CUBE and ROLLUP operators in Hive
[ https://issues.apache.org/jira/browse/HIVE-3433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3433: - Status: Open (was: Patch Available) @Namit: I left some comments on phabricator. Thanks. Implement CUBE and ROLLUP operators in Hive --- Key: HIVE-3433 URL: https://issues.apache.org/jira/browse/HIVE-3433 Project: Hive Issue Type: New Feature Components: Query Processor Reporter: Sambavi Muthukrishnan Assignee: Namit Jain Attachments: hive.3433.1.patch -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (HIVE-3521) Concurrent metastore calls provoke Datanucleus IllegalStateException: Table object has not been been initialised
Carl Steinbach created HIVE-3521: Summary: Concurrent metastore calls provoke Datanucleus IllegalStateException: Table object has not been been initialised Key: HIVE-3521 URL: https://issues.apache.org/jira/browse/HIVE-3521 Project: Hive Issue Type: Bug Components: Metastore Affects Versions: 0.9.0 Reporter: Carl Steinbach -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3521) Concurrent metastore calls provoke Datanucleus IllegalStateException: Table object has not been been initialised
[
https://issues.apache.org/jira/browse/HIVE-3521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13467293#comment-13467293
]
Carl Steinbach commented on HIVE-3521:
--
I encountered this error while running a test fires metastore calls from
approximately ten threads at the same time.
Here is the exception stack trace:
{noformat}
12/10/01 14:46:53 ERROR hive.log: java.lang.IllegalStateException: Table object
has not been been initialised : `TBLS`
at
org.datanucleus.store.rdbms.table.AbstractTable.assertIsInitialized(AbstractTable.java:625)
at
org.datanucleus.store.rdbms.table.ClassTable.getMemberMapping(ClassTable.java:2996)
at
org.datanucleus.store.mapped.expression.LogicSetExpression.newFieldExpression(LogicSetExpression.java:195)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileIdentifier(JDOQLQueryCompiler.java:953)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compilePrimary(JDOQLQueryCompiler.java:774)
at
org.datanucleus.store.rdbms.query.legacy.QueryCompiler.compileUnaryExpressionNotPlusMinus(QueryCompiler.java:1333)
at
org.datanucleus.store.rdbms.query.legacy.QueryCompiler.compileUnaryExpression(QueryCompiler.java:1314)
at
org.datanucleus.store.rdbms.query.legacy.QueryCompiler.compileMultiplicativeExpression(QueryCompiler.java:1267)
at
org.datanucleus.store.rdbms.query.legacy.QueryCompiler.compileAdditiveExpression(QueryCompiler.java:1244)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileRelationalExpression(JDOQLQueryCompiler.java:687)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileEqualityExpression(JDOQLQueryCompiler.java:659)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileAndExpression(JDOQLQueryCompiler.java:647)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileExclusiveOrExpression(JDOQLQueryCompiler.java:635)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileInclusiveOrExpression(JDOQLQueryCompiler.java:623)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileConditionalAndExpression(JDOQLQueryCompiler.java:611)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileConditionalOrExpression(JDOQLQueryCompiler.java:593)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileExpression(JDOQLQueryCompiler.java:570)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compileExpressionFromString(JDOQLQueryCompiler.java:550)
at
org.datanucleus.store.rdbms.query.legacy.QueryCompiler.compileExpressionsFromString(QueryCompiler.java:1221)
at
org.datanucleus.store.rdbms.query.legacy.QueryCompiler.compileResult(QueryCompiler.java:876)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.performCompile(JDOQLQueryCompiler.java:246)
at
org.datanucleus.store.rdbms.query.legacy.QueryCompiler.executionCompile(QueryCompiler.java:374)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQueryCompiler.compile(JDOQLQueryCompiler.java:225)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.compileInternal(JDOQLQuery.java:175)
at org.datanucleus.store.query.Query.executeQuery(Query.java:1628)
at
org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.executeQuery(JDOQLQuery.java:245)
at org.datanucleus.store.query.Query.executeWithArray(Query.java:1499)
at org.datanucleus.jdo.JDOQuery.execute(JDOQuery.java:243)
at
org.apache.hadoop.hive.metastore.ObjectStore.getTables(ObjectStore.java:759)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hive.metastore.RetryingRawStore.invoke(RetryingRawStore.java:111)
at $Proxy0.getTables(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_tables(HiveMetaStore.java:1928)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTables(HiveMetaStoreClient.java:786)
at
org.apache.hadoop.hive.ql.metadata.Hive.getTablesByPattern(Hive.java:976)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllTables(Hive.java:950)
at org.apache.hadoop.hive.ql.exec.DDLTask.showTables(DDLTask.java:2007)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:324)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at
[jira] [Commented] (HIVE-3501) Track table and keys used in joins and group bys for logging
[ https://issues.apache.org/jira/browse/HIVE-3501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13467460#comment-13467460 ] Carl Steinbach commented on HIVE-3501: -- I added a couple minor comments too. Looks really good overall. Track table and keys used in joins and group bys for logging Key: HIVE-3501 URL: https://issues.apache.org/jira/browse/HIVE-3501 Project: Hive Issue Type: Task Components: Query Processor Affects Versions: 0.10.0 Reporter: Sambavi Muthukrishnan Assignee: Sambavi Muthukrishnan Priority: Minor Attachments: table_access_keys.1.patch, table_access_keys.2.patch, table_access_keys.3.patch, table_access_keys.4.patch Original Estimate: 96h Remaining Estimate: 96h For all operators that could benefit from bucketing, it will be useful to keep track of and log the table names and key column names in order for the operator to be converted to the bucketed version. This task is to track this information for joins and group bys when the keys can be directly mapped back to table scans and columns on that table. This information will be tracked on the QueryPlan object so it is available to any pre/post execution hooks for logging. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2206) add a new optimizer for query correlation discovery and optimization
[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466576#comment-13466576 ] Carl Steinbach commented on HIVE-2206: -- @Yongqiang: I don't see a +1 vote in this JIRA. According to the project bylaws (https://cwiki.apache.org/confluence/display/Hive/Bylaws) this patch should not have been committed. Please back this patch out. Thanks. add a new optimizer for query correlation discovery and optimization Key: HIVE-2206 URL: https://issues.apache.org/jira/browse/HIVE-2206 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.10.0 Reporter: He Yongqiang Assignee: Yin Huai Attachments: HIVE-2206.10-r1384442.patch.txt, HIVE-2206.11-r1385084.patch.txt, HIVE-2206.12-r1386996.patch.txt, HIVE-2206.13-r1389072.patch.txt, HIVE-2206.14-r1389704.patch.txt, HIVE-2206.1.patch.txt, HIVE-2206.2.patch.txt, HIVE-2206.3.patch.txt, HIVE-2206.4.patch.txt, HIVE-2206.5-1.patch.txt, HIVE-2206.5.patch.txt, HIVE-2206.6.patch.txt, HIVE-2206.7.patch.txt, HIVE-2206.8.r1224646.patch.txt, HIVE-2206.8-r1237253.patch.txt, testQueries.2.q, YSmartPatchForHive.patch reference: http://www.cse.ohio-state.edu/hpcs/WWW/HTML/publications/papers/TR-11-7.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2206) add a new optimizer for query correlation discovery and optimization
[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466582#comment-13466582 ] Carl Steinbach commented on HIVE-2206: -- @Yongqiang: Sorry, but that's not the way it works. You vote +1 first, wait 24 hours, and then commit the patch. This is all covered in the project bylaws. Please revert this patch. Thanks. add a new optimizer for query correlation discovery and optimization Key: HIVE-2206 URL: https://issues.apache.org/jira/browse/HIVE-2206 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.10.0 Reporter: He Yongqiang Assignee: Yin Huai Attachments: HIVE-2206.10-r1384442.patch.txt, HIVE-2206.11-r1385084.patch.txt, HIVE-2206.12-r1386996.patch.txt, HIVE-2206.13-r1389072.patch.txt, HIVE-2206.14-r1389704.patch.txt, HIVE-2206.1.patch.txt, HIVE-2206.2.patch.txt, HIVE-2206.3.patch.txt, HIVE-2206.4.patch.txt, HIVE-2206.5-1.patch.txt, HIVE-2206.5.patch.txt, HIVE-2206.6.patch.txt, HIVE-2206.7.patch.txt, HIVE-2206.8.r1224646.patch.txt, HIVE-2206.8-r1237253.patch.txt, testQueries.2.q, YSmartPatchForHive.patch reference: http://www.cse.ohio-state.edu/hpcs/WWW/HTML/publications/papers/TR-11-7.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HIVE-2206) add a new optimizer for query correlation discovery and optimization
[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-2206: -- add a new optimizer for query correlation discovery and optimization Key: HIVE-2206 URL: https://issues.apache.org/jira/browse/HIVE-2206 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.10.0 Reporter: He Yongqiang Assignee: Yin Huai Attachments: HIVE-2206.10-r1384442.patch.txt, HIVE-2206.11-r1385084.patch.txt, HIVE-2206.12-r1386996.patch.txt, HIVE-2206.13-r1389072.patch.txt, HIVE-2206.14-r1389704.patch.txt, HIVE-2206.1.patch.txt, HIVE-2206.2.patch.txt, HIVE-2206.3.patch.txt, HIVE-2206.4.patch.txt, HIVE-2206.5-1.patch.txt, HIVE-2206.5.patch.txt, HIVE-2206.6.patch.txt, HIVE-2206.7.patch.txt, HIVE-2206.8.r1224646.patch.txt, HIVE-2206.8-r1237253.patch.txt, testQueries.2.q, YSmartPatchForHive.patch reference: http://www.cse.ohio-state.edu/hpcs/WWW/HTML/publications/papers/TR-11-7.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2206) add a new optimizer for query correlation discovery and optimization
[ https://issues.apache.org/jira/browse/HIVE-2206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466631#comment-13466631 ] Carl Steinbach commented on HIVE-2206: -- bq. I did not see a 24 hours waiting on the bylaw page? This is specified in the minimum length column in the table that appears in the Actions section of the bylaws document. We could definitely make this easier to undertand, but all of the other committers already follow the convention that you +1 a patch before committing it, and allow some time to elapse in between those two actions in order to give other people a chance to weigh in. add a new optimizer for query correlation discovery and optimization Key: HIVE-2206 URL: https://issues.apache.org/jira/browse/HIVE-2206 Project: Hive Issue Type: New Feature Components: Query Processor Affects Versions: 0.10.0 Reporter: He Yongqiang Assignee: Yin Huai Attachments: HIVE-2206.10-r1384442.patch.txt, HIVE-2206.11-r1385084.patch.txt, HIVE-2206.12-r1386996.patch.txt, HIVE-2206.13-r1389072.patch.txt, HIVE-2206.14-r1389704.patch.txt, HIVE-2206.1.patch.txt, HIVE-2206.2.patch.txt, HIVE-2206.3.patch.txt, HIVE-2206.4.patch.txt, HIVE-2206.5-1.patch.txt, HIVE-2206.5.patch.txt, HIVE-2206.6.patch.txt, HIVE-2206.7.patch.txt, HIVE-2206.8.r1224646.patch.txt, HIVE-2206.8-r1237253.patch.txt, testQueries.2.q, YSmartPatchForHive.patch reference: http://www.cse.ohio-state.edu/hpcs/WWW/HTML/publications/papers/TR-11-7.pdf -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-3500) To add instrumentation to capture if there is skew in reducers
[ https://issues.apache.org/jira/browse/HIVE-3500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-3500: - Resolution: Fixed Fix Version/s: 0.10.0 Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk. Thanks Arun. To add instrumentation to capture if there is skew in reducers -- Key: HIVE-3500 URL: https://issues.apache.org/jira/browse/HIVE-3500 Project: Hive Issue Type: Improvement Components: Query Processor Affects Versions: 0.10.0 Reporter: Arun Dobriyal Assignee: Arun Dobriyal Priority: Minor Fix For: 0.10.0 Attachments: Hive-3500.1.patch.txt, Hive-3500.2.patch.txt, Hive-3500.3.patch.txt, Hive-3500.4.patch.txt To add the reducer skew information. To log min/max/mean/(standard deviation) of the reducers run times in the QueryPlan. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Reopened] (HIVE-2618) Describe partition returns table columns but should return partition columns
[ https://issues.apache.org/jira/browse/HIVE-2618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach reopened HIVE-2618: -- Describe partition returns table columns but should return partition columns Key: HIVE-2618 URL: https://issues.apache.org/jira/browse/HIVE-2618 Project: Hive Issue Type: Bug Reporter: Kevin Wilfong Assignee: Namit Jain If a partitioned table and some partitions are created, and then the table is altered adding a columns, if describe is called on the partitions created before the columns were added it will show the new columns, even though it should not. In particular, in the metastore, the partition will not have these columns. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
