[jira] [Updated] (HIVE-5343) Add equals method to ObjectInspectorUtils
[ https://issues.apache.org/jira/browse/HIVE-5343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Navis updated HIVE-5343: Attachment: HIVE-5343.3.patch.txt Add equals method to ObjectInspectorUtils - Key: HIVE-5343 URL: https://issues.apache.org/jira/browse/HIVE-5343 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Trivial Attachments: D13053.1.patch, HIVE-5343.2.patch.txt, HIVE-5343.3.patch.txt Might provide shortcut for some use cases. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7231) Improve ORC padding
[ https://issues.apache.org/jira/browse/HIVE-7231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-7231: - Labels: TODOC14 orcfile (was: orcfile) Improve ORC padding --- Key: HIVE-7231 URL: https://issues.apache.org/jira/browse/HIVE-7231 Project: Hive Issue Type: Improvement Components: File Formats Affects Versions: 0.14.0 Reporter: Prasanth J Assignee: Prasanth J Labels: TODOC14, orcfile Fix For: 0.14.0 Attachments: HIVE-7231.1.patch, HIVE-7231.2.patch, HIVE-7231.3.patch, HIVE-7231.4.patch, HIVE-7231.5.patch, HIVE-7231.6.patch, HIVE-7231.7.patch, HIVE-7231.8.patch Current ORC padding is not optimal because of fixed stripe sizes within block. The padding overhead will be significant in some cases. Also padding percentage relative to stripe size is not configurable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7231) Improve ORC padding
[ https://issues.apache.org/jira/browse/HIVE-7231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053398#comment-14053398 ] Lefty Leverenz commented on HIVE-7231: -- Facepalm! Now that the patch is committed I've finally noticed that hive.exec.orc.block.padding.tolerance is not a percentage but a decimal fraction. For example, with a 64 MB stripe size the default 0.05 gives 3.2 MB tolerance (0.05 * 64, not 0.05% of 64). This is only a tech-writer's quibble which isn't likely to confuse anyone. I'll explain it in the wiki and put a request in HIVE-6586 to fix it with HIVE-6037. Improve ORC padding --- Key: HIVE-7231 URL: https://issues.apache.org/jira/browse/HIVE-7231 Project: Hive Issue Type: Improvement Components: File Formats Affects Versions: 0.14.0 Reporter: Prasanth J Assignee: Prasanth J Labels: TODOC14, orcfile Fix For: 0.14.0 Attachments: HIVE-7231.1.patch, HIVE-7231.2.patch, HIVE-7231.3.patch, HIVE-7231.4.patch, HIVE-7231.5.patch, HIVE-7231.6.patch, HIVE-7231.7.patch, HIVE-7231.8.patch Current ORC padding is not optimal because of fixed stripe sizes within block. The padding overhead will be significant in some cases. Also padding percentage relative to stripe size is not configurable. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6586) Add new parameters to HiveConf.java after commit HIVE-6037 (also fix typos)
[ https://issues.apache.org/jira/browse/HIVE-6586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053403#comment-14053403 ] Lefty Leverenz commented on HIVE-6586: -- HIVE-7231 adds hive.exec.orc.default.block.size hive.exec.orc.block.padding.tolerance in 0.14.0 with descriptions in hive-default.xml.template. It also changes the default for hive.exec.orc.default.stripe.size to 64L * 1024 * 1024 (HiveConf.java) or 67108864 (template, same value). Note: The description of hive.exec.orc.block.padding.tolerance is slightly inaccurate -- instead of saying as a percentage of stripe size it should say as a decimal fraction of stripe size. Add new parameters to HiveConf.java after commit HIVE-6037 (also fix typos) --- Key: HIVE-6586 URL: https://issues.apache.org/jira/browse/HIVE-6586 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Reporter: Lefty Leverenz Labels: TODOC14 HIVE-6037 puts the definitions of configuration parameters into the HiveConf.java file, but several recent jiras for release 0.13.0 introduce new parameters that aren't in HiveConf.java yet and some parameter definitions need to be altered for 0.13.0. This jira will patch HiveConf.java after HIVE-6037 gets committed. Also, four typos patched in HIVE-6582 need to be fixed in the new HiveConf.java. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5690) Support subquery for single sourced multi query
[
https://issues.apache.org/jira/browse/HIVE-5690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053420#comment-14053420
]
Hive QA commented on HIVE-5690:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12654244/HIVE-5690.6.patch.txt
{color:red}ERROR:{color} -1 due to 36 failed/errored test(s), 5678 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_alter_partition_coltype
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_combine2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_11
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_9
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_multiskew_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_multiskew_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_multiskew_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_oneskew_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_oneskew_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_query_oneskew_3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_sample8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_show_create_table_serde
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_exists_explain_rewrite
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_in_explain_rewrite
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_transform_ppr1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_transform_ppr2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_truncate_column_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_ppr
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket4
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket5
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_disable_merge_for_bucketing
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_list_bucket_dml_10
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_reduce_deduplicate
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/687/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/687/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-687/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 36 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12654244
Support subquery for single sourced multi query
---
Key: HIVE-5690
URL: https://issues.apache.org/jira/browse/HIVE-5690
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: D13791.1.patch, HIVE-5690.2.patch.txt,
HIVE-5690.3.patch.txt, HIVE-5690.4.patch.txt, HIVE-5690.5.patch.txt,
HIVE-5690.6.patch.txt
Single sourced multi (insert) query is very useful for various ETL processes
but it does not allow subqueries included. For example,
{noformat}
explain from src
insert overwrite table x1 select * from (select distinct key,value) b order
by key
insert overwrite table x2 select * from (select distinct key,value) c order
by value;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7079) Hive logs errors about missing tables when parsing CTE expressions
[
https://issues.apache.org/jira/browse/HIVE-7079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053470#comment-14053470
]
Hive QA commented on HIVE-7079:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12654246/HIVE-7079.2.patch.txt
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 5692 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/688/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/688/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-688/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12654246
Hive logs errors about missing tables when parsing CTE expressions
--
Key: HIVE-7079
URL: https://issues.apache.org/jira/browse/HIVE-7079
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.13.0
Reporter: Craig Condit
Assignee: Navis
Priority: Minor
Attachments: HIVE-7079.1.patch.txt, HIVE-7079.2.patch.txt
Given a query containing common table expressions (CTE) such as:
WITH a AS (SELECT ...), b AS (SELECT ...)
SELECT * FROM a JOIN b on a.col = b.col ...;
Hive CLI executes the query, but logs stack traces at ERROR level during
query parsing:
{noformat}
ERROR metadata.Hive: NoSuchObjectException(message:ccondit.a table not found)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29338)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29306)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result.read(ThriftHiveMetastore.java:29237)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table(ThriftHiveMetastore.java:1036)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table(ThriftHiveMetastore.java:1022)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:997)
at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
at com.sun.proxy.$Proxy7.getTable(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:967)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:909)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1223)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1192)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9209)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:391)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:291)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:944)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1009)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:880)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:870)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
[jira] [Updated] (HIVE-5690) Support subquery for single sourced multi query
[
https://issues.apache.org/jira/browse/HIVE-5690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Navis updated HIVE-5690:
Attachment: HIVE-5690.7.patch.txt
Support subquery for single sourced multi query
---
Key: HIVE-5690
URL: https://issues.apache.org/jira/browse/HIVE-5690
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: D13791.1.patch, HIVE-5690.2.patch.txt,
HIVE-5690.3.patch.txt, HIVE-5690.4.patch.txt, HIVE-5690.5.patch.txt,
HIVE-5690.6.patch.txt, HIVE-5690.7.patch.txt
Single sourced multi (insert) query is very useful for various ETL processes
but it does not allow subqueries included. For example,
{noformat}
explain from src
insert overwrite table x1 select * from (select distinct key,value) b order
by key
insert overwrite table x2 select * from (select distinct key,value) c order
by value;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Review Request 23303: Hive logs errors about missing tables when parsing CTE expressions
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23303/
---
Review request for hive and Ashutosh Chauhan.
Bugs: HIVE-7079
https://issues.apache.org/jira/browse/HIVE-7079
Repository: hive-git
Description
---
Given a query containing common table expressions (CTE) such as:
WITH a AS (SELECT ...), b AS (SELECT ...)
SELECT * FROM a JOIN b on a.col = b.col ...;
Hive CLI executes the query, but logs stack traces at ERROR level during query
parsing:
{noformat}
ERROR metadata.Hive: NoSuchObjectException(message:ccondit.a table not found)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29338)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29306)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result.read(ThriftHiveMetastore.java:29237)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table(ThriftHiveMetastore.java:1036)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table(ThriftHiveMetastore.java:1022)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:997)
at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
at com.sun.proxy.$Proxy7.getTable(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:967)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:909)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1223)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1192)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9209)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:391)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:291)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:944)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1009)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:880)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:870)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
It looks like Hive is attempting to resolve the CTE aliases as physical tables.
Diffs
-
ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java d8d900b
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 4d35176
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 399f92a
ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java b9890af
Diff: https://reviews.apache.org/r/23303/diff/
Testing
---
Thanks,
Navis Ryu
[jira] [Commented] (HIVE-7111) Extend join transitivity PPD to non-column expressions
[
https://issues.apache.org/jira/browse/HIVE-7111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053562#comment-14053562
]
Hive QA commented on HIVE-7111:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12654255/HIVE-7111.3.patch.txt
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 5678 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchAbort
org.apache.hive.hcatalog.streaming.TestStreaming.testTransactionBatchCommit_Delimited
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/689/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/689/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-689/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12654255
Extend join transitivity PPD to non-column expressions
--
Key: HIVE-7111
URL: https://issues.apache.org/jira/browse/HIVE-7111
Project: Hive
Issue Type: Task
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-7111.1.patch.txt, HIVE-7111.2.patch.txt,
HIVE-7111.3.patch.txt
Join transitive in PPD only supports column expressions, but it's possible to
extend this to generic expressions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6259) Support truncate for non-native tables
[
https://issues.apache.org/jira/browse/HIVE-6259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053607#comment-14053607
]
Hive QA commented on HIVE-6259:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12654257/HIVE-6259.5.patch.txt
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 5692 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/690/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/690/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-690/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12654257
Support truncate for non-native tables
--
Key: HIVE-6259
URL: https://issues.apache.org/jira/browse/HIVE-6259
Project: Hive
Issue Type: Bug
Components: StorageHandler
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: HIVE-6259.1.patch.txt, HIVE-6259.2.patch.txt,
HIVE-6259.3.patch.txt, HIVE-6259.4.patch.txt, HIVE-6259.5.patch.txt
Tables on HBase might be truncated by similar method in HBaseShell.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7344) Add streaming support in Windowing mode for FirstVal, LastVal
[ https://issues.apache.org/jira/browse/HIVE-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-7344: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Harish! Add streaming support in Windowing mode for FirstVal, LastVal - Key: HIVE-7344 URL: https://issues.apache.org/jira/browse/HIVE-7344 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Harish Butani Assignee: Harish Butani Fix For: 0.14.0 Attachments: HIVE-7344.1.patch Continuation of HIVE-7062, HIVE-7143 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7294) sql std auth - authorize show grant statements
[ https://issues.apache.org/jira/browse/HIVE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-7294: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Thejas! sql std auth - authorize show grant statements -- Key: HIVE-7294 URL: https://issues.apache.org/jira/browse/HIVE-7294 Project: Hive Issue Type: Bug Components: Authorization, SQLStandardAuthorization Reporter: Thejas M Nair Assignee: Thejas M Nair Fix For: 0.14.0 Attachments: HIVE-7294.1.patch, HIVE-7294.2.patch A non admin user should not be allowed to run show grant commands only for themselves or a role they belong to. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5718) Support direct fetch for lateral views, sub queries, etc.
[
https://issues.apache.org/jira/browse/HIVE-5718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053662#comment-14053662
]
Hive QA commented on HIVE-5718:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12654260/HIVE-5718.4.patch.txt
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 5692 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_view_sqlstd
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nonmr_fetch_threshold
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_subquery_alias
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_explode
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udtf_explode
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.ql.plan.TestReadEntityDirect.testSelectEntityInDirect
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/691/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/691/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-691/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12654260
Support direct fetch for lateral views, sub queries, etc.
-
Key: HIVE-5718
URL: https://issues.apache.org/jira/browse/HIVE-5718
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: D13857.1.patch, D13857.2.patch, D13857.3.patch,
HIVE-5718.4.patch.txt
Extend HIVE-2925 with LV and SubQ.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7243) Print padding information in ORC file dump
[
https://issues.apache.org/jira/browse/HIVE-7243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053716#comment-14053716
]
Hive QA commented on HIVE-7243:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12654261/HIVE-7243.3.patch
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 5699 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/692/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/692/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-692/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12654261
Print padding information in ORC file dump
--
Key: HIVE-7243
URL: https://issues.apache.org/jira/browse/HIVE-7243
Project: Hive
Issue Type: Improvement
Components: File Formats
Affects Versions: 0.14.0
Reporter: Prasanth J
Assignee: Prasanth J
Priority: Minor
Labels: orcfile
Attachments: HIVE-7243.1.patch, HIVE-7243.2.patch, HIVE-7243.3.patch
It will be useful to print the padding information in orc file dump utility.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7079) Hive logs errors about missing tables when parsing CTE expressions
[
https://issues.apache.org/jira/browse/HIVE-7079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053753#comment-14053753
]
Ashutosh Chauhan commented on HIVE-7079:
+1
Hive logs errors about missing tables when parsing CTE expressions
--
Key: HIVE-7079
URL: https://issues.apache.org/jira/browse/HIVE-7079
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.13.0
Reporter: Craig Condit
Assignee: Navis
Priority: Minor
Attachments: HIVE-7079.1.patch.txt, HIVE-7079.2.patch.txt
Given a query containing common table expressions (CTE) such as:
WITH a AS (SELECT ...), b AS (SELECT ...)
SELECT * FROM a JOIN b on a.col = b.col ...;
Hive CLI executes the query, but logs stack traces at ERROR level during
query parsing:
{noformat}
ERROR metadata.Hive: NoSuchObjectException(message:ccondit.a table not found)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29338)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29306)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result.read(ThriftHiveMetastore.java:29237)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table(ThriftHiveMetastore.java:1036)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table(ThriftHiveMetastore.java:1022)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:997)
at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
at com.sun.proxy.$Proxy7.getTable(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:967)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:909)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1223)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1192)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9209)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:391)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:291)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:944)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1009)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:880)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:870)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
It looks like Hive is attempting to resolve the CTE aliases as physical
tables.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7351) ANALYZE TABLE statement fails on postgres metastore
Damien Carol created HIVE-7351:
--
Summary: ANALYZE TABLE statement fails on postgres metastore
Key: HIVE-7351
URL: https://issues.apache.org/jira/browse/HIVE-7351
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.13.1, 0.13.0
Environment: postgresSQL
Reporter: Damien Carol
Priority: Minor
Metastore code use method {{PreparedStatement.setQueryTimeout(int)}} of JDBC
Driver :
Current JDBC driver doesn't implements this method.
{noformat}
2014-07-07 17:52:38,239 ERROR
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher: Error during JDBC
connection to
jdbc:postgresql://nc-h04:5432/metastore?user=hiveuserpassword=mvsmt4521.
org.postgresql.util.PSQLException: Method
org.postgresql.jdbc4.Jdbc4PreparedStatement.setQueryTimeout(int) is not yet
implemented.
at org.postgresql.Driver.notImplemented(Driver.java:753)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.setQueryTimeout(AbstractJdbc2Statement.java:666)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher$1.run(JDBCStatsPublisher.java:80)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher$1.run(JDBCStatsPublisher.java:77)
at
org.apache.hadoop.hive.ql.exec.Utilities.executeWithRetry(Utilities.java:2637)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher.connect(JDBCStatsPublisher.java:96)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.publishStats(TableScanOperator.java:280)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.closeOp(TableScanOperator.java:226)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:583)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:595)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:227)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7351) ANALYZE TABLE statement fails on postgres metastore
[
https://issues.apache.org/jira/browse/HIVE-7351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053796#comment-14053796
]
Damien Carol commented on HIVE-7351:
A quick fix should be to call this method with value 0. This doesn't throw
exception in this case.
ANALYZE TABLE statement fails on postgres metastore
---
Key: HIVE-7351
URL: https://issues.apache.org/jira/browse/HIVE-7351
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.13.0, 0.13.1
Environment: postgresSQL
Reporter: Damien Carol
Priority: Minor
Metastore code use method {{PreparedStatement.setQueryTimeout(int)}} of JDBC
Driver :
Current JDBC driver doesn't implements this method.
{noformat}
2014-07-07 17:52:38,239 ERROR
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher: Error during JDBC
connection to
jdbc:postgresql://nc-h04:5432/metastore?user=hiveuserpassword=mvsmt4521.
org.postgresql.util.PSQLException: Method
org.postgresql.jdbc4.Jdbc4PreparedStatement.setQueryTimeout(int) is not yet
implemented.
at org.postgresql.Driver.notImplemented(Driver.java:753)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.setQueryTimeout(AbstractJdbc2Statement.java:666)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher$1.run(JDBCStatsPublisher.java:80)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher$1.run(JDBCStatsPublisher.java:77)
at
org.apache.hadoop.hive.ql.exec.Utilities.executeWithRetry(Utilities.java:2637)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher.connect(JDBCStatsPublisher.java:96)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.publishStats(TableScanOperator.java:280)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.closeOp(TableScanOperator.java:226)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:583)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:595)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:227)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7351) ANALYZE TABLE statement fails on postgres metastore
[
https://issues.apache.org/jira/browse/HIVE-7351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053803#comment-14053803
]
Damien Carol commented on HIVE-7351:
Using this property seems to solve the problem.
{code}
property
namehive.stats.jdbc.timeout/name
value0/value
/property
{code}
More a hack than a solution IMHO.
ANALYZE TABLE statement fails on postgres metastore
---
Key: HIVE-7351
URL: https://issues.apache.org/jira/browse/HIVE-7351
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.13.0, 0.13.1
Environment: postgresSQL
Reporter: Damien Carol
Priority: Minor
Metastore code use method {{PreparedStatement.setQueryTimeout(int)}} of JDBC
Driver :
Current JDBC driver doesn't implements this method.
{noformat}
2014-07-07 17:52:38,239 ERROR
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher: Error during JDBC
connection to
jdbc:postgresql://nc-h04:5432/metastore?user=hiveuserpassword=mvsmt4521.
org.postgresql.util.PSQLException: Method
org.postgresql.jdbc4.Jdbc4PreparedStatement.setQueryTimeout(int) is not yet
implemented.
at org.postgresql.Driver.notImplemented(Driver.java:753)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.setQueryTimeout(AbstractJdbc2Statement.java:666)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher$1.run(JDBCStatsPublisher.java:80)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher$1.run(JDBCStatsPublisher.java:77)
at
org.apache.hadoop.hive.ql.exec.Utilities.executeWithRetry(Utilities.java:2637)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher.connect(JDBCStatsPublisher.java:96)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.publishStats(TableScanOperator.java:280)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.closeOp(TableScanOperator.java:226)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:583)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:595)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:227)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7351) ANALYZE TABLE statement fails on postgres metastore
[
https://issues.apache.org/jira/browse/HIVE-7351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Damien Carol updated HIVE-7351:
---
Labels: metastore postgres (was: )
ANALYZE TABLE statement fails on postgres metastore
---
Key: HIVE-7351
URL: https://issues.apache.org/jira/browse/HIVE-7351
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.13.0, 0.13.1
Environment: postgresSQL
Reporter: Damien Carol
Priority: Minor
Labels: metastore, postgres
Metastore code use method {{PreparedStatement.setQueryTimeout(int)}} of JDBC
Driver :
Current JDBC driver doesn't implements this method.
{noformat}
2014-07-07 17:52:38,239 ERROR
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher: Error during JDBC
connection to
jdbc:postgresql://nc-h04:5432/metastore?user=hiveuserpassword=mvsmt4521.
org.postgresql.util.PSQLException: Method
org.postgresql.jdbc4.Jdbc4PreparedStatement.setQueryTimeout(int) is not yet
implemented.
at org.postgresql.Driver.notImplemented(Driver.java:753)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.setQueryTimeout(AbstractJdbc2Statement.java:666)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher$1.run(JDBCStatsPublisher.java:80)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher$1.run(JDBCStatsPublisher.java:77)
at
org.apache.hadoop.hive.ql.exec.Utilities.executeWithRetry(Utilities.java:2637)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher.connect(JDBCStatsPublisher.java:96)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.publishStats(TableScanOperator.java:280)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.closeOp(TableScanOperator.java:226)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:583)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:595)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:227)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7351) ANALYZE TABLE statement fails on postgres metastore
[
https://issues.apache.org/jira/browse/HIVE-7351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053818#comment-14053818
]
Ashutosh Chauhan commented on HIVE-7351:
You can also try {{set hive.stats.dbclass=fs}} This skips database altogether
and instead uses filesystems for gathering stats.
ANALYZE TABLE statement fails on postgres metastore
---
Key: HIVE-7351
URL: https://issues.apache.org/jira/browse/HIVE-7351
Project: Hive
Issue Type: Bug
Components: Metastore
Affects Versions: 0.13.0, 0.13.1
Environment: postgresSQL
Reporter: Damien Carol
Priority: Minor
Labels: metastore, postgres
Metastore code use method {{PreparedStatement.setQueryTimeout(int)}} of JDBC
Driver :
Current JDBC driver doesn't implements this method.
{noformat}
2014-07-07 17:52:38,239 ERROR
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher: Error during JDBC
connection to
jdbc:postgresql://nc-h04:5432/metastore?user=hiveuserpassword=mvsmt4521.
org.postgresql.util.PSQLException: Method
org.postgresql.jdbc4.Jdbc4PreparedStatement.setQueryTimeout(int) is not yet
implemented.
at org.postgresql.Driver.notImplemented(Driver.java:753)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.setQueryTimeout(AbstractJdbc2Statement.java:666)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher$1.run(JDBCStatsPublisher.java:80)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher$1.run(JDBCStatsPublisher.java:77)
at
org.apache.hadoop.hive.ql.exec.Utilities.executeWithRetry(Utilities.java:2637)
at
org.apache.hadoop.hive.ql.stats.jdbc.JDBCStatsPublisher.connect(JDBCStatsPublisher.java:96)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.publishStats(TableScanOperator.java:280)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.closeOp(TableScanOperator.java:226)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:583)
at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:595)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:227)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:366)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7079) Hive logs errors about missing tables when parsing CTE expressions
[
https://issues.apache.org/jira/browse/HIVE-7079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-7079:
---
Resolution: Fixed
Fix Version/s: 0.14.0
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks, Navis!
Hive logs errors about missing tables when parsing CTE expressions
--
Key: HIVE-7079
URL: https://issues.apache.org/jira/browse/HIVE-7079
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.13.0
Reporter: Craig Condit
Assignee: Navis
Priority: Minor
Fix For: 0.14.0
Attachments: HIVE-7079.1.patch.txt, HIVE-7079.2.patch.txt
Given a query containing common table expressions (CTE) such as:
WITH a AS (SELECT ...), b AS (SELECT ...)
SELECT * FROM a JOIN b on a.col = b.col ...;
Hive CLI executes the query, but logs stack traces at ERROR level during
query parsing:
{noformat}
ERROR metadata.Hive: NoSuchObjectException(message:ccondit.a table not found)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29338)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result$get_table_resultStandardScheme.read(ThriftHiveMetastore.java:29306)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_table_result.read(ThriftHiveMetastore.java:29237)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:78)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_table(ThriftHiveMetastore.java:1036)
at
org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_table(ThriftHiveMetastore.java:1022)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:997)
at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:89)
at com.sun.proxy.$Proxy7.getTable(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:967)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:909)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1223)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getMetaData(SemanticAnalyzer.java:1192)
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:9209)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:327)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:391)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:291)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:944)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1009)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:880)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:870)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:792)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
It looks like Hive is attempting to resolve the CTE aliases as physical
tables.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7346) Wrong results caused by hive ppd under specific join condition
[
https://issues.apache.org/jira/browse/HIVE-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053842#comment-14053842
]
Ashutosh Chauhan commented on HIVE-7346:
+1
Wrong results caused by hive ppd under specific join condition
--
Key: HIVE-7346
URL: https://issues.apache.org/jira/browse/HIVE-7346
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0, 0.13.0, 0.13.1
Reporter: dima machlin
Assignee: Navis
Attachments: HIVE-7346.1.patch.txt
Assuming two tables :
{code:sql} t1(id1 string,id2 string) , t2 (id string,d int) {code}
t1 contains 1 row : 'a','a'
t2 contains 1 row : 'a',2
The following query :
{code:sql} select a.*,b.d d1,c.d d2
from t1 a join t2 b on (a.id1=b.id)
join t2 c on (a.id2=b.id)
where b.d =1 and c.d=1 {code}
Returns 0 rows as expected because t2.d = 2
Wrapping this query, like so :
{code:sql} select * from (
select a.*,b.d d1,c.d d2
from t1 a join t2 b on (a.id1=b.id)
join t2 c on (a.id2=b.id)
where b.d =1 and c.d=1
) z where d11 or d21 {code}
Where another filter was add on the columns causes the plan to lack the
filter of the =1 and return a single row - *Wrong Results*.
The plan is :
{code:sql}
ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_JOIN
(TOK_TABREF (TOK_TABNAME t1) a) (TOK_TABREF (TOK_TABNAME t2) b) (= (.
(TOK_TABLE_OR_COL a) id1) (. (TOK_TABLE_OR_COL b) id))) (TOK_TABREF
(TOK_TABNAME t2) c) (= (. (TOK_TABLE_OR_COL a) id2) (. (TOK_TABLE_OR_COL b)
id (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT
(TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME a))) (TOK_SELEXPR (.
(TOK_TABLE_OR_COL b) d) d1) (TOK_SELEXPR (. (TOK_TABLE_OR_COL c) d) d2))
(TOK_WHERE (and (= (. (TOK_TABLE_OR_COL b) d) 1) (= (. (TOK_TABLE_OR_COL c)
d) 1) z)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE))
(TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF)) (TOK_WHERE (or ( (TOK_TABLE_OR_COL
d1) 1) ( (TOK_TABLE_OR_COL d2) 1)
STAGE DEPENDENCIES:
Stage-7 is a root stage
Stage-5 depends on stages: Stage-7
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-7
Map Reduce Local Work
Alias - Map Local Tables:
z:b
Fetch Operator
limit: -1
z:c
Fetch Operator
limit: -1
Alias - Map Local Operator Tree:
z:b
TableScan
alias: b
HashTable Sink Operator
condition expressions:
0 {id1} {id2}
1 {id} {d}
handleSkewJoin: false
keys:
0 [Column[id1]]
1 [Column[id]]
Position of Big Table: 0
z:c
TableScan
alias: c
HashTable Sink Operator
condition expressions:
0 {_col5} {_col0} {_col1}
1 {d}
handleSkewJoin: false
keys:
0 []
1 []
Position of Big Table: 0
Stage: Stage-5
Map Reduce
Alias - Map Operator Tree:
z:a
TableScan
alias: a
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {id1} {id2}
1 {id} {d}
handleSkewJoin: false
keys:
0 [Column[id1]]
1 [Column[id]]
outputColumnNames: _col0, _col1, _col4, _col5
Position of Big Table: 0
Filter Operator
predicate:
expr: (_col1 = _col4)
type: boolean
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {_col5} {_col0} {_col1}
1 {d}
handleSkewJoin: false
keys:
0 []
1 []
outputColumnNames: _col1, _col4, _col5, _col9
Position of Big Table: 0
Filter Operator
predicate:
expr: ((_col1 1) or (_col9 1))
type: boolean
Select Operator
expressions:
expr: _col4
type: string
expr: _col5
type: string
expr: _col1
type: int
expr: _col9
type: int
[jira] [Created] (HIVE-7352) Queries without tables fail under Tez
Craig Condit created HIVE-7352:
--
Summary: Queries without tables fail under Tez
Key: HIVE-7352
URL: https://issues.apache.org/jira/browse/HIVE-7352
Project: Hive
Issue Type: Bug
Components: Tez
Affects Versions: 0.13.1, 0.13.0
Reporter: Craig Condit
Hive 0.13.0 added support for queries that do not reference tables (such as
'SELECT 1'). These queries fail under Tez:
{noformat}
Vertex failed as one or more tasks failed. failedTasks:1]
14/07/07 09:54:42 ERROR tez.TezJobMonitor: Vertex failed, vertexName=Map 1,
vertexId=vertex_1404652697071_4487_1_00, diagnostics=[Task failed,
taskId=task_1404652697071_4487_1_00_00,
diagnostics=[AttemptID:attempt_1404652697071_4487_1_00_00_0 Info:Error:
java.lang.RuntimeException: java.lang.IllegalArgumentException: Can not create
a Path from an empty string
at
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:174)
at
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.init(TezGroupedSplitsInputFormat.java:113)
at
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat.getRecordReader(TezGroupedSplitsInputFormat.java:79)
at
org.apache.tez.mapreduce.input.MRInput.setupOldRecordReader(MRInput.java:205)
at
org.apache.tez.mapreduce.input.MRInput.initFromEventInternal(MRInput.java:362)
at
org.apache.tez.mapreduce.input.MRInput.initFromEvent(MRInput.java:341)
at
org.apache.tez.mapreduce.input.MRInputLegacy.checkAndAwaitRecordReaderInitialization(MRInputLegacy.java:99)
at
org.apache.tez.mapreduce.input.MRInputLegacy.init(MRInputLegacy.java:68)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:141)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:307)
at
org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:562)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at
org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:551)
Caused by: java.lang.IllegalArgumentException: Can not create a Path from an
empty string
at org.apache.hadoop.fs.Path.checkPathArg(Path.java:127)
at org.apache.hadoop.fs.Path.init(Path.java:135)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat$HiveInputSplit.getPath(HiveInputFormat.java:110)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:228)
at
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:171)
... 14 more
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7045) Wrong results in multi-table insert aggregating without group by clause
[ https://issues.apache.org/jira/browse/HIVE-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053854#comment-14053854 ] Ashutosh Chauhan commented on HIVE-7045: +1 Wrong results in multi-table insert aggregating without group by clause --- Key: HIVE-7045 URL: https://issues.apache.org/jira/browse/HIVE-7045 Project: Hive Issue Type: Bug Affects Versions: 0.10.0, 0.12.0 Reporter: dima machlin Assignee: Navis Priority: Blocker Attachments: HIVE-7045.1.patch.txt This happens whenever there are more than 1 reducers. The scenario : CREATE TABLE t1 (a int, b int); CREATE TABLE t2 (cnt int) PARTITIONED BY (var_name string); insert into table t1 select 1,1 from asd limit 1; insert into table t1 select 2,2 from asd limit 1; t1 contains : 1 1 2 2 from t1 insert overwrite table t2 partition(var_name='a') select count(a) cnt insert overwrite table t2 partition(var_name='b') select count(b) cnt ; select * from t2; returns : 2 a 2 b as expected. Setting the number of reducers higher than 1 : set mapred.reduce.tasks=2; from t1 insert overwrite table t2 partition(var_name='a') select count(a) cnt insert overwrite table t2 partition(var_name='b') select count(b) cnt; select * from t2; 1 a 1 a 1 b 1 b Wrong results. This happens when ever t1 is big enough to automatically generate more than 1 reducers and without specifying it directly. adding group by 1 in the end of each insert solves the problem : from t1 insert overwrite table t2 partition(var_name='a') select count(a) cnt group by 1 insert overwrite table t2 partition(var_name='b') select count(b) cnt group by 1; generates : 2 a 2 b This should work without the group by... The number of rows for each partition will be the amount of reducers. Each reducer calculated a sub total of the count. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7205) Wrong results when union all of grouping followed by group by with correlation optimization
[
https://issues.apache.org/jira/browse/HIVE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053867#comment-14053867
]
Ashutosh Chauhan commented on HIVE-7205:
[~yhuai] Can you review this one?
Wrong results when union all of grouping followed by group by with
correlation optimization
---
Key: HIVE-7205
URL: https://issues.apache.org/jira/browse/HIVE-7205
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0, 0.13.0, 0.13.1
Reporter: dima machlin
Assignee: Navis
Priority: Critical
Attachments: HIVE-7205.1.patch.txt, HIVE-7205.2.patch.txt,
HIVE-7205.3.patch.txt
use case :
table TBL (a string,b string) contains single row : 'a','a'
the following query :
{code:sql}
select b, sum(cc) from (
select b,count(1) as cc from TBL group by b
union all
select a as b,count(1) as cc from TBL group by a
) z
group by b
{code}
returns
a 1
a 1
while set hive.optimize.correlation=true;
if we change set hive.optimize.correlation=false;
it returns correct results : a 2
The plan with correlation optimization :
{code:sql}
ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_SUBQUERY (TOK_UNION (TOK_QUERY (TOK_FROM
(TOK_TABREF (TOK_TABNAME DB TBL))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR
TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_TABLE_OR_COL b)) (TOK_SELEXPR
(TOK_FUNCTION count 1) cc)) (TOK_GROUPBY (TOK_TABLE_OR_COL b (TOK_QUERY
(TOK_FROM (TOK_TABREF (TOK_TABNAME DB TBL))) (TOK_INSERT (TOK_DESTINATION
(TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_TABLE_OR_COL a) b)
(TOK_SELEXPR (TOK_FUNCTION count 1) cc)) (TOK_GROUPBY (TOK_TABLE_OR_COL
a) z)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT
(TOK_SELEXPR (TOK_TABLE_OR_COL b)) (TOK_SELEXPR (TOK_FUNCTION sum
(TOK_TABLE_OR_COL cc (TOK_GROUPBY (TOK_TABLE_OR_COL b
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Alias - Map Operator Tree:
null-subquery1:z-subquery1:TBL
TableScan
alias: TBL
Select Operator
expressions:
expr: b
type: string
outputColumnNames: b
Group By Operator
aggregations:
expr: count(1)
bucketGroup: false
keys:
expr: b
type: string
mode: hash
outputColumnNames: _col0, _col1
Reduce Output Operator
key expressions:
expr: _col0
type: string
sort order: +
Map-reduce partition columns:
expr: _col0
type: string
tag: 0
value expressions:
expr: _col1
type: bigint
null-subquery2:z-subquery2:TBL
TableScan
alias: TBL
Select Operator
expressions:
expr: a
type: string
outputColumnNames: a
Group By Operator
aggregations:
expr: count(1)
bucketGroup: false
keys:
expr: a
type: string
mode: hash
outputColumnNames: _col0, _col1
Reduce Output Operator
key expressions:
expr: _col0
type: string
sort order: +
Map-reduce partition columns:
expr: _col0
type: string
tag: 1
value expressions:
expr: _col1
type: bigint
Reduce Operator Tree:
Demux Operator
Group By Operator
aggregations:
expr: count(VALUE._col0)
bucketGroup: false
keys:
expr: KEY._col0
type: string
mode: mergepartial
outputColumnNames: _col0, _col1
Select Operator
expressions:
expr: _col0
type: string
expr: _col1
type: bigint
outputColumnNames: _col0, _col1
Union
Select Operator
expressions:
expr: _col0
[jira] [Commented] (HIVE-7326) Hive complains invalid column reference with 'having' aggregate predicates
[ https://issues.apache.org/jira/browse/HIVE-7326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053871#comment-14053871 ] Ashutosh Chauhan commented on HIVE-7326: +1 Hive complains invalid column reference with 'having' aggregate predicates -- Key: HIVE-7326 URL: https://issues.apache.org/jira/browse/HIVE-7326 Project: Hive Issue Type: Bug Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Hari Sankar Sivarama Subramaniyan Attachments: HIVE-7326.1.patch.txt, HIVE-7326.2.patch.txt CREATE TABLE TestV1_Staples ( Item_Count INT, Ship_Priority STRING, Order_Priority STRING, Order_Status STRING, Order_Quantity DOUBLE, Sales_Total DOUBLE, Discount DOUBLE, Tax_Rate DOUBLE, Ship_Mode STRING, Fill_Time DOUBLE, Gross_Profit DOUBLE, Price DOUBLE, Ship_Handle_Cost DOUBLE, Employee_Name STRING, Employee_Dept STRING, Manager_Name STRING, Employee_Yrs_Exp DOUBLE, Employee_Salary DOUBLE, Customer_Name STRING, Customer_State STRING, Call_Center_Region STRING, Customer_Balance DOUBLE, Customer_Segment STRING, Prod_Type1 STRING, Prod_Type2 STRING, Prod_Type3 STRING, Prod_Type4 STRING, Product_Name STRING, Product_Container STRING, Ship_Promo STRING, Supplier_Name STRING, Supplier_Balance DOUBLE, Supplier_Region STRING, Supplier_State STRING, Order_ID STRING, Order_Year INT, Order_Month INT, Order_Day INT, Order_Date_ STRING, Order_Quarter STRING, Product_Base_Margin DOUBLE, Product_ID STRING, Receive_Time DOUBLE, Received_Date_ STRING, Ship_Date_ STRING, Ship_Charge DOUBLE, Total_Cycle_Time DOUBLE, Product_In_Stock STRING, PID INT, Market_Segment STRING ); Query that works: SELECT customer_name, SUM(customer_balance), SUM(order_quantity) FROM default.testv1_staples s1 GROUP BY customer_name HAVING ( (COUNT(s1.discount) = 822) AND (SUM(customer_balance) = 4074689.00041) ); Query that fails: SELECT customer_name, SUM(customer_balance), SUM(order_quantity) FROM default.testv1_staples s1 GROUP BY customer_name HAVING ( (SUM(customer_balance) = 4074689.00041) AND (COUNT(s1.discount) = 822) ); -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7323) Date type stats in ORC sometimes go stale
[ https://issues.apache.org/jira/browse/HIVE-7323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053874#comment-14053874 ] Ashutosh Chauhan commented on HIVE-7323: Can you explain whats the bug here? Not having a test case is ok, as long as we understand what is the bug here. Date type stats in ORC sometimes go stale - Key: HIVE-7323 URL: https://issues.apache.org/jira/browse/HIVE-7323 Project: Hive Issue Type: Bug Components: Statistics Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-7323.1.patch.txt I cannot make proper test case but sometimes min/max value in date type stats is changed in runtime. Stats for other type contains non-mutable values in it but date type stats contains DateWritable, which of inner value can be changed anytime. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5510) [WebHCat] GET job/queue return wrong job information
[ https://issues.apache.org/jira/browse/HIVE-5510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053877#comment-14053877 ] Eugene Koifman commented on HIVE-5510: -- [~leftylev] the 1st example (under JSON Output (fields)) seems to be of the behavior before the bug fix - isn't likely to confuse users. Should the example be of 'correct' output? [~daijy] Does that make sense to you? [WebHCat] GET job/queue return wrong job information Key: HIVE-5510 URL: https://issues.apache.org/jira/browse/HIVE-5510 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.12.0 Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.13.0 Attachments: HIVE-5510-1.patch, HIVE-5510-2.patch, HIVE-5510-3.patch, HIVE-5510-4.patch, test_harnesss_1381798977 GET job/queue of a TempletonController job return weird information. It is a mix of child job and itself. It should only pull the information of the controller job itself. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7213) COUNT(*) returns out-dated count value after TRUNCATE or INSERT INTO
[
https://issues.apache.org/jira/browse/HIVE-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053881#comment-14053881
]
Ashutosh Chauhan commented on HIVE-7213:
[~totto90] Did you get a chance to run explain per my previous comment?
COUNT(*) returns out-dated count value after TRUNCATE or INSERT INTO
Key: HIVE-7213
URL: https://issues.apache.org/jira/browse/HIVE-7213
Project: Hive
Issue Type: Bug
Components: Query Processor, Statistics
Affects Versions: 0.13.0
Environment: HDP 2.1
Windows Server 2012 64-bit
Reporter: Moustafa Aboul Atta
Running a query to count number of rows in a table through
{{SELECT COUNT( * ) FROM t}}
always returns the last number of rows added through the following statement:
{{INSERT INTO TABLE t SELECT r FROM t2}}
However, running
{{SELECT * FROM t}}
returns the expected results i.e. the old and newly added rows.
Also running
{{TRUNCATE TABLE t;}}
returns the original count of rows in the table, however running
{{SELECT * FROM t;}}
returns nothing as expected
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-5025) Column aliases for input argument of GenericUDFs
[
https://issues.apache.org/jira/browse/HIVE-5025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053894#comment-14053894
]
Hive QA commented on HIVE-5025:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12654262/HIVE-5025.4.patch.txt
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 5685 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/694/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/694/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-694/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12654262
Column aliases for input argument of GenericUDFs
-
Key: HIVE-5025
URL: https://issues.apache.org/jira/browse/HIVE-5025
Project: Hive
Issue Type: Improvement
Components: UDF
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: D12093.2.patch, D12093.3.patch, HIVE-5025.4.patch.txt,
HIVE-5025.D12093.1.patch
In some cases, column aliases for input argument are very useful to know. But
I cannot sure of this in the sense that UDFs should not be dependent to
contextual information like column alias.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7308) Show auto-parallelism in extended explain
[ https://issues.apache.org/jira/browse/HIVE-7308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053930#comment-14053930 ] Vikram Dixit K commented on HIVE-7308: -- +1 LGTM Show auto-parallelism in extended explain - Key: HIVE-7308 URL: https://issues.apache.org/jira/browse/HIVE-7308 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-7308.1.patch, HIVE-7308.2.patch Add auto-parallelism flag to explain so that we can write tests verifying that we don't break bmj, etc... -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7299) Enable metadata only optimization on Tez
[ https://issues.apache.org/jira/browse/HIVE-7299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053973#comment-14053973 ] Vikram Dixit K commented on HIVE-7299: -- Left some comments on review board. Enable metadata only optimization on Tez Key: HIVE-7299 URL: https://issues.apache.org/jira/browse/HIVE-7299 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-7299.1.patch, HIVE-7299.2.patch, HIVE-7299.3.patch, HIVE-7299.4.patch, HIVE-7299.5.patch Enables the metadata only optimization (the one with OneNullRowInputFormat not the query-result-from-stats optimizaton) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7090) Support session-level temporary tables in Hive
[ https://issues.apache.org/jira/browse/HIVE-7090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-7090: - Attachment: HIVE-7090.8.patch Patch v8 fixes authorization for temporary table, and adds positive authorization test. Previously just had negative auth test. Support session-level temporary tables in Hive -- Key: HIVE-7090 URL: https://issues.apache.org/jira/browse/HIVE-7090 Project: Hive Issue Type: Bug Components: SQL Reporter: Gunther Hagleitner Assignee: Jason Dere Attachments: HIVE-7090.1.patch, HIVE-7090.2.patch, HIVE-7090.3.patch, HIVE-7090.4.patch, HIVE-7090.5.patch, HIVE-7090.6.patch, HIVE-7090.7.patch, HIVE-7090.8.patch It's common to see sql scripts that create some temporary table as an intermediate result, run some additional queries against it and then clean up at the end. We should support temporary tables properly, meaning automatically manage the life cycle and make sure the visibility is restricted to the creating connection/session. Without these it's common to see left over tables in meta-store or weird errors with clashing tmp table names. Proposed syntax: CREATE TEMPORARY TABLE CTAS, CTL, INSERT INTO, should all be supported as usual. Knowing that a user wants a temp table can enable us to further optimize access to it. E.g.: temp tables should be kept in memory where possible, compactions and merging table files aren't required, ... -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7243) Print padding information in ORC file dump
[ https://issues.apache.org/jira/browse/HIVE-7243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053912#comment-14053912 ] Gunther Hagleitner commented on HIVE-7243: -- +1 Print padding information in ORC file dump -- Key: HIVE-7243 URL: https://issues.apache.org/jira/browse/HIVE-7243 Project: Hive Issue Type: Improvement Components: File Formats Affects Versions: 0.14.0 Reporter: Prasanth J Assignee: Prasanth J Priority: Minor Labels: orcfile Attachments: HIVE-7243.1.patch, HIVE-7243.2.patch, HIVE-7243.3.patch It will be useful to print the padding information in orc file dump utility. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7299) Enable metadata only optimization on Tez
[ https://issues.apache.org/jira/browse/HIVE-7299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14053995#comment-14053995 ] Gunther Hagleitner commented on HIVE-7299: -- [~vikram.dixit] I've answered the question about setMapWork. The rest are ws or javadoc, which i can do at commit time. Are you fine otherwise? Enable metadata only optimization on Tez Key: HIVE-7299 URL: https://issues.apache.org/jira/browse/HIVE-7299 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-7299.1.patch, HIVE-7299.2.patch, HIVE-7299.3.patch, HIVE-7299.4.patch, HIVE-7299.5.patch Enables the metadata only optimization (the one with OneNullRowInputFormat not the query-result-from-stats optimizaton) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7326) Hive complains invalid column reference with 'having' aggregate predicates
[ https://issues.apache.org/jira/browse/HIVE-7326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054014#comment-14054014 ] Hari Sankar Sivarama Subramaniyan commented on HIVE-7326: - Navis, Thanks for looking into this. I am assigning the jira to [~navis] Thanks Hari Hive complains invalid column reference with 'having' aggregate predicates -- Key: HIVE-7326 URL: https://issues.apache.org/jira/browse/HIVE-7326 Project: Hive Issue Type: Bug Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Hari Sankar Sivarama Subramaniyan Attachments: HIVE-7326.1.patch.txt, HIVE-7326.2.patch.txt CREATE TABLE TestV1_Staples ( Item_Count INT, Ship_Priority STRING, Order_Priority STRING, Order_Status STRING, Order_Quantity DOUBLE, Sales_Total DOUBLE, Discount DOUBLE, Tax_Rate DOUBLE, Ship_Mode STRING, Fill_Time DOUBLE, Gross_Profit DOUBLE, Price DOUBLE, Ship_Handle_Cost DOUBLE, Employee_Name STRING, Employee_Dept STRING, Manager_Name STRING, Employee_Yrs_Exp DOUBLE, Employee_Salary DOUBLE, Customer_Name STRING, Customer_State STRING, Call_Center_Region STRING, Customer_Balance DOUBLE, Customer_Segment STRING, Prod_Type1 STRING, Prod_Type2 STRING, Prod_Type3 STRING, Prod_Type4 STRING, Product_Name STRING, Product_Container STRING, Ship_Promo STRING, Supplier_Name STRING, Supplier_Balance DOUBLE, Supplier_Region STRING, Supplier_State STRING, Order_ID STRING, Order_Year INT, Order_Month INT, Order_Day INT, Order_Date_ STRING, Order_Quarter STRING, Product_Base_Margin DOUBLE, Product_ID STRING, Receive_Time DOUBLE, Received_Date_ STRING, Ship_Date_ STRING, Ship_Charge DOUBLE, Total_Cycle_Time DOUBLE, Product_In_Stock STRING, PID INT, Market_Segment STRING ); Query that works: SELECT customer_name, SUM(customer_balance), SUM(order_quantity) FROM default.testv1_staples s1 GROUP BY customer_name HAVING ( (COUNT(s1.discount) = 822) AND (SUM(customer_balance) = 4074689.00041) ); Query that fails: SELECT customer_name, SUM(customer_balance), SUM(order_quantity) FROM default.testv1_staples s1 GROUP BY customer_name HAVING ( (SUM(customer_balance) = 4074689.00041) AND (COUNT(s1.discount) = 822) ); -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7326) Hive complains invalid column reference with 'having' aggregate predicates
[ https://issues.apache.org/jira/browse/HIVE-7326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sankar Sivarama Subramaniyan updated HIVE-7326: Assignee: Navis (was: Hari Sankar Sivarama Subramaniyan) Hive complains invalid column reference with 'having' aggregate predicates -- Key: HIVE-7326 URL: https://issues.apache.org/jira/browse/HIVE-7326 Project: Hive Issue Type: Bug Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Navis Attachments: HIVE-7326.1.patch.txt, HIVE-7326.2.patch.txt CREATE TABLE TestV1_Staples ( Item_Count INT, Ship_Priority STRING, Order_Priority STRING, Order_Status STRING, Order_Quantity DOUBLE, Sales_Total DOUBLE, Discount DOUBLE, Tax_Rate DOUBLE, Ship_Mode STRING, Fill_Time DOUBLE, Gross_Profit DOUBLE, Price DOUBLE, Ship_Handle_Cost DOUBLE, Employee_Name STRING, Employee_Dept STRING, Manager_Name STRING, Employee_Yrs_Exp DOUBLE, Employee_Salary DOUBLE, Customer_Name STRING, Customer_State STRING, Call_Center_Region STRING, Customer_Balance DOUBLE, Customer_Segment STRING, Prod_Type1 STRING, Prod_Type2 STRING, Prod_Type3 STRING, Prod_Type4 STRING, Product_Name STRING, Product_Container STRING, Ship_Promo STRING, Supplier_Name STRING, Supplier_Balance DOUBLE, Supplier_Region STRING, Supplier_State STRING, Order_ID STRING, Order_Year INT, Order_Month INT, Order_Day INT, Order_Date_ STRING, Order_Quarter STRING, Product_Base_Margin DOUBLE, Product_ID STRING, Receive_Time DOUBLE, Received_Date_ STRING, Ship_Date_ STRING, Ship_Charge DOUBLE, Total_Cycle_Time DOUBLE, Product_In_Stock STRING, PID INT, Market_Segment STRING ); Query that works: SELECT customer_name, SUM(customer_balance), SUM(order_quantity) FROM default.testv1_staples s1 GROUP BY customer_name HAVING ( (COUNT(s1.discount) = 822) AND (SUM(customer_balance) = 4074689.00041) ); Query that fails: SELECT customer_name, SUM(customer_balance), SUM(order_quantity) FROM default.testv1_staples s1 GROUP BY customer_name HAVING ( (SUM(customer_balance) = 4074689.00041) AND (COUNT(s1.discount) = 822) ); -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7308) Show auto-parallelism in extended explain
[ https://issues.apache.org/jira/browse/HIVE-7308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-7308: - Attachment: HIVE-7308.3.patch .3 is rebased Show auto-parallelism in extended explain - Key: HIVE-7308 URL: https://issues.apache.org/jira/browse/HIVE-7308 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-7308.1.patch, HIVE-7308.2.patch, HIVE-7308.3.patch Add auto-parallelism flag to explain so that we can write tests verifying that we don't break bmj, etc... -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7308) Show auto-parallelism in extended explain
[ https://issues.apache.org/jira/browse/HIVE-7308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-7308: - Status: Open (was: Patch Available) Show auto-parallelism in extended explain - Key: HIVE-7308 URL: https://issues.apache.org/jira/browse/HIVE-7308 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-7308.1.patch, HIVE-7308.2.patch, HIVE-7308.3.patch Add auto-parallelism flag to explain so that we can write tests verifying that we don't break bmj, etc... -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7308) Show auto-parallelism in extended explain
[ https://issues.apache.org/jira/browse/HIVE-7308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-7308: - Status: Patch Available (was: Open) Show auto-parallelism in extended explain - Key: HIVE-7308 URL: https://issues.apache.org/jira/browse/HIVE-7308 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-7308.1.patch, HIVE-7308.2.patch, HIVE-7308.3.patch Add auto-parallelism flag to explain so that we can write tests verifying that we don't break bmj, etc... -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7299) Enable metadata only optimization on Tez
[ https://issues.apache.org/jira/browse/HIVE-7299?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054021#comment-14054021 ] Vikram Dixit K commented on HIVE-7299: -- I see. I understand now. Thanks! It looks good to me to fix the other comments at commit time. +1 Enable metadata only optimization on Tez Key: HIVE-7299 URL: https://issues.apache.org/jira/browse/HIVE-7299 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-7299.1.patch, HIVE-7299.2.patch, HIVE-7299.3.patch, HIVE-7299.4.patch, HIVE-7299.5.patch Enables the metadata only optimization (the one with OneNullRowInputFormat not the query-result-from-stats optimizaton) -- This message was sent by Atlassian JIRA (v6.2#6252)
[GitHub] hive pull request: Branch 0.12
GitHub user thaparraj opened a pull request: https://github.com/apache/hive/pull/18 Branch 0.12 You can merge this pull request into a Git repository by running: $ git pull https://github.com/apache/hive branch-0.12 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/hive/pull/18.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #18 commit 2f6f9ca7b62ab27097d0927361e75c7544363bfa Author: Thejas Madhavan Nair the...@apache.org Date: 2013-09-07T01:23:56Z Branching for 0.12 release git-svn-id: https://svn.apache.org/repos/asf/hive/branches/branch-0.12@1520718 13f79535-47bb-0310-9956-ffa450edef68 commit afa12690b34ad9b2050ab79bdc704e54f48ace91 Author: Thejas Madhavan Nair the...@apache.org Date: 2013-09-07T01:35:25Z Preparing for release 0.12.0 git-svn-id: https://svn.apache.org/repos/asf/hive/branches/branch-0.12@1520720 13f79535-47bb-0310-9956-ffa450edef68 commit 7b911507d316c4c4de1e218ed947a4ee40653e8e Author: Carl Steinbach c...@apache.org Date: 2013-09-08T06:16:46Z Fix build (cws) git-svn-id: https://svn.apache.org/repos/asf/hive/branches/branch-0.12@1520838 13f79535-47bb-0310-9956-ffa450edef68 commit f9576b07be892e24998e546f34a2a555a42911f6 Author: Brock Noland br...@apache.org Date: 2013-09-09T17:16:36Z HIVE-4003 - NullPointerException in exec.Utilities (Mark Grover via Brock Noland) git-svn-id: https://svn.apache.org/repos/asf/hive/branches/branch-0.12@1521196 13f79535-47bb-0310-9956-ffa450edef68 commit c0c2313a08e9d52cc7872cb078865dcc3f4145da Author: Thejas Madhavan Nair the...@apache.org Date: 2013-09-10T02:14:54Z HIVE-4617: Asynchronous execution in HiveServer2 to run a query in non-blocking mode (Jaideep Dhok Vaibhav Gumashta via Thejas Nair) git-svn-id: https://svn.apache.org/repos/asf/hive/branches/branch-0.12@1521350 13f79535-47bb-0310-9956-ffa450edef68 commit c97ddc6a745ca75b848a7e407a2e61147b313d8c Author: Brock Noland br...@apache.org Date: 2013-09-10T19:22:48Z HIVE-4619 - Hive 0.11.0 is not working with pre-cdh3u6 and hadoop-0.23 (Navis via Brock Noland) git-svn-id: https://svn.apache.org/repos/asf/hive/branches/branch-0.12@1521595 13f79535-47bb-0310-9956-ffa450edef68 commit 8a7f1723b315f31dfb4f4108e425eb58258a20d6 Author: Brock Noland br...@apache.org Date: 2013-09-10T20:20:19Z HIVE-5255 - Missing metastore schema files for version 0.11 (Jarek Jarcec Cecho via Brock Noland) git-svn-id: https://svn.apache.org/repos/asf/hive/branches/branch-0.12@1521627 13f79535-47bb-0310-9956-ffa450edef68 commit 856d1641afd0d7fd17a899c0ebb4d3a2f6a25e20 Author: Thejas Madhavan Nair the...@apache.org Date: 2013-09-11T21:41:30Z HIVE-5056 : MapJoinProcessor ignores order of values in removing RS (Navis Ryu via Ashutosh Chauhan) git-svn-id: https://svn.apache.org/repos/asf/hive/branches/branch-0.12@1522058 13f79535-47bb-0310-9956-ffa450edef68 commit e75159085bfacf7c1c8424130443369d5f88e3f6 Author: Thejas Madhavan Nair the...@apache.org Date: 2013-09-12T21:06:05Z HIVE-5265 : Direct SQL fallback broken on Postgres (Sergey Shelukhin via Ashutosh Chauhan) git-svn-id: https://svn.apache.org/repos/asf/hive/branches/branch-0.12@1522729 13f79535-47bb-0310-9956-ffa450edef68 commit c2c133fcb7066a037eddcd2572e2eaa0856b7373 Author: Thejas Madhavan Nair the...@apache.org Date: 2013-09-13T21:02:13Z HIVE-4171 : Current database in metastore.Hive is not consistent with SessionState (Thejas Nair via Ashutosh Chauhan) git-svn-id: https://svn.apache.org/repos/asf/hive/branches/branch-0.12@1523102 13f79535-47bb-0310-9956-ffa450edef68 commit a2c3733106c8b0f6aeb843326f73496b5cdd0070 Author: Thejas Madhavan Nair the...@apache.org Date: 2013-09-13T21:46:25Z HIVE-5239 : LazyDate goes into irretrievable NULL mode once inited with NULL once (Jason Dere via Ashutosh Chauhan) git-svn-id: https://svn.apache.org/repos/asf/hive/branches/branch-0.12@1523119 13f79535-47bb-0310-9956-ffa450edef68 commit 922c7281113533dd322436e047d42255a0ed291a Author: Thejas Madhavan Nair the...@apache.org Date: 2013-09-13T22:27:48Z HIVE-5241: Default log4j log level for WebHCat should be INFO not DEBUG (Eugene Koifman via Thejas Nair) git-svn-id: https://svn.apache.org/repos/asf/hive/branches/branch-0.12@1523131 13f79535-47bb-0310-9956-ffa450edef68 commit b7459347fd65698cd52757bb8929c68b2c0ca400 Author: Thejas Madhavan Nair the...@apache.org Date: 2013-09-13T22:29:38Z HIVE-5127: Upgrade xerces and xalan for WebHCat (Eugene Koifman via Thejas Nair) git-svn-id: https://svn.apache.org/repos/asf/hive/branches/branch-0.12@1523132 13f79535-47bb-0310-9956-ffa450edef68 commit a363cfa51e3337278897746da0625554baedb217 Author: Thejas
Re: Review Request 22996: HIVE-7090 Support session-level temporary tables in Hive
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/22996/ --- (Updated July 7, 2014, 7:41 p.m.) Review request for hive, Gunther Hagleitner, Navis Ryu, and Harish Butani. Changes --- fix/positive test for authorization and temp tables Bugs: HIVE-7090 https://issues.apache.org/jira/browse/HIVE-7090 Repository: hive-git Description --- Temp tables managed in memory by SessionState. SessionHiveMetaStoreClient overrides table-related methods in HiveMetaStore to access the temp tables saved in the SessionState when appropriate. Diffs (updated) - itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcWithMiniMr.java 9fb7550 itests/qtest/testconfiguration.properties 1462ecd metastore/if/hive_metastore.thrift cc802c6 metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h 413256f metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 8014f2a metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java 1e7fca3 metastore/src/gen/thrift/gen-php/metastore/Types.php 46f6a04 metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py b3eeb89 metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb 757461f metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 664dccd metastore/src/java/org/apache/hadoop/hive/metastore/Warehouse.java 9e8d912 ql/src/java/org/apache/hadoop/hive/ql/Context.java abc4290 ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java 4d8e10c ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 250756c ql/src/java/org/apache/hadoop/hive/ql/metadata/SessionHiveMetaStoreClient.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 3df2690 ql/src/java/org/apache/hadoop/hive/ql/parse/ColumnStatsSemanticAnalyzer.java 6c9876d ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g f934ac4 ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java 71471f4 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 70ec577 ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java 2537b75 ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableLikeDesc.java cb5d64c ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java 2de476e ql/src/test/org/apache/hadoop/hive/ql/exec/tez/TestTezTask.java 43125f7 ql/src/test/org/apache/hadoop/hive/ql/lockmgr/TestDbTxnManager.java 98c3cc3 ql/src/test/org/apache/hadoop/hive/ql/parse/TestMacroSemanticAnalyzer.java 91de8da ql/src/test/org/apache/hadoop/hive/ql/parse/authorization/TestHiveAuthorizationTaskFactory.java 20d08b3 ql/src/test/queries/clientnegative/temp_table_authorize_create_tbl.q PRE-CREATION ql/src/test/queries/clientnegative/temp_table_column_stats.q PRE-CREATION ql/src/test/queries/clientnegative/temp_table_create_like_partitions.q PRE-CREATION ql/src/test/queries/clientnegative/temp_table_index.q PRE-CREATION ql/src/test/queries/clientnegative/temp_table_partitions.q PRE-CREATION ql/src/test/queries/clientnegative/temp_table_rename.q PRE-CREATION ql/src/test/queries/clientpositive/authorization_create_temp_table.q PRE-CREATION ql/src/test/queries/clientpositive/show_create_table_temp_table.q PRE-CREATION ql/src/test/queries/clientpositive/stats19.q 51514bd ql/src/test/queries/clientpositive/temp_table.q PRE-CREATION ql/src/test/queries/clientpositive/temp_table_external.q PRE-CREATION ql/src/test/queries/clientpositive/temp_table_gb1.q PRE-CREATION ql/src/test/queries/clientpositive/temp_table_join1.q PRE-CREATION ql/src/test/queries/clientpositive/temp_table_names.q PRE-CREATION ql/src/test/queries/clientpositive/temp_table_options1.q PRE-CREATION ql/src/test/queries/clientpositive/temp_table_precedence.q PRE-CREATION ql/src/test/queries/clientpositive/temp_table_subquery1.q PRE-CREATION ql/src/test/queries/clientpositive/temp_table_windowing_expressions.q PRE-CREATION ql/src/test/results/clientnegative/temp_table_authorize_create_tbl.q.out PRE-CREATION ql/src/test/results/clientnegative/temp_table_column_stats.q.out PRE-CREATION ql/src/test/results/clientnegative/temp_table_create_like_partitions.q.out PRE-CREATION ql/src/test/results/clientnegative/temp_table_index.q.out PRE-CREATION ql/src/test/results/clientnegative/temp_table_partitions.q.out PRE-CREATION ql/src/test/results/clientnegative/temp_table_rename.q.out PRE-CREATION ql/src/test/results/clientpositive/authorization_create_temp_table.q.out PRE-CREATION ql/src/test/results/clientpositive/nullformat.q.out d311825 ql/src/test/results/clientpositive/nullformatCTAS.q.out cab23d5 ql/src/test/results/clientpositive/show_create_table_alter.q.out 206f4f8 ql/src/test/results/clientpositive/show_create_table_db_table.q.out 528dd36
[jira] [Updated] (HIVE-7299) Enable metadata only optimization on Tez
[ https://issues.apache.org/jira/browse/HIVE-7299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-7299: - Status: Patch Available (was: Open) Enable metadata only optimization on Tez Key: HIVE-7299 URL: https://issues.apache.org/jira/browse/HIVE-7299 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-7299.1.patch, HIVE-7299.2.patch, HIVE-7299.3.patch, HIVE-7299.4.patch, HIVE-7299.5.patch, HIVE-7299.6.patch Enables the metadata only optimization (the one with OneNullRowInputFormat not the query-result-from-stats optimizaton) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7299) Enable metadata only optimization on Tez
[ https://issues.apache.org/jira/browse/HIVE-7299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-7299: - Attachment: HIVE-7299.6.patch .6 is rebased (and has review comments fixed). Enable metadata only optimization on Tez Key: HIVE-7299 URL: https://issues.apache.org/jira/browse/HIVE-7299 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-7299.1.patch, HIVE-7299.2.patch, HIVE-7299.3.patch, HIVE-7299.4.patch, HIVE-7299.5.patch, HIVE-7299.6.patch Enables the metadata only optimization (the one with OneNullRowInputFormat not the query-result-from-stats optimizaton) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7299) Enable metadata only optimization on Tez
[ https://issues.apache.org/jira/browse/HIVE-7299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-7299: - Status: Open (was: Patch Available) Enable metadata only optimization on Tez Key: HIVE-7299 URL: https://issues.apache.org/jira/browse/HIVE-7299 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-7299.1.patch, HIVE-7299.2.patch, HIVE-7299.3.patch, HIVE-7299.4.patch, HIVE-7299.5.patch, HIVE-7299.6.patch Enables the metadata only optimization (the one with OneNullRowInputFormat not the query-result-from-stats optimizaton) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7342) support hiveserver2,metastore specific config files
[ https://issues.apache.org/jira/browse/HIVE-7342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-7342: Attachment: HIVE-7342.1.patch support hiveserver2,metastore specific config files --- Key: HIVE-7342 URL: https://issues.apache.org/jira/browse/HIVE-7342 Project: Hive Issue Type: Bug Components: Configuration, HiveServer2, Metastore Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-7342.1.patch There is currently a single configuration file for all components in hive. ie, components such as hive cli, hiveserver2 and metastore all read from the same hive-site.xml. It will be useful to have a server specific hive-site.xml, so that you can have some different configuration value set for a server. For example, you might want to enabled authorization checks for hiveserver2, while disabling the checks for hive cli. The workaround today is to add any component specific configuration as a commandline (-hiveconf) argument. Using server specific config files (eg hiveserver2-site.xml, metastore-site.xml) that override the entries in hive-site.xml will make the configuration much more easy to manage. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Branch for HIVE-7292
Hi all, I went ahead and created a branch as below: https://svn.apache.org/repos/asf/hive/branches/spark. Let me know if you have any questions. Thanks, Xuefu On Wed, Jul 2, 2014 at 7:18 AM, Xuefu Zhang xzh...@cloudera.com wrote: Hi all, I have started working HIVE-7292, for which I think a branch would make sense so that the trunk will be kept stable all the time, due to the fair amount of integration work between Hive and Spark. Thus, I'd like to propose creating a branch in order to be able to do this incrementally and collaboratively. Secondly, there will be limited amount of refactoring work to support HIVE-7292. For this, we will work directly on trunk. Please let me know if you have any questions or concerns. At the same time, design doc has been posted on JIRA and wiki for quite some time. Thank you for those who have provided feedback, but feedback is welcome any time. Regards, Xuefu
[jira] [Commented] (HIVE-4790) MapredLocalTask task does not make virtual columns
[
https://issues.apache.org/jira/browse/HIVE-4790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054137#comment-14054137
]
Hive QA commented on HIVE-4790:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12654264/HIVE-4790.10.patch.txt
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 5684 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/695/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/695/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-695/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12654264
MapredLocalTask task does not make virtual columns
--
Key: HIVE-4790
URL: https://issues.apache.org/jira/browse/HIVE-4790
Project: Hive
Issue Type: Bug
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: D11511.3.patch, D11511.4.patch, HIVE-4790.10.patch.txt,
HIVE-4790.5.patch.txt, HIVE-4790.6.patch.txt, HIVE-4790.7.patch.txt,
HIVE-4790.8.patch.txt, HIVE-4790.9.patch.txt, HIVE-4790.D11511.1.patch,
HIVE-4790.D11511.2.patch
From mailing list,
http://www.mail-archive.com/user@hive.apache.org/msg08264.html
{noformat}
SELECT *,b.BLOCK__OFFSET__INSIDE__FILE FROM a JOIN b ON
b.rownumber = a.number;
fails with this error:
SELECT *,b.BLOCK__OFFSET__INSIDE__FILE FROM a JOIN b ON b.rownumber =
a.number;
Automatically selecting local only mode for query
Total MapReduce jobs = 1
setting HADOOP_USER_NAMEpmarron
13/06/25 10:52:56 WARN conf.HiveConf: DEPRECATED: Configuration property
hive.metastore.local no longer has any effect. Make sure to provide a valid
value for hive.metastore.uris if you are connecting to a remote metastore.
Execution log at: /tmp/pmarron/.log
2013-06-25 10:52:56 Starting to launch local task to process map join;
maximum memory = 932118528
java.lang.RuntimeException: cannot find field block__offset__inside__file
from [0:rownumber, 1:offset]
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.getStandardStructFieldRef(ObjectInspectorUtils.java:366)
at
org.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldRef(LazySimpleStructObjectInspector.java:168)
at
org.apache.hadoop.hive.serde2.objectinspector.DelegatedStructObjectInspector.getStructFieldRef(DelegatedStructObjectInspector.java:74)
at
org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.initialize(ExprNodeColumnEvaluator.java:57)
at
org.apache.hadoop.hive.ql.exec.JoinUtil.getObjectInspectorsFromEvaluators(JoinUtil.java:68)
at
org.apache.hadoop.hive.ql.exec.HashTableSinkOperator.initializeOp(HashTableSinkOperator.java:222)
at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:186)
at
org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at
org.apache.hadoop.hive.ql.exec.MapredLocalTask.initializeOperators(MapredLocalTask.java:394)
at
org.apache.hadoop.hive.ql.exec.MapredLocalTask.executeFromChildJVM(MapredLocalTask.java:277)
at org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:676)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Execution failed with exit status: 2
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7353) HiveServer2 using embedded MetaStore leaks JDOPersistanceManager
Vaibhav Gumashta created HIVE-7353: -- Summary: HiveServer2 using embedded MetaStore leaks JDOPersistanceManager Key: HIVE-7353 URL: https://issues.apache.org/jira/browse/HIVE-7353 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.14.0 While using embedded metastore, when creating background threads to run async operations, HiveServer2 ends up creating new instances of JDOPersistanceManager rather than using the one from the foreground (handler) thread. Since JDOPersistanceManagerFactory caches JDOPersistanceManager instances, they are never GCed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7353) HiveServer2 using embedded MetaStore leaks JDOPersistanceManager
[ https://issues.apache.org/jira/browse/HIVE-7353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-7353: --- Component/s: HiveServer2 HiveServer2 using embedded MetaStore leaks JDOPersistanceManager Key: HIVE-7353 URL: https://issues.apache.org/jira/browse/HIVE-7353 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.14.0 While using embedded metastore, when creating background threads to run async operations, HiveServer2 ends up creating new instances of JDOPersistanceManager rather than using the one from the foreground (handler) thread. Since JDOPersistanceManagerFactory caches JDOPersistanceManager instances, they are never GCed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7353) HiveServer2 using embedded MetaStore leaks JDOPersistanceManager
[ https://issues.apache.org/jira/browse/HIVE-7353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-7353: --- Description: While using embedded metastore, while creating background threads to run async operations, HiveServer2 ends up creating new instances of JDOPersistanceManager rather than using the one from the foreground (handler) thread. Since JDOPersistanceManagerFactory caches JDOPersistanceManager instances, they are never GCed. (was: While using embedded metastore, when creating background threads to run async operations, HiveServer2 ends up creating new instances of JDOPersistanceManager rather than using the one from the foreground (handler) thread. Since JDOPersistanceManagerFactory caches JDOPersistanceManager instances, they are never GCed.) HiveServer2 using embedded MetaStore leaks JDOPersistanceManager Key: HIVE-7353 URL: https://issues.apache.org/jira/browse/HIVE-7353 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.14.0 While using embedded metastore, while creating background threads to run async operations, HiveServer2 ends up creating new instances of JDOPersistanceManager rather than using the one from the foreground (handler) thread. Since JDOPersistanceManagerFactory caches JDOPersistanceManager instances, they are never GCed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7341) Support for Table replication across HCatalog instances
[
https://issues.apache.org/jira/browse/HIVE-7341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mithun Radhakrishnan updated HIVE-7341:
---
Attachment: (was: HIVE-7341.1.patch)
Support for Table replication across HCatalog instances
---
Key: HIVE-7341
URL: https://issues.apache.org/jira/browse/HIVE-7341
Project: Hive
Issue Type: New Feature
Components: HCatalog
Affects Versions: 0.13.1
Reporter: Mithun Radhakrishnan
Assignee: Mithun Radhakrishnan
Fix For: 0.14.0
The HCatClient currently doesn't provide very much support for replicating
HCatTable definitions between 2 HCatalog Server (i.e. Hive metastore)
instances.
Systems similar to Apache Falcon might find the need to replicate partition
data between 2 clusters, and keep the HCatalog metadata in sync between the
two. This poses a couple of problems:
# The definition of the source table might change (in column schema, I/O
formats, record-formats, serde-parameters, etc.) The system will need a way
to diff 2 tables and update the target-metastore with the changes. E.g.
{code}
targetTable.resolve( sourceTable, targetTable.diff(sourceTable) );
hcatClient.updateTableSchema(dbName, tableName, targetTable);
{code}
# The current {{HCatClient.addPartitions()}} API requires that the
partition's schema be derived from the table's schema, thereby requiring that
the table-schema be resolved *before* partitions with the new schema are
added to the table. This is problematic, because it introduces race
conditions when 2 partitions with differing column-schemas (e.g. right after
a schema change) are copied in parallel. This can be avoided if each
HCatAddPartitionDesc kept track of the partition's schema, in flight.
# The source and target metastores might be running different/incompatible
versions of Hive.
The impending patch attempts to address these concerns (with some caveats).
# {{HCatTable}} now has
## a {{diff()}} method, to compare against another HCatTable instance
## a {{resolve(diff)}} method to copy over specified table-attributes from
another HCatTable
## a serialize/deserialize mechanism (via {{HCatClient.serializeTable()}} and
{{HCatClient.deserializeTable()}}), so that HCatTable instances constructed
in other class-loaders may be used for comparison
# {{HCatPartition}} now provides finer-grained control over a Partition's
column-schema, StorageDescriptor settings, etc. This allows partitions to be
copied completely from source, with the ability to override specific
properties if required (e.g. location).
# {{HCatClient.updateTableSchema()}} can now update the entire
table-definition, not just the column schema.
# I've cleaned up and removed most of the redundancy between the HCatTable,
HCatCreateTableDesc and HCatCreateTableDesc.Builder. The prior API failed to
separate the table-attributes from the add-table-operation's attributes. By
providing fluent-interfaces in HCatTable, and composing an HCatTable instance
in HCatCreateTableDesc, the interfaces are cleaner(ish). The old setters are
deprecated, in favour of those in HCatTable. Likewise, HCatPartition and
HCatAddPartitionDesc.
I'll post a patch for trunk shortly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7341) Support for Table replication across HCatalog instances
[
https://issues.apache.org/jira/browse/HIVE-7341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mithun Radhakrishnan updated HIVE-7341:
---
Attachment: HIVE-7341.1.patch
Updated patch with the missing class.
Support for Table replication across HCatalog instances
---
Key: HIVE-7341
URL: https://issues.apache.org/jira/browse/HIVE-7341
Project: Hive
Issue Type: New Feature
Components: HCatalog
Affects Versions: 0.13.1
Reporter: Mithun Radhakrishnan
Assignee: Mithun Radhakrishnan
Fix For: 0.14.0
Attachments: HIVE-7341.1.patch
The HCatClient currently doesn't provide very much support for replicating
HCatTable definitions between 2 HCatalog Server (i.e. Hive metastore)
instances.
Systems similar to Apache Falcon might find the need to replicate partition
data between 2 clusters, and keep the HCatalog metadata in sync between the
two. This poses a couple of problems:
# The definition of the source table might change (in column schema, I/O
formats, record-formats, serde-parameters, etc.) The system will need a way
to diff 2 tables and update the target-metastore with the changes. E.g.
{code}
targetTable.resolve( sourceTable, targetTable.diff(sourceTable) );
hcatClient.updateTableSchema(dbName, tableName, targetTable);
{code}
# The current {{HCatClient.addPartitions()}} API requires that the
partition's schema be derived from the table's schema, thereby requiring that
the table-schema be resolved *before* partitions with the new schema are
added to the table. This is problematic, because it introduces race
conditions when 2 partitions with differing column-schemas (e.g. right after
a schema change) are copied in parallel. This can be avoided if each
HCatAddPartitionDesc kept track of the partition's schema, in flight.
# The source and target metastores might be running different/incompatible
versions of Hive.
The impending patch attempts to address these concerns (with some caveats).
# {{HCatTable}} now has
## a {{diff()}} method, to compare against another HCatTable instance
## a {{resolve(diff)}} method to copy over specified table-attributes from
another HCatTable
## a serialize/deserialize mechanism (via {{HCatClient.serializeTable()}} and
{{HCatClient.deserializeTable()}}), so that HCatTable instances constructed
in other class-loaders may be used for comparison
# {{HCatPartition}} now provides finer-grained control over a Partition's
column-schema, StorageDescriptor settings, etc. This allows partitions to be
copied completely from source, with the ability to override specific
properties if required (e.g. location).
# {{HCatClient.updateTableSchema()}} can now update the entire
table-definition, not just the column schema.
# I've cleaned up and removed most of the redundancy between the HCatTable,
HCatCreateTableDesc and HCatCreateTableDesc.Builder. The prior API failed to
separate the table-attributes from the add-table-operation's attributes. By
providing fluent-interfaces in HCatTable, and composing an HCatTable instance
in HCatCreateTableDesc, the interfaces are cleaner(ish). The old setters are
deprecated, in favour of those in HCatTable. Likewise, HCatPartition and
HCatAddPartitionDesc.
I'll post a patch for trunk shortly.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Review Request 23319: HIVE-7342 - support hiveserver2, metastore specific config files
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23319/ --- Review request for hive. Bugs: HIVE-7342 https://issues.apache.org/jira/browse/HIVE-7342 Repository: hive-git Description --- See jira Diffs - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8bff2a9 data/conf/hive-site.xml 1c9c598 data/conf/hivemetastore-site.xml PRE-CREATION data/conf/hiveserver2-site.xml PRE-CREATION itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestServerSpecificConfig.java PRE-CREATION metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java acef599 metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java 664dccd service/src/java/org/apache/hive/service/cli/thrift/EmbeddedThriftBinaryCLIService.java 62b1d9c service/src/java/org/apache/hive/service/server/HiveServer2.java e7ed267 Diff: https://reviews.apache.org/r/23319/diff/ Testing --- New tests added Thanks, Thejas Nair
Review Request 23320: HiveServer2 using embedded MetaStore leaks JDOPersistanceManager
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23320/ --- Review request for hive, Sushanth Sowmyan and Thejas Nair. Bugs: HIVE-7353 https://issues.apache.org/jira/browse/HIVE-7353 Repository: hive-git Description --- https://issues.apache.org/jira/browse/HIVE-7353 Diffs - metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java acef599 service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java de54ca1 Diff: https://reviews.apache.org/r/23320/diff/ Testing --- Manual testing using Yourkit. Thanks, Vaibhav Gumashta
[jira] [Commented] (HIVE-5510) [WebHCat] GET job/queue return wrong job information
[ https://issues.apache.org/jira/browse/HIVE-5510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054171#comment-14054171 ] Lefty Leverenz commented on HIVE-5510: -- You're right, and I was confused -- I thought the first example was for Hive 0.11 and earlier, but GET jobs was introduced in Hive 0.12 so why do we even have that section? Similarly, in JSON Output (fields, Hive 0.12.0 and later) the version information isn't needed because the new behavior of the fields parameter came in the same release as GET jobs. Recommendations: # Delete the Bug Fix box after the curl command. # Delete the entire JSON Output (fields) section. # Rename JSON Output (fields, Hive 0.12.0 and later) to JSON Output (fields). # Rename the version box Bug in Hive 0.12 or some such. # Delete the first sentence in the version box. * [Curl Command (fields) | https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference+Jobs#WebHCatReferenceJobs-CurlCommand(fields)] * [JSON Output (fields) | https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference+Jobs#WebHCatReferenceJobs-JSONOutput(fields)] * [JSON Output (fields, Hive 0.12.0 and later) | https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference+Jobs#WebHCatReferenceJobs-JSONOutput(fields,Hive0.12.0andlater)] [WebHCat] GET job/queue return wrong job information Key: HIVE-5510 URL: https://issues.apache.org/jira/browse/HIVE-5510 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.12.0 Reporter: Daniel Dai Assignee: Daniel Dai Fix For: 0.13.0 Attachments: HIVE-5510-1.patch, HIVE-5510-2.patch, HIVE-5510-3.patch, HIVE-5510-4.patch, test_harnesss_1381798977 GET job/queue of a TempletonController job return weird information. It is a mix of child job and itself. It should only pull the information of the controller job itself. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7353) HiveServer2 using embedded MetaStore leaks JDOPersistanceManager
[ https://issues.apache.org/jira/browse/HIVE-7353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-7353: --- Attachment: HIVE-7353.1.patch HiveServer2 using embedded MetaStore leaks JDOPersistanceManager Key: HIVE-7353 URL: https://issues.apache.org/jira/browse/HIVE-7353 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.14.0 Attachments: HIVE-7353.1.patch While using embedded metastore, while creating background threads to run async operations, HiveServer2 ends up creating new instances of JDOPersistanceManager rather than using the one from the foreground (handler) thread. Since JDOPersistanceManagerFactory caches JDOPersistanceManager instances, they are never GCed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7353) HiveServer2 using embedded MetaStore leaks JDOPersistanceManager
[ https://issues.apache.org/jira/browse/HIVE-7353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054185#comment-14054185 ] Vaibhav Gumashta commented on HIVE-7353: [~bbowman410] Thanks for working with me on this bug. I've attached the patch for review. cc [~thejas] HiveServer2 using embedded MetaStore leaks JDOPersistanceManager Key: HIVE-7353 URL: https://issues.apache.org/jira/browse/HIVE-7353 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.13.0 Reporter: Vaibhav Gumashta Assignee: Vaibhav Gumashta Fix For: 0.14.0 Attachments: HIVE-7353.1.patch While using embedded metastore, while creating background threads to run async operations, HiveServer2 ends up creating new instances of JDOPersistanceManager rather than using the one from the foreground (handler) thread. Since JDOPersistanceManagerFactory caches JDOPersistanceManager instances, they are never GCed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7346) Wrong results caused by hive ppd under specific join condition
[
https://issues.apache.org/jira/browse/HIVE-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-7346:
---
Resolution: Fixed
Fix Version/s: 0.14.0
Status: Resolved (was: Patch Available)
Committed to trunk. Thanks, Navis!
Wrong results caused by hive ppd under specific join condition
--
Key: HIVE-7346
URL: https://issues.apache.org/jira/browse/HIVE-7346
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0, 0.13.0, 0.13.1
Reporter: dima machlin
Assignee: Navis
Fix For: 0.14.0
Attachments: HIVE-7346.1.patch.txt
Assuming two tables :
{code:sql} t1(id1 string,id2 string) , t2 (id string,d int) {code}
t1 contains 1 row : 'a','a'
t2 contains 1 row : 'a',2
The following query :
{code:sql} select a.*,b.d d1,c.d d2
from t1 a join t2 b on (a.id1=b.id)
join t2 c on (a.id2=b.id)
where b.d =1 and c.d=1 {code}
Returns 0 rows as expected because t2.d = 2
Wrapping this query, like so :
{code:sql} select * from (
select a.*,b.d d1,c.d d2
from t1 a join t2 b on (a.id1=b.id)
join t2 c on (a.id2=b.id)
where b.d =1 and c.d=1
) z where d11 or d21 {code}
Where another filter was add on the columns causes the plan to lack the
filter of the =1 and return a single row - *Wrong Results*.
The plan is :
{code:sql}
ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_SUBQUERY (TOK_QUERY (TOK_FROM (TOK_JOIN (TOK_JOIN
(TOK_TABREF (TOK_TABNAME t1) a) (TOK_TABREF (TOK_TABNAME t2) b) (= (.
(TOK_TABLE_OR_COL a) id1) (. (TOK_TABLE_OR_COL b) id))) (TOK_TABREF
(TOK_TABNAME t2) c) (= (. (TOK_TABLE_OR_COL a) id2) (. (TOK_TABLE_OR_COL b)
id (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT
(TOK_SELEXPR (TOK_ALLCOLREF (TOK_TABNAME a))) (TOK_SELEXPR (.
(TOK_TABLE_OR_COL b) d) d1) (TOK_SELEXPR (. (TOK_TABLE_OR_COL c) d) d2))
(TOK_WHERE (and (= (. (TOK_TABLE_OR_COL b) d) 1) (= (. (TOK_TABLE_OR_COL c)
d) 1) z)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE))
(TOK_SELECT (TOK_SELEXPR TOK_ALLCOLREF)) (TOK_WHERE (or ( (TOK_TABLE_OR_COL
d1) 1) ( (TOK_TABLE_OR_COL d2) 1)
STAGE DEPENDENCIES:
Stage-7 is a root stage
Stage-5 depends on stages: Stage-7
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-7
Map Reduce Local Work
Alias - Map Local Tables:
z:b
Fetch Operator
limit: -1
z:c
Fetch Operator
limit: -1
Alias - Map Local Operator Tree:
z:b
TableScan
alias: b
HashTable Sink Operator
condition expressions:
0 {id1} {id2}
1 {id} {d}
handleSkewJoin: false
keys:
0 [Column[id1]]
1 [Column[id]]
Position of Big Table: 0
z:c
TableScan
alias: c
HashTable Sink Operator
condition expressions:
0 {_col5} {_col0} {_col1}
1 {d}
handleSkewJoin: false
keys:
0 []
1 []
Position of Big Table: 0
Stage: Stage-5
Map Reduce
Alias - Map Operator Tree:
z:a
TableScan
alias: a
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {id1} {id2}
1 {id} {d}
handleSkewJoin: false
keys:
0 [Column[id1]]
1 [Column[id]]
outputColumnNames: _col0, _col1, _col4, _col5
Position of Big Table: 0
Filter Operator
predicate:
expr: (_col1 = _col4)
type: boolean
Map Join Operator
condition map:
Inner Join 0 to 1
condition expressions:
0 {_col5} {_col0} {_col1}
1 {d}
handleSkewJoin: false
keys:
0 []
1 []
outputColumnNames: _col1, _col4, _col5, _col9
Position of Big Table: 0
Filter Operator
predicate:
expr: ((_col1 1) or (_col9 1))
type: boolean
Select Operator
expressions:
expr: _col4
type: string
expr: _col5
type: string
expr: _col1
[jira] [Commented] (HIVE-7294) sql std auth - authorize show grant statements
[ https://issues.apache.org/jira/browse/HIVE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054216#comment-14054216 ] Lefty Leverenz commented on HIVE-7294: -- This needs to be documented for SHOW GRANT and presumably also SHOW ROLE GRANT: * [SQL Standard Based Hive Authorization - Show Grant | https://cwiki.apache.org/confluence/display/Hive/SQL+Standard+Based+Hive+Authorization#SQLStandardBasedHiveAuthorization-ShowGrant] * [SQL Standard Based Hive Authorization - Show Role Grant | https://cwiki.apache.org/confluence/display/Hive/SQL+Standard+Based+Hive+Authorization#SQLStandardBasedHiveAuthorization-ShowRoleGrant] (Is this just for SQL standards based authorization? If not, see [https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Authorization#LanguageManualAuthorization-ViewingGrantedPrivileges] and [https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Authorization#LanguageManualAuthorization-ViewingGrantedRoles].) sql std auth - authorize show grant statements -- Key: HIVE-7294 URL: https://issues.apache.org/jira/browse/HIVE-7294 Project: Hive Issue Type: Bug Components: Authorization, SQLStandardAuthorization Reporter: Thejas M Nair Assignee: Thejas M Nair Fix For: 0.14.0 Attachments: HIVE-7294.1.patch, HIVE-7294.2.patch A non admin user should not be allowed to run show grant commands only for themselves or a role they belong to. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7294) sql std auth - authorize show grant statements
[ https://issues.apache.org/jira/browse/HIVE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-7294: - Labels: TODOC14 (was: ) sql std auth - authorize show grant statements -- Key: HIVE-7294 URL: https://issues.apache.org/jira/browse/HIVE-7294 Project: Hive Issue Type: Bug Components: Authorization, SQLStandardAuthorization Reporter: Thejas M Nair Assignee: Thejas M Nair Labels: TODOC14 Fix For: 0.14.0 Attachments: HIVE-7294.1.patch, HIVE-7294.2.patch A non admin user should not be allowed to run show grant commands only for themselves or a role they belong to. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7045) Wrong results in multi-table insert aggregating without group by clause
[ https://issues.apache.org/jira/browse/HIVE-7045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-7045: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Navis! Wrong results in multi-table insert aggregating without group by clause --- Key: HIVE-7045 URL: https://issues.apache.org/jira/browse/HIVE-7045 Project: Hive Issue Type: Bug Affects Versions: 0.10.0, 0.12.0 Reporter: dima machlin Assignee: Navis Priority: Blocker Fix For: 0.14.0 Attachments: HIVE-7045.1.patch.txt This happens whenever there are more than 1 reducers. The scenario : CREATE TABLE t1 (a int, b int); CREATE TABLE t2 (cnt int) PARTITIONED BY (var_name string); insert into table t1 select 1,1 from asd limit 1; insert into table t1 select 2,2 from asd limit 1; t1 contains : 1 1 2 2 from t1 insert overwrite table t2 partition(var_name='a') select count(a) cnt insert overwrite table t2 partition(var_name='b') select count(b) cnt ; select * from t2; returns : 2 a 2 b as expected. Setting the number of reducers higher than 1 : set mapred.reduce.tasks=2; from t1 insert overwrite table t2 partition(var_name='a') select count(a) cnt insert overwrite table t2 partition(var_name='b') select count(b) cnt; select * from t2; 1 a 1 a 1 b 1 b Wrong results. This happens when ever t1 is big enough to automatically generate more than 1 reducers and without specifying it directly. adding group by 1 in the end of each insert solves the problem : from t1 insert overwrite table t2 partition(var_name='a') select count(a) cnt group by 1 insert overwrite table t2 partition(var_name='b') select count(b) cnt group by 1; generates : 2 a 2 b This should work without the group by... The number of rows for each partition will be the amount of reducers. Each reducer calculated a sub total of the count. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Work started] (HIVE-7354) windows:Need to set hbase jars in hadoop classpath explicitly
[ https://issues.apache.org/jira/browse/HIVE-7354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-7354 started by Hari Sankar Sivarama Subramaniyan. windows:Need to set hbase jars in hadoop classpath explicitly - Key: HIVE-7354 URL: https://issues.apache.org/jira/browse/HIVE-7354 Project: Hive Issue Type: Bug Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Hari Sankar Sivarama Subramaniyan Attachments: HIVE-7354.1.patch n windows, when I run following hive-hbase integration test without setting hbase jars in hadoop classpath, it fails with ClassNotFoundException: drop table if exists hbase_1; create table hbase_1(key string, age int) stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' with serdeproperties ( hbase.columns.mapping = info:age); insert overwrite table hbase_1 select name, SUM(age) from studenttab10k group by name; However, in linux this test works even if jars are not explicitly added in hadoop_classpath. On windows, tests work fine if I add necessary hbase jars in classpath. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7354) windows:Need to set hbase jars in hadoop classpath explicitly
[ https://issues.apache.org/jira/browse/HIVE-7354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sankar Sivarama Subramaniyan updated HIVE-7354: Attachment: HIVE-7354.1.patch cc-ing [~sushanth] for review. windows:Need to set hbase jars in hadoop classpath explicitly - Key: HIVE-7354 URL: https://issues.apache.org/jira/browse/HIVE-7354 Project: Hive Issue Type: Bug Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Hari Sankar Sivarama Subramaniyan Attachments: HIVE-7354.1.patch n windows, when I run following hive-hbase integration test without setting hbase jars in hadoop classpath, it fails with ClassNotFoundException: drop table if exists hbase_1; create table hbase_1(key string, age int) stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' with serdeproperties ( hbase.columns.mapping = info:age); insert overwrite table hbase_1 select name, SUM(age) from studenttab10k group by name; However, in linux this test works even if jars are not explicitly added in hadoop_classpath. On windows, tests work fine if I add necessary hbase jars in classpath. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7354) windows:Need to set hbase jars in hadoop classpath explicitly
[ https://issues.apache.org/jira/browse/HIVE-7354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hari Sankar Sivarama Subramaniyan updated HIVE-7354: Status: Patch Available (was: In Progress) windows:Need to set hbase jars in hadoop classpath explicitly - Key: HIVE-7354 URL: https://issues.apache.org/jira/browse/HIVE-7354 Project: Hive Issue Type: Bug Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Hari Sankar Sivarama Subramaniyan Attachments: HIVE-7354.1.patch n windows, when I run following hive-hbase integration test without setting hbase jars in hadoop classpath, it fails with ClassNotFoundException: drop table if exists hbase_1; create table hbase_1(key string, age int) stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' with serdeproperties ( hbase.columns.mapping = info:age); insert overwrite table hbase_1 select name, SUM(age) from studenttab10k group by name; However, in linux this test works even if jars are not explicitly added in hadoop_classpath. On windows, tests work fine if I add necessary hbase jars in classpath. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-7354) windows:Need to set hbase jars in hadoop classpath explicitly
Hari Sankar Sivarama Subramaniyan created HIVE-7354: --- Summary: windows:Need to set hbase jars in hadoop classpath explicitly Key: HIVE-7354 URL: https://issues.apache.org/jira/browse/HIVE-7354 Project: Hive Issue Type: Bug Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Hari Sankar Sivarama Subramaniyan Attachments: HIVE-7354.1.patch n windows, when I run following hive-hbase integration test without setting hbase jars in hadoop classpath, it fails with ClassNotFoundException: drop table if exists hbase_1; create table hbase_1(key string, age int) stored by 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' with serdeproperties ( hbase.columns.mapping = info:age); insert overwrite table hbase_1 select name, SUM(age) from studenttab10k group by name; However, in linux this test works even if jars are not explicitly added in hadoop_classpath. On windows, tests work fine if I add necessary hbase jars in classpath. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7326) Hive complains invalid column reference with 'having' aggregate predicates
[ https://issues.apache.org/jira/browse/HIVE-7326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-7326: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Navis! Hive complains invalid column reference with 'having' aggregate predicates -- Key: HIVE-7326 URL: https://issues.apache.org/jira/browse/HIVE-7326 Project: Hive Issue Type: Bug Reporter: Hari Sankar Sivarama Subramaniyan Assignee: Navis Fix For: 0.14.0 Attachments: HIVE-7326.1.patch.txt, HIVE-7326.2.patch.txt CREATE TABLE TestV1_Staples ( Item_Count INT, Ship_Priority STRING, Order_Priority STRING, Order_Status STRING, Order_Quantity DOUBLE, Sales_Total DOUBLE, Discount DOUBLE, Tax_Rate DOUBLE, Ship_Mode STRING, Fill_Time DOUBLE, Gross_Profit DOUBLE, Price DOUBLE, Ship_Handle_Cost DOUBLE, Employee_Name STRING, Employee_Dept STRING, Manager_Name STRING, Employee_Yrs_Exp DOUBLE, Employee_Salary DOUBLE, Customer_Name STRING, Customer_State STRING, Call_Center_Region STRING, Customer_Balance DOUBLE, Customer_Segment STRING, Prod_Type1 STRING, Prod_Type2 STRING, Prod_Type3 STRING, Prod_Type4 STRING, Product_Name STRING, Product_Container STRING, Ship_Promo STRING, Supplier_Name STRING, Supplier_Balance DOUBLE, Supplier_Region STRING, Supplier_State STRING, Order_ID STRING, Order_Year INT, Order_Month INT, Order_Day INT, Order_Date_ STRING, Order_Quarter STRING, Product_Base_Margin DOUBLE, Product_ID STRING, Receive_Time DOUBLE, Received_Date_ STRING, Ship_Date_ STRING, Ship_Charge DOUBLE, Total_Cycle_Time DOUBLE, Product_In_Stock STRING, PID INT, Market_Segment STRING ); Query that works: SELECT customer_name, SUM(customer_balance), SUM(order_quantity) FROM default.testv1_staples s1 GROUP BY customer_name HAVING ( (COUNT(s1.discount) = 822) AND (SUM(customer_balance) = 4074689.00041) ); Query that fails: SELECT customer_name, SUM(customer_balance), SUM(order_quantity) FROM default.testv1_staples s1 GROUP BY customer_name HAVING ( (SUM(customer_balance) = 4074689.00041) AND (COUNT(s1.discount) = 822) ); -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7342) support hiveserver2,metastore specific config files
[ https://issues.apache.org/jira/browse/HIVE-7342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-7342: Status: Patch Available (was: Open) support hiveserver2,metastore specific config files --- Key: HIVE-7342 URL: https://issues.apache.org/jira/browse/HIVE-7342 Project: Hive Issue Type: Bug Components: Configuration, HiveServer2, Metastore Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-7342.1.patch There is currently a single configuration file for all components in hive. ie, components such as hive cli, hiveserver2 and metastore all read from the same hive-site.xml. It will be useful to have a server specific hive-site.xml, so that you can have some different configuration value set for a server. For example, you might want to enabled authorization checks for hiveserver2, while disabling the checks for hive cli. The workaround today is to add any component specific configuration as a commandline (-hiveconf) argument. Using server specific config files (eg hiveserver2-site.xml, metastore-site.xml) that override the entries in hive-site.xml will make the configuration much more easy to manage. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7143) Add Streaming support in Windowing mode for more UDAFs (min/max, lead/lag, fval/lval)
[ https://issues.apache.org/jira/browse/HIVE-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054244#comment-14054244 ] Lefty Leverenz commented on HIVE-7143: -- See user doc comments on HIVE-7062. Add Streaming support in Windowing mode for more UDAFs (min/max, lead/lag, fval/lval) - Key: HIVE-7143 URL: https://issues.apache.org/jira/browse/HIVE-7143 Project: Hive Issue Type: Bug Reporter: Harish Butani Assignee: Harish Butani Fix For: 0.14.0 Attachments: HIVE-7143.1.patch, HIVE-7143.3.patch Provided implementations for Streaming for the above fns. Min/Max based on Alg by Daniel Lemire: http://www.archipel.uqam.ca/309/1/webmaximinalgo.pdf -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7319) Add allStreamable to PTFDesc object
[ https://issues.apache.org/jira/browse/HIVE-7319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-7319: --- Summary: Add allStreamable to PTFDesc object (was: Add isStreamable to PTFDesc object) Add allStreamable to PTFDesc object --- Key: HIVE-7319 URL: https://issues.apache.org/jira/browse/HIVE-7319 Project: Hive Issue Type: Sub-task Reporter: Matt McCline Assignee: Matt McCline Need isStreamable in PTFDesc to allow Vectorizer class to decide whether to vectorize the PTF operator. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7344) Add streaming support in Windowing mode for FirstVal, LastVal
[ https://issues.apache.org/jira/browse/HIVE-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054246#comment-14054246 ] Lefty Leverenz commented on HIVE-7344: -- Add user doc for this with HIVE-7062. Add streaming support in Windowing mode for FirstVal, LastVal - Key: HIVE-7344 URL: https://issues.apache.org/jira/browse/HIVE-7344 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Harish Butani Assignee: Harish Butani Fix For: 0.14.0 Attachments: HIVE-7344.1.patch Continuation of HIVE-7062, HIVE-7143 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7143) Add Streaming support in Windowing mode for more UDAFs (min/max, lead/lag, fval/lval)
[ https://issues.apache.org/jira/browse/HIVE-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-7143: - Labels: TODOC14 (was: ) Add Streaming support in Windowing mode for more UDAFs (min/max, lead/lag, fval/lval) - Key: HIVE-7143 URL: https://issues.apache.org/jira/browse/HIVE-7143 Project: Hive Issue Type: Bug Reporter: Harish Butani Assignee: Harish Butani Labels: TODOC14 Fix For: 0.14.0 Attachments: HIVE-7143.1.patch, HIVE-7143.3.patch Provided implementations for Streaming for the above fns. Min/Max based on Alg by Daniel Lemire: http://www.archipel.uqam.ca/309/1/webmaximinalgo.pdf -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7322) Vectorize PTF Operator {Windowable, Streamable} add constant lag/lead window aggregates / functions (i.e. window max/max, etc)
[
https://issues.apache.org/jira/browse/HIVE-7322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt McCline updated HIVE-7322:
---
Summary: Vectorize PTF Operator {Windowable, Streamable} add constant
lag/lead window aggregates / functions (i.e. window max/max, etc) (was:
Vectorize PTF Operator {Windowable, Streamable} add constant lag/lead window
aggregates / functions (i.e. window max, window max, etc))
Vectorize PTF Operator {Windowable, Streamable} add constant lag/lead window
aggregates / functions (i.e. window max/max, etc)
--
Key: HIVE-7322
URL: https://issues.apache.org/jira/browse/HIVE-7322
Project: Hive
Issue Type: Sub-task
Reporter: Matt McCline
Assignee: Matt McCline
Multiple output buffers needed
Can’t forward oldest output buffer until all computed columns are finished.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7344) Add streaming support in Windowing mode for FirstVal, LastVal
[ https://issues.apache.org/jira/browse/HIVE-7344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-7344: - Labels: TODOC14 (was: ) Add streaming support in Windowing mode for FirstVal, LastVal - Key: HIVE-7344 URL: https://issues.apache.org/jira/browse/HIVE-7344 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Harish Butani Assignee: Harish Butani Labels: TODOC14 Fix For: 0.14.0 Attachments: HIVE-7344.1.patch Continuation of HIVE-7062, HIVE-7143 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-7355) Vectorize Remaining Data Types
Matt McCline created HIVE-7355: -- Summary: Vectorize Remaining Data Types Key: HIVE-7355 URL: https://issues.apache.org/jira/browse/HIVE-7355 Project: Hive Issue Type: Bug Reporter: Matt McCline Assignee: Matt McCline Vectorized the remaining data types so queries are not arbitrarily not vectorized because they happen to reference a non-supported data type. Current non-supported vector data types: CHAR, VARCHAR, BINARY and the collection types (STRUCT, ARRAY, MAP). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-7356) Table level stats collection fail for partitioned tables
Ashutosh Chauhan created HIVE-7356: -- Summary: Table level stats collection fail for partitioned tables Key: HIVE-7356 URL: https://issues.apache.org/jira/browse/HIVE-7356 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.14.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7144) GC pressure during ORC StringDictionary writes
[ https://issues.apache.org/jira/browse/HIVE-7144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-7144: -- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk, thanks [~hagleitn]! GC pressure during ORC StringDictionary writes --- Key: HIVE-7144 URL: https://issues.apache.org/jira/browse/HIVE-7144 Project: Hive Issue Type: Bug Components: File Formats Affects Versions: 0.14.0 Environment: ORC Table ~ 12 string columns Reporter: Gopal V Assignee: Gopal V Labels: ORC, Performance Fix For: 0.14.0 Attachments: HIVE-7144.1.patch, HIVE-7144.2.patch, HIVE-7144.3.patch, orc-string-write.png When ORC string dictionary writes data out, it suffers from bad GC performance due to a few allocations in-loop. !orc-string-write.png! The conversions are as follows StringTreeWriter::getStringValue() causes 2 conversions LazyString - Text (LazyString::getWritableObject) Text - String (LazyStringObjectInspector::getPrimitiveJavaObject) Then StringRedBlackTree::add() does one conversion String - Text This causes some GC pressure with un-necessary String and byte[] array allocations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7356) Table level stats collection fail for partitioned tables
[ https://issues.apache.org/jira/browse/HIVE-7356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-7356: --- Attachment: HIVE-7356.patch Table level stats collection fail for partitioned tables Key: HIVE-7356 URL: https://issues.apache.org/jira/browse/HIVE-7356 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.14.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-7356.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7356) Table level stats collection fail for partitioned tables
[ https://issues.apache.org/jira/browse/HIVE-7356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-7356: --- Status: Patch Available (was: Open) Table level stats collection fail for partitioned tables Key: HIVE-7356 URL: https://issues.apache.org/jira/browse/HIVE-7356 Project: Hive Issue Type: Bug Components: Statistics Affects Versions: 0.14.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-7356.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
Review Request 23325: Table level stats collection fail for partitioned tables
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/23325/ --- Review request for hive. Bugs: HIVE-7356 https://issues.apache.org/jira/browse/HIVE-7356 Repository: hive Description --- Table level stats collection fail for partitioned tables Diffs - trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsTask.java 1608624 trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ColumnStatsSemanticAnalyzer.java 1608624 trunk/ql/src/test/queries/clientpositive/columnstats_partlvl.q 1608624 trunk/ql/src/test/results/clientpositive/columnstats_partlvl.q.out 1608624 Diff: https://reviews.apache.org/r/23325/diff/ Testing --- Added new test. Thanks, Ashutosh Chauhan
[jira] [Updated] (HIVE-5760) Add vectorized support for CHAR/VARCHAR data type
[ https://issues.apache.org/jira/browse/HIVE-5760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-5760: --- Issue Type: Bug (was: Sub-task) Parent: (was: HIVE-4160) Add vectorized support for CHAR/VARCHAR data type - Key: HIVE-5760 URL: https://issues.apache.org/jira/browse/HIVE-5760 Project: Hive Issue Type: Bug Reporter: Eric Hanson Add support to allow queries referencing VARCHAR columns and expression results to run efficiently in vectorized mode. This should re-use the code for the STRING type to the extent possible and beneficial. Include unit tests and end-to-end tests. Consider re-using or extending existing end-to-end tests for vectorized string operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5760) Add vectorized support for CHAR/VARCHAR data type
[ https://issues.apache.org/jira/browse/HIVE-5760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-5760: --- Summary: Add vectorized support for CHAR/VARCHAR data type (was: Add vectorized support for VARCHAR data type) Add vectorized support for CHAR/VARCHAR data type - Key: HIVE-5760 URL: https://issues.apache.org/jira/browse/HIVE-5760 Project: Hive Issue Type: Sub-task Reporter: Eric Hanson Add support to allow queries referencing VARCHAR columns and expression results to run efficiently in vectorized mode. This should re-use the code for the STRING type to the extent possible and beneficial. Include unit tests and end-to-end tests. Consider re-using or extending existing end-to-end tests for vectorized string operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HIVE-5760) Add vectorized support for CHAR/VARCHAR data type
[ https://issues.apache.org/jira/browse/HIVE-5760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline reassigned HIVE-5760: -- Assignee: Matt McCline Add vectorized support for CHAR/VARCHAR data type - Key: HIVE-5760 URL: https://issues.apache.org/jira/browse/HIVE-5760 Project: Hive Issue Type: Sub-task Reporter: Eric Hanson Assignee: Matt McCline Add support to allow queries referencing VARCHAR columns and expression results to run efficiently in vectorized mode. This should re-use the code for the STRING type to the extent possible and beneficial. Include unit tests and end-to-end tests. Consider re-using or extending existing end-to-end tests for vectorized string operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-7357) Add vectorized support for BINARY data type
Matt McCline created HIVE-7357: -- Summary: Add vectorized support for BINARY data type Key: HIVE-7357 URL: https://issues.apache.org/jira/browse/HIVE-7357 Project: Hive Issue Type: Sub-task Reporter: Matt McCline Assignee: Matt McCline -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5760) Add vectorized support for CHAR/VARCHAR data type
[ https://issues.apache.org/jira/browse/HIVE-5760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-5760: --- Issue Type: Sub-task (was: Bug) Parent: HIVE-7355 Add vectorized support for CHAR/VARCHAR data type - Key: HIVE-5760 URL: https://issues.apache.org/jira/browse/HIVE-5760 Project: Hive Issue Type: Sub-task Reporter: Eric Hanson Add support to allow queries referencing VARCHAR columns and expression results to run efficiently in vectorized mode. This should re-use the code for the STRING type to the extent possible and beneficial. Include unit tests and end-to-end tests. Consider re-using or extending existing end-to-end tests for vectorized string operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-7358) Add vectorized support for STRUCT data type
Matt McCline created HIVE-7358: -- Summary: Add vectorized support for STRUCT data type Key: HIVE-7358 URL: https://issues.apache.org/jira/browse/HIVE-7358 Project: Hive Issue Type: Sub-task Reporter: Matt McCline Assignee: Matt McCline ORC separates STRUCT fields into vectors. So, we would probably have vectors for each field, too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5760) Add vectorized support for CHAR/VARCHAR data types
[ https://issues.apache.org/jira/browse/HIVE-5760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-5760: --- Summary: Add vectorized support for CHAR/VARCHAR data types (was: Add vectorized support for CHAR/VARCHAR data type) Add vectorized support for CHAR/VARCHAR data types -- Key: HIVE-5760 URL: https://issues.apache.org/jira/browse/HIVE-5760 Project: Hive Issue Type: Sub-task Reporter: Eric Hanson Assignee: Matt McCline Add support to allow queries referencing VARCHAR columns and expression results to run efficiently in vectorized mode. This should re-use the code for the STRING type to the extent possible and beneficial. Include unit tests and end-to-end tests. Consider re-using or extending existing end-to-end tests for vectorized string operations. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-7359) Stats based compute query replies fail to do simple column transforms
Gopal V created HIVE-7359:
-
Summary: Stats based compute query replies fail to do simple
column transforms
Key: HIVE-7359
URL: https://issues.apache.org/jira/browse/HIVE-7359
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Reporter: Gopal V
Assignee: Ashutosh Chauhan
The following two queries return the same answer (the second one is incorrect)
{code}
hive set hive.compute.query.using.stats=true;
hive select count(1) from trips;
OK
187271461
Time taken: 0.173 seconds, Fetched: 1 row(s)
hive select count(1)/5109828 from trips;
OK
187271461
Time taken: 0.125 seconds, Fetched: 1 row(s)
{code}
The second query should have output 36.649 instead of the returning the value
of count(1).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7360) Add vectorized support for ARRAY/MAP data types
Matt McCline created HIVE-7360: -- Summary: Add vectorized support for ARRAY/MAP data types Key: HIVE-7360 URL: https://issues.apache.org/jira/browse/HIVE-7360 Project: Hive Issue Type: Sub-task Reporter: Matt McCline Assignee: Matt McCline -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-4616) Simple reconnection support for jdbc2
[
https://issues.apache.org/jira/browse/HIVE-4616?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054259#comment-14054259
]
Hive QA commented on HIVE-4616:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12654271/HIVE-4616.4.patch.txt
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 5684 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_schemeAuthority
org.apache.hive.minikdc.TestJdbcWithMiniKdc.testConnection
org.apache.hive.minikdc.TestJdbcWithMiniKdc.testNegativeTokenAuth
org.apache.hive.minikdc.TestJdbcWithMiniKdc.testProxyAuth
org.apache.hive.minikdc.TestJdbcWithMiniKdc.testTokenAuth
org.apache.hive.minikdc.TestJdbcWithMiniKdcSQLAuthBinary.testAuthorization1
org.apache.hive.minikdc.TestJdbcWithMiniKdcSQLAuthHttp.testAuthorization1
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/696/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/696/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-696/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12654271
Simple reconnection support for jdbc2
-
Key: HIVE-4616
URL: https://issues.apache.org/jira/browse/HIVE-4616
Project: Hive
Issue Type: Improvement
Components: JDBC
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: HIVE-4616.3.patch.txt, HIVE-4616.4.patch.txt,
HIVE-4616.D10953.1.patch, HIVE-4616.D10953.2.patch
jdbc:hive2://localhost:1/db2;autoReconnect=true
simple reconnection on TransportException. If hiveserver2 has not been
shutdown, session could be reused.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7361) using authorization api for SET, RESET, DFS, ADD, DELETE, COMPILE commands
Thejas M Nair created HIVE-7361: --- Summary: using authorization api for SET, RESET, DFS, ADD, DELETE, COMPILE commands Key: HIVE-7361 URL: https://issues.apache.org/jira/browse/HIVE-7361 Project: Hive Issue Type: Bug Components: Authorization Reporter: Thejas M Nair Assignee: Thejas M Nair The only way to disable the commands SET, RESET, DFS, ADD, DELETE and COMPILE that is available currently is to use the hive.security.command.whitelist parameter. Some of these commands are disabled using this configuration parameter for security reasons when SQL standard authorization is enabled. However, it gets disabled in all cases. If authorization api is used authorize the use of these commands, it will give authorization implementations the flexibility to allow/disallow these commands based on user privileges. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-7362) Enabling Correlation Optimizer by default.
Yin Huai created HIVE-7362: -- Summary: Enabling Correlation Optimizer by default. Key: HIVE-7362 URL: https://issues.apache.org/jira/browse/HIVE-7362 Project: Hive Issue Type: Task Components: Query Processor Reporter: Yin Huai Assignee: Yin Huai -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5343) Add equals method to ObjectInspectorUtils
[
https://issues.apache.org/jira/browse/HIVE-5343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054299#comment-14054299
]
Hive QA commented on HIVE-5343:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12654275/HIVE-5343.3.patch.txt
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 5701 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/697/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/697/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-697/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12654275
Add equals method to ObjectInspectorUtils
-
Key: HIVE-5343
URL: https://issues.apache.org/jira/browse/HIVE-5343
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Trivial
Attachments: D13053.1.patch, HIVE-5343.2.patch.txt,
HIVE-5343.3.patch.txt
Might provide shortcut for some use cases.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7205) Wrong results when union all of grouping followed by group by with correlation optimization
[
https://issues.apache.org/jira/browse/HIVE-7205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054303#comment-14054303
]
Yin Huai commented on HIVE-7205:
Sure. I will take a look at it.
Seems the issue is that the MuxOperator for the last GroupByOperator cannot
correctly determine when to call flush/endGroup/processGroup of the
GroupByOperator because the UnionOperator creates a merging point of two
branches in the operator tree.
Wrong results when union all of grouping followed by group by with
correlation optimization
---
Key: HIVE-7205
URL: https://issues.apache.org/jira/browse/HIVE-7205
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0, 0.13.0, 0.13.1
Reporter: dima machlin
Assignee: Navis
Priority: Critical
Attachments: HIVE-7205.1.patch.txt, HIVE-7205.2.patch.txt,
HIVE-7205.3.patch.txt
use case :
table TBL (a string,b string) contains single row : 'a','a'
the following query :
{code:sql}
select b, sum(cc) from (
select b,count(1) as cc from TBL group by b
union all
select a as b,count(1) as cc from TBL group by a
) z
group by b
{code}
returns
a 1
a 1
while set hive.optimize.correlation=true;
if we change set hive.optimize.correlation=false;
it returns correct results : a 2
The plan with correlation optimization :
{code:sql}
ABSTRACT SYNTAX TREE:
(TOK_QUERY (TOK_FROM (TOK_SUBQUERY (TOK_UNION (TOK_QUERY (TOK_FROM
(TOK_TABREF (TOK_TABNAME DB TBL))) (TOK_INSERT (TOK_DESTINATION (TOK_DIR
TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_TABLE_OR_COL b)) (TOK_SELEXPR
(TOK_FUNCTION count 1) cc)) (TOK_GROUPBY (TOK_TABLE_OR_COL b (TOK_QUERY
(TOK_FROM (TOK_TABREF (TOK_TABNAME DB TBL))) (TOK_INSERT (TOK_DESTINATION
(TOK_DIR TOK_TMP_FILE)) (TOK_SELECT (TOK_SELEXPR (TOK_TABLE_OR_COL a) b)
(TOK_SELEXPR (TOK_FUNCTION count 1) cc)) (TOK_GROUPBY (TOK_TABLE_OR_COL
a) z)) (TOK_INSERT (TOK_DESTINATION (TOK_DIR TOK_TMP_FILE)) (TOK_SELECT
(TOK_SELEXPR (TOK_TABLE_OR_COL b)) (TOK_SELEXPR (TOK_FUNCTION sum
(TOK_TABLE_OR_COL cc (TOK_GROUPBY (TOK_TABLE_OR_COL b
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Alias - Map Operator Tree:
null-subquery1:z-subquery1:TBL
TableScan
alias: TBL
Select Operator
expressions:
expr: b
type: string
outputColumnNames: b
Group By Operator
aggregations:
expr: count(1)
bucketGroup: false
keys:
expr: b
type: string
mode: hash
outputColumnNames: _col0, _col1
Reduce Output Operator
key expressions:
expr: _col0
type: string
sort order: +
Map-reduce partition columns:
expr: _col0
type: string
tag: 0
value expressions:
expr: _col1
type: bigint
null-subquery2:z-subquery2:TBL
TableScan
alias: TBL
Select Operator
expressions:
expr: a
type: string
outputColumnNames: a
Group By Operator
aggregations:
expr: count(1)
bucketGroup: false
keys:
expr: a
type: string
mode: hash
outputColumnNames: _col0, _col1
Reduce Output Operator
key expressions:
expr: _col0
type: string
sort order: +
Map-reduce partition columns:
expr: _col0
type: string
tag: 1
value expressions:
expr: _col1
type: bigint
Reduce Operator Tree:
Demux Operator
Group By Operator
aggregations:
expr: count(VALUE._col0)
bucketGroup: false
keys:
expr: KEY._col0
type: string
mode: mergepartial
outputColumnNames: _col0, _col1
Select Operator
expressions:
expr: _col0
type: string
expr:
[jira] [Commented] (HIVE-7294) sql std auth - authorize show grant statements
[ https://issues.apache.org/jira/browse/HIVE-7294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054311#comment-14054311 ] Thejas M Nair commented on HIVE-7294: - Yes, this is applicable to both 'show grant' and 'show role grant' This is applicable only to sql standard based authorization. If a user is not in admin role, the user can run show grants only using a itself as the principal or a role the user belongs to as the principal. The show grant statements without principal names are not allowed for users not in admin role. sql std auth - authorize show grant statements -- Key: HIVE-7294 URL: https://issues.apache.org/jira/browse/HIVE-7294 Project: Hive Issue Type: Bug Components: Authorization, SQLStandardAuthorization Reporter: Thejas M Nair Assignee: Thejas M Nair Labels: TODOC14 Fix For: 0.14.0 Attachments: HIVE-7294.1.patch, HIVE-7294.2.patch A non admin user should not be allowed to run show grant commands only for themselves or a role they belong to. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-7363) VectorExpressionWriterDecimal is missing null check in setValue()
Jason Dere created HIVE-7363:
Summary: VectorExpressionWriterDecimal is missing null check in
setValue()
Key: HIVE-7363
URL: https://issues.apache.org/jira/browse/HIVE-7363
Project: Hive
Issue Type: Bug
Components: Vectorization
Reporter: Jason Dere
Assignee: Jason Dere
The VectorExpressionWriterDecimal class created by
VectorExpressionWriterFactory.genVectorExpressionWritableDecimal() does not
check for null field in setValue, while all the other data types check for null.
This can result in NPE when inserting to/from Orc tables:
{noformat}
java.lang.Exception: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:401)
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:195)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:233)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:680)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing rowi
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:45)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177)
... 10 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableHiveDecimalObjectInspector.set(WritableHiveDecimalObjectInspector.java:66)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$1.setValue(VectorExpressionWriterFactory.java:481)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterDecimal.setValue(VectorExpressionWriterFactory.java:347)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterSetter.setValue(VectorExpressionWriterFactory.java:1070)
at
org.apache.hadoop.hive.ql.io.orc.VectorizedOrcSerde.serialize(VectorizedOrcSerde.java:75)
at
org.apache.hadoop.hive.ql.io.orc.OrcSerde.serializeVector(OrcSerde.java:148)
at
org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator.processOp(VectorFileSinkOperator.java:79)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:139)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.processOp(VectorFilterOperator.java:102)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
... 11 more
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7363) VectorExpressionWriterDecimal is missing null check in setValue()
[
https://issues.apache.org/jira/browse/HIVE-7363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-7363:
-
Attachment: HIVE-7363.1.patch
VectorExpressionWriterDecimal is missing null check in setValue()
-
Key: HIVE-7363
URL: https://issues.apache.org/jira/browse/HIVE-7363
Project: Hive
Issue Type: Bug
Components: Vectorization
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-7363.1.patch
The VectorExpressionWriterDecimal class created by
VectorExpressionWriterFactory.genVectorExpressionWritableDecimal() does not
check for null field in setValue, while all the other data types check for
null.
This can result in NPE when inserting to/from Orc tables:
{noformat}
java.lang.Exception: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:401)
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:195)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:233)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:680)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing rowi
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:45)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177)
... 10 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableHiveDecimalObjectInspector.set(WritableHiveDecimalObjectInspector.java:66)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$1.setValue(VectorExpressionWriterFactory.java:481)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterDecimal.setValue(VectorExpressionWriterFactory.java:347)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterSetter.setValue(VectorExpressionWriterFactory.java:1070)
at
org.apache.hadoop.hive.ql.io.orc.VectorizedOrcSerde.serialize(VectorizedOrcSerde.java:75)
at
org.apache.hadoop.hive.ql.io.orc.OrcSerde.serializeVector(OrcSerde.java:148)
at
org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator.processOp(VectorFileSinkOperator.java:79)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:139)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.processOp(VectorFilterOperator.java:102)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
... 11 more
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7363) VectorExpressionWriterDecimal is missing null check in setValue()
[
https://issues.apache.org/jira/browse/HIVE-7363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Dere updated HIVE-7363:
-
Status: Patch Available (was: Open)
VectorExpressionWriterDecimal is missing null check in setValue()
-
Key: HIVE-7363
URL: https://issues.apache.org/jira/browse/HIVE-7363
Project: Hive
Issue Type: Bug
Components: Vectorization
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-7363.1.patch
The VectorExpressionWriterDecimal class created by
VectorExpressionWriterFactory.genVectorExpressionWritableDecimal() does not
check for null field in setValue, while all the other data types check for
null.
This can result in NPE when inserting to/from Orc tables:
{noformat}
java.lang.Exception: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:401)
Caused by: java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
processing row
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:195)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:429)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at
org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:233)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:680)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing rowi
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:45)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177)
... 10 more
Caused by: java.lang.NullPointerException
at
org.apache.hadoop.hive.serde2.objectinspector.primitive.WritableHiveDecimalObjectInspector.set(WritableHiveDecimalObjectInspector.java:66)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$1.setValue(VectorExpressionWriterFactory.java:481)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterDecimal.setValue(VectorExpressionWriterFactory.java:347)
at
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorExpressionWriterFactory$VectorExpressionWriterSetter.setValue(VectorExpressionWriterFactory.java:1070)
at
org.apache.hadoop.hive.ql.io.orc.VectorizedOrcSerde.serialize(VectorizedOrcSerde.java:75)
at
org.apache.hadoop.hive.ql.io.orc.OrcSerde.serializeVector(OrcSerde.java:148)
at
org.apache.hadoop.hive.ql.exec.vector.VectorFileSinkOperator.processOp(VectorFileSinkOperator.java:79)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.processOp(VectorSelectOperator.java:139)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorFilterOperator.processOp(VectorFilterOperator.java:102)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:95)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:800)
at
org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.process(VectorMapOperator.java:43)
... 11 more
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-5690) Support subquery for single sourced multi query
[
https://issues.apache.org/jira/browse/HIVE-5690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14054336#comment-14054336
]
Hive QA commented on HIVE-5690:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12654292/HIVE-5690.7.patch.txt
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 5702 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_sortmerge_join_2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynpart_sort_optimization
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket4
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucket5
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin7
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_disable_merge_for_bucketing
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_list_bucket_dml_10
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_reduce_deduplicate
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hive.jdbc.miniHS2.TestHiveServer2.testConnection
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/698/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/698/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-698/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12654292
Support subquery for single sourced multi query
---
Key: HIVE-5690
URL: https://issues.apache.org/jira/browse/HIVE-5690
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Navis
Assignee: Navis
Priority: Minor
Attachments: D13791.1.patch, HIVE-5690.2.patch.txt,
HIVE-5690.3.patch.txt, HIVE-5690.4.patch.txt, HIVE-5690.5.patch.txt,
HIVE-5690.6.patch.txt, HIVE-5690.7.patch.txt
Single sourced multi (insert) query is very useful for various ETL processes
but it does not allow subqueries included. For example,
{noformat}
explain from src
insert overwrite table x1 select * from (select distinct key,value) b order
by key
insert overwrite table x2 select * from (select distinct key,value) c order
by value;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
