Re: [ANNOUNCE] New Hive Committers - Prasanth J and Vaibhav Gumashta
Congratulations! -- Lefty On Fri, Apr 25, 2014 at 12:10 AM, Hari Subramaniyan hsubramani...@hortonworks.com wrote: Congrats Prasanth and Vaibhav! Thanks Hari On Thu, Apr 24, 2014 at 8:45 PM, Chinna Rao Lalam lalamchinnara...@gmail.com wrote: Congratulations to Prasanth and Vaibhav! On Fri, Apr 25, 2014 at 8:23 AM, Shengjun Xin s...@gopivotal.com wrote: Congratulations ~~ On Fri, Apr 25, 2014 at 10:33 AM, Carl Steinbach cwsteinb...@gmail.com wrote: + Prasanth's correct email address On Thu, Apr 24, 2014 at 7:31 PM, Xuefu Zhang xzh...@cloudera.com wrote: Congratulations to Prasanth and Vaibhav! --Xuefu On Thu, Apr 24, 2014 at 7:26 PM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Prasanth J and Vaibhav Gumashta committers on the Apache Hive Project. Please join me in congratulating Prasanth and Vaibhav! Thanks. - Carl -- Regards Shengjun -- Hope It Helps, Chinna -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-5528) hive log file name in local is .log
[
https://issues.apache.org/jira/browse/HIVE-5528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980791#comment-13980791
]
Lefty Leverenz commented on HIVE-5528:
--
[~brocknoland] and [~thejas], can I change the wiki to say this was fixed in
Hive 0.13.0?
* [Getting Started - Error Logs
|https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-ErrorLogs]:
bq. Note: In local mode, the log file name is .log instead of hive.log.
This is a bug which will be fixed in a future release (see HIVE-5528 and
HIVE-5676).
hive log file name in local is .log
-
Key: HIVE-5528
URL: https://issues.apache.org/jira/browse/HIVE-5528
Project: Hive
Issue Type: Bug
Affects Versions: 0.11.0, 0.12.0
Reporter: Thejas M Nair
Fix For: 0.13.0
In local mode the log is getting written to /tmp/{user.name}/.log instead of
/tmp/{user.name}/hive.log
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6571) query id should be available for logging during query compilation
[ https://issues.apache.org/jira/browse/HIVE-6571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980795#comment-13980795 ] Lefty Leverenz commented on HIVE-6571: -- I'll assume this doesn't need to be documented in the wiki. query id should be available for logging during query compilation - Key: HIVE-6571 URL: https://issues.apache.org/jira/browse/HIVE-6571 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Priority: Minor Fix For: 0.13.0 Attachments: HIVE-6571.1.patch Would be nice to have the query id set during compilation to tie logs together etc. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6835) Reading of partitioned Avro data fails if partition schema does not match table schema
[
https://issues.apache.org/jira/browse/HIVE-6835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980802#comment-13980802
]
Hive QA commented on HIVE-6835:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12641763/HIVE-6835.5.patch
{color:red}ERROR:{color} -1 due to 40 failed/errored test(s), 5418 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_filter_numeric
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_test_outer
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_createas1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_truncate_column_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_19
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_20
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_21
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_22
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_24
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_9
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/34/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/34/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 40 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12641763
Reading of partitioned Avro data fails if partition schema does not match
table schema
--
Key: HIVE-6835
URL: https://issues.apache.org/jira/browse/HIVE-6835
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0
Reporter: Anthony Hsu
Assignee: Anthony Hsu
Attachments: HIVE-6835.1.patch, HIVE-6835.2.patch, HIVE-6835.3.patch,
HIVE-6835.4.patch, HIVE-6835.5.patch
To reproduce:
{code}
create table testarray (a arraystring);
load data local inpath '/home/ahsu/test/array.txt' into table testarray;
# create partitioned Avro table with one array column
create table avroarray partitioned by (y string) row format serde
'org.apache.hadoop.hive.serde2.avro.AvroSerDe' with serdeproperties
[jira] [Updated] (HIVE-4577) hive CLI can't handle hadoop dfs command with space and quotes.
[ https://issues.apache.org/jira/browse/HIVE-4577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bing Li updated HIVE-4577: -- Fix Version/s: (was: 0.12.1) 0.14.0 hive CLI can't handle hadoop dfs command with space and quotes. Key: HIVE-4577 URL: https://issues.apache.org/jira/browse/HIVE-4577 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.9.0, 0.10.0 Reporter: Bing Li Assignee: Bing Li Fix For: 0.14.0 Attachments: HIVE-4577.1.patch, HIVE-4577.2.patch, HIVE-4577.3.patch.txt As design, hive could support hadoop dfs command in hive shell, like hive dfs -mkdir /user/biadmin/mydir; but has different behavior with hadoop if the path contains space and quotes hive dfs -mkdir hello; drwxr-xr-x - biadmin supergroup 0 2013-04-23 09:40 /user/biadmin/hello hive dfs -mkdir 'world'; drwxr-xr-x - biadmin supergroup 0 2013-04-23 09:43 /user/biadmin/'world' hive dfs -mkdir bei jing; drwxr-xr-x - biadmin supergroup 0 2013-04-23 09:44 /user/biadmin/bei drwxr-xr-x - biadmin supergroup 0 2013-04-23 09:44 /user/biadmin/jing -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-4577) hive CLI can't handle hadoop dfs command with space and quotes.
[ https://issues.apache.org/jira/browse/HIVE-4577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bing Li updated HIVE-4577: -- Fix Version/s: 0.12.1 hive CLI can't handle hadoop dfs command with space and quotes. Key: HIVE-4577 URL: https://issues.apache.org/jira/browse/HIVE-4577 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.9.0, 0.10.0 Reporter: Bing Li Assignee: Bing Li Fix For: 0.14.0 Attachments: HIVE-4577.1.patch, HIVE-4577.2.patch, HIVE-4577.3.patch.txt As design, hive could support hadoop dfs command in hive shell, like hive dfs -mkdir /user/biadmin/mydir; but has different behavior with hadoop if the path contains space and quotes hive dfs -mkdir hello; drwxr-xr-x - biadmin supergroup 0 2013-04-23 09:40 /user/biadmin/hello hive dfs -mkdir 'world'; drwxr-xr-x - biadmin supergroup 0 2013-04-23 09:43 /user/biadmin/'world' hive dfs -mkdir bei jing; drwxr-xr-x - biadmin supergroup 0 2013-04-23 09:44 /user/biadmin/bei drwxr-xr-x - biadmin supergroup 0 2013-04-23 09:44 /user/biadmin/jing -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-4577) hive CLI can't handle hadoop dfs command with space and quotes.
[ https://issues.apache.org/jira/browse/HIVE-4577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980845#comment-13980845 ] Bing Li commented on HIVE-4577: --- Hi, Thejas I noticed this fix hasn't been included in 0.12 release. I updated the fix for 0.14. thank you. hive CLI can't handle hadoop dfs command with space and quotes. Key: HIVE-4577 URL: https://issues.apache.org/jira/browse/HIVE-4577 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.9.0, 0.10.0 Reporter: Bing Li Assignee: Bing Li Fix For: 0.14.0 Attachments: HIVE-4577.1.patch, HIVE-4577.2.patch, HIVE-4577.3.patch.txt As design, hive could support hadoop dfs command in hive shell, like hive dfs -mkdir /user/biadmin/mydir; but has different behavior with hadoop if the path contains space and quotes hive dfs -mkdir hello; drwxr-xr-x - biadmin supergroup 0 2013-04-23 09:40 /user/biadmin/hello hive dfs -mkdir 'world'; drwxr-xr-x - biadmin supergroup 0 2013-04-23 09:43 /user/biadmin/'world' hive dfs -mkdir bei jing; drwxr-xr-x - biadmin supergroup 0 2013-04-23 09:44 /user/biadmin/bei drwxr-xr-x - biadmin supergroup 0 2013-04-23 09:44 /user/biadmin/jing -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6820) HiveServer(2) ignores HIVE_OPTS
[
https://issues.apache.org/jira/browse/HIVE-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bing Li updated HIVE-6820:
--
Fix Version/s: 0.14.0
HiveServer(2) ignores HIVE_OPTS
---
Key: HIVE-6820
URL: https://issues.apache.org/jira/browse/HIVE-6820
Project: Hive
Issue Type: Bug
Components: HiveServer2
Affects Versions: 0.12.0
Reporter: Richard Ding
Assignee: Bing Li
Priority: Minor
Fix For: 0.14.0
Attachments: HIVE-6820.1.patch
In hiveserver2.sh:
{code}
exec $HADOOP jar $JAR $CLASS $@
{code}
While cli.sh having:
{code}
exec $HADOOP jar ${HIVE_LIB}/hive-cli-*.jar $CLASS $HIVE_OPTS $@
{code}
Hence some hive commands that run properly in Hive shell fail in HiveServer.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-3685) TestCliDriver (script_pipe.q) failed with IBM JDK
[ https://issues.apache.org/jira/browse/HIVE-3685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bing Li updated HIVE-3685: -- Fix Version/s: 0.14.0 TestCliDriver (script_pipe.q) failed with IBM JDK - Key: HIVE-3685 URL: https://issues.apache.org/jira/browse/HIVE-3685 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.7.1, 0.8.0, 0.9.0, 0.11.0 Environment: ant-1.8.2 IBM JDK 1.6 Reporter: Bing Li Assignee: Bing Li Fix For: 0.14.0 Attachments: HIVE-3685.1.patch-trunk.txt, HIVE_3685.patch 1 failed: TestCliDriver (script_pipe.q) [junit] Begin query: script_pipe.q [junit] java.io.IOException: No such file or directory [junit] at java.io.FileOutputStream.writeBytes(Native Method) [junit] at java.io.FileOutputStream.write(FileOutputStream.java:293) [junit] at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:76) [junit] at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:134) [junit] at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:135) [junit] at java.io.DataOutputStream.flush(DataOutputStream.java:117) [junit] at org.apache.hadoop.hive.ql.exec.TextRecordWriter.close(TextRecordWriter.java:48) [junit] at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:365) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:303) [junit] at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:473) [junit] at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216) [junit] org.apache.hadoop.hive.ql.metadata.HiveException: Hit error while closing .. [junit] at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:452) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:303) [junit] at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:473) [junit] at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216) [junit] org.apache.hadoop.hive.ql.metadata.HiveException: Hit error while closing .. [junit] at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:452) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:303) [junit] at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:473) [junit] at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216) [junit] org.apache.hadoop.hive.ql.metadata.HiveException: Hit error while closing .. [junit] at org.apache.hadoop.hive.ql.exec.ScriptOperator.close(ScriptOperator.java:452) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:566) [junit] at org.apache.hadoop.hive.ql.exec.ExecReducer.close(ExecReducer.java:303) [junit] at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:473) [junit] at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411) [junit] at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216) [junit] Ended Job = job_local_0001 with errors [junit] Error during job, obtaining debugging information... [junit] Exception: Client Execution failed with error code = 9 [junit] See build/ql/tmp/hive.log, or try ant test ... -Dtest.silent=false to get more logs. [junit]
[jira] [Updated] (HIVE-3574) Allow Hive to Submit MapReduce jobs via the MapReduce API (instead of using Hadoop BIN)
[ https://issues.apache.org/jira/browse/HIVE-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bing Li updated HIVE-3574: -- Assignee: (was: Bing Li) Allow Hive to Submit MapReduce jobs via the MapReduce API (instead of using Hadoop BIN) --- Key: HIVE-3574 URL: https://issues.apache.org/jira/browse/HIVE-3574 Project: Hive Issue Type: Improvement Components: Query Processor, SQL Affects Versions: 0.3.0, 0.4.0, 0.4.1, 0.5.0, 0.6.0, 0.7.0, 0.7.1, 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.10.0 Environment: All environments would be affected by this Reporter: Jeremy A. Lucas Priority: Minor Labels: feature, test The current behavior of the MapRedTask is to start a process that invokes the hadoop jar command, passing each additional jobconf property as an argument to this Hadoop CLI. Having Hive to submit generated jobs to an M/R cluster via the MapReduce API would allow for potentially greater compatibility across platforms, in addition to allowing for these jobs to be run easily against pseudo-clusters in tests (think MiniMRCluster). This kind of change could involve something as simple as using a Hadoop Configuration object with a generic ToolRunner or something similar to run jobs. Specifically, this kind of change would most likely occur in the execute() method of org.apache.hadoop.hive.ql.exec.MapRedTask. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6957) SQL authorization does not work with HS2 binary mode and Kerberos auth
[
https://issues.apache.org/jira/browse/HIVE-6957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980939#comment-13980939
]
Hive QA commented on HIVE-6957:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12641843/HIVE-6957.4.patch
{color:red}ERROR:{color} -1 due to 40 failed/errored test(s), 5420 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_filter_numeric
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_test_outer
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_createas1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_truncate_column_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_19
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_20
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_21
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_22
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_24
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_9
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/35/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/35/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 40 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12641843
SQL authorization does not work with HS2 binary mode and Kerberos auth
--
Key: HIVE-6957
URL: https://issues.apache.org/jira/browse/HIVE-6957
Project: Hive
Issue Type: Bug
Components: Authorization, HiveServer2
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Attachments: HIVE-6957.1.patch, HIVE-6957.2.patch, HIVE-6957.3.patch,
HIVE-6957.4.patch
In HiveServer2, when Kerberos auth and binary transport modes are used, the
user name that gets passed on to authorization is the long kerberos username.
The username that is used in grant/revoke statements tend to be the short
usernames.
This also fails in authorizing statements that involve URI, as the
authorization mode checks the file system permissions for
[jira] [Created] (HIVE-6975) Adding partitions with a custom location ignores the locations
Calin-Andrei Burloiu created HIVE-6975: -- Summary: Adding partitions with a custom location ignores the locations Key: HIVE-6975 URL: https://issues.apache.org/jira/browse/HIVE-6975 Project: Hive Issue Type: Bug Affects Versions: 0.12.0 Reporter: Calin-Andrei Burloiu Context: an external table with AvroSerDe which is partitioned by (month STRING, day STRING). Problem: if I run `ALTER TABLE my_table ADD PARTITION (month='201401', day='03') LOCATION 'hdfs://nameservice1/user/my_user/data/my_table/avro/201401/03';` hive uses the location hdfs://nameservice1/user/my_user/data/my_table/avro/month=201401/day=03 -- This message was sent by Atlassian JIRA (v6.2#6252)
Review Request 20710: HIVE-6920 - Parquet Serde Simplification
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20710/ --- Review request for hive. Repository: hive-git Description --- Refactoring for simplification of the parquet-hive serde. Diffs - pom.xml 426dca8 ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ParquetHiveSerDe.java b689336 ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestParquetSerDe.java be518b9 ql/src/test/org/apache/hadoop/hive/ql/io/parquet/serde/TestParquetHiveSerDe.java PRE-CREATION Diff: https://reviews.apache.org/r/20710/diff/ Testing --- Thanks, justin coffey
[jira] [Commented] (HIVE-5823) Support for DECIMAL primitive type in AvroSerDe
[
https://issues.apache.org/jira/browse/HIVE-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981069#comment-13981069
]
Hive QA commented on HIVE-5823:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12641826/HIVE-5823.patch
{color:red}ERROR:{color} -1 due to 57 failed/errored test(s), 5489 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_decimal
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_evolved_schemas
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_joins
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_schema_literal
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_filter_numeric
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_test_outer
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_createas1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_truncate_column_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_19
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_20
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_21
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_22
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_24
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_9
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_bucket_map_join_tez1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_count
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_insert1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_limit_pushdown
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_load_dyn_part1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_mrr
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_dml
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union2
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union3
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union5
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union7
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_union9
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/36/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/36/console
Messages:
{noformat}
Executing
[jira] [Commented] (HIVE-4577) hive CLI can't handle hadoop dfs command with space and quotes.
[
https://issues.apache.org/jira/browse/HIVE-4577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981071#comment-13981071
]
Hive QA commented on HIVE-4577:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12596273/HIVE-4577.3.patch.txt
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/39/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/39/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n '' ]]
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-39/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 'serde/src/test/org/apache/hadoop/hive/serde2/avro/Utils.java'
Reverted
'serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroSerializer.java'
Reverted
'serde/src/test/org/apache/hadoop/hive/serde2/avro/TestGenericAvroRecordWritable.java'
Reverted 'serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerializer.java'
Reverted
'serde/src/java/org/apache/hadoop/hive/serde2/avro/SchemaToTypeInfo.java'
Reverted
'serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java'
Reverted 'serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java'
Reverted
'serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroGenericRecordWritable.java'
Reverted 'ql/src/test/queries/clientpositive/avro_schema_literal.q'
Reverted
'ql/src/java/org/apache/hadoop/hive/ql/io/avro/AvroGenericRecordReader.java'
++ awk '{print $2}'
++ egrep -v '^X|^Performing status on external'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target
shims/0.20S/target shims/0.23/target shims/aggregator/target
shims/common/target shims/common-secure/target packaging/target
hbase-handler/target testutils/target jdbc/target metastore/target
data/files/dec.txt itests/target itests/hcatalog-unit/target
itests/test-serde/target itests/qtest/target itests/hive-minikdc/target
itests/hive-unit/target itests/custom-serde/target itests/util/target
hcatalog/target hcatalog/core/target hcatalog/streaming/target
hcatalog/server-extensions/target hcatalog/hcatalog-pig-adapter/target
hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target hwi/target
common/target common/src/gen service/target contrib/target serde/target
beeline/target odbc/target cli/target ql/dependency-reduced-pom.xml ql/target
ql/src/test/results/clientpositive/avro_decimal.q.out
ql/src/test/queries/clientpositive/avro_decimal.q
+ svn update
Fetching external item into 'hcatalog/src/test/e2e/harness'
External at revision 1590044.
At revision 1590044.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12596273
hive CLI can't handle hadoop dfs command with space and quotes.
Key: HIVE-4577
URL: https://issues.apache.org/jira/browse/HIVE-4577
Project: Hive
Issue Type: Bug
Components: CLI
Affects Versions: 0.9.0, 0.10.0
Reporter: Bing Li
Assignee: Bing Li
Fix For: 0.14.0
Attachments: HIVE-4577.1.patch, HIVE-4577.2.patch,
HIVE-4577.3.patch.txt
As design, hive could support hadoop dfs command in hive shell, like
hive dfs -mkdir /user/biadmin/mydir;
but has different behavior with hadoop if the path contains space and quotes
hive dfs -mkdir hello;
drwxr-xr-x - biadmin supergroup 0 2013-04-23 09:40
/user/biadmin/hello
hive dfs -mkdir 'world';
drwxr-xr-x - biadmin supergroup 0 2013-04-23 09:43
/user/biadmin/'world'
hive dfs -mkdir bei jing;
drwxr-xr-x - biadmin supergroup
[jira] [Commented] (HIVE-6920) Parquet Serde Simplification
[ https://issues.apache.org/jira/browse/HIVE-6920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981072#comment-13981072 ] Justin Coffey commented on HIVE-6920: - It's actually mostly just code reduction. Here's the RB link: https://reviews.apache.org/r/20710/ thanks :) Parquet Serde Simplification Key: HIVE-6920 URL: https://issues.apache.org/jira/browse/HIVE-6920 Project: Hive Issue Type: Improvement Components: Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Justin Coffey Assignee: Justin Coffey Priority: Minor Fix For: 0.14.0 Attachments: HIVE-6920.patch Various fixes and code simplification in the ParquetHiveSerde (with minor optimizations) -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [ANNOUNCE] New Hive Committers - Prasanth J and Vaibhav Gumashta
Congrats, guys! :) On Fri, Apr 25, 2014 at 12:33 AM, Lefty Leverenz leftylever...@gmail.com wrote: Congratulations! -- Lefty On Fri, Apr 25, 2014 at 12:10 AM, Hari Subramaniyan hsubramani...@hortonworks.com wrote: Congrats Prasanth and Vaibhav! Thanks Hari On Thu, Apr 24, 2014 at 8:45 PM, Chinna Rao Lalam lalamchinnara...@gmail.com wrote: Congratulations to Prasanth and Vaibhav! On Fri, Apr 25, 2014 at 8:23 AM, Shengjun Xin s...@gopivotal.com wrote: Congratulations ~~ On Fri, Apr 25, 2014 at 10:33 AM, Carl Steinbach cwsteinb...@gmail.com wrote: + Prasanth's correct email address On Thu, Apr 24, 2014 at 7:31 PM, Xuefu Zhang xzh...@cloudera.com wrote: Congratulations to Prasanth and Vaibhav! --Xuefu On Thu, Apr 24, 2014 at 7:26 PM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Prasanth J and Vaibhav Gumashta committers on the Apache Hive Project. Please join me in congratulating Prasanth and Vaibhav! Thanks. - Carl -- Regards Shengjun -- Hope It Helps, Chinna -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Apache Hive 0.13.1
Hi Folks, Given the quickly increasing scope (from a perspective of sheer number of jiras) of hive 0.13, it was important to get hive 0.13 out of the door, and stop accepting patches, and move new development off to 0.14, but we should begin discussion of a 0.13.1 release with major bug fixes only (no feature additions, nothing like refactoring) as a stabilization of 0.13. There are some jiras here from talking to a couple of people that I think should definitely be part of such a release : Sql-std auth related: HIVE-6919 - hive sql std auth select query fails on partitioned tables HIVE-6921 - index creation fails with sql std auth turned on HIVE-6957 - SQL auth does not work with HS2 binary mode and Kerberos authentication Metastore HIVE-6945 - issues with dropping partitions with oracle as backing db HIVE-6862 - MsSQL upgrade scripts Other HIVE-6883 - Dynamic partitioning does not honour sort order or order-by HIVE-4576 - WebHCat does not allow values with commas I'd be willing to throw my hat in to help curate/manage such a release. Any thoughts/comments/additional jiras to add to this lot? Thanks, -Sushanth
Re: Apache Hive 0.13.1
I would like to request: HIVE-6952 : Hive 0.13 HiveOutputFormat breaks backwards compatibility On Fri, Apr 25, 2014 at 9:09 AM, Sushanth Sowmyan khorg...@gmail.comwrote: Hi Folks, Given the quickly increasing scope (from a perspective of sheer number of jiras) of hive 0.13, it was important to get hive 0.13 out of the door, and stop accepting patches, and move new development off to 0.14, but we should begin discussion of a 0.13.1 release with major bug fixes only (no feature additions, nothing like refactoring) as a stabilization of 0.13. There are some jiras here from talking to a couple of people that I think should definitely be part of such a release : Sql-std auth related: HIVE-6919 - hive sql std auth select query fails on partitioned tables HIVE-6921 - index creation fails with sql std auth turned on HIVE-6957 - SQL auth does not work with HS2 binary mode and Kerberos authentication Metastore HIVE-6945 - issues with dropping partitions with oracle as backing db HIVE-6862 - MsSQL upgrade scripts Other HIVE-6883 - Dynamic partitioning does not honour sort order or order-by HIVE-4576 - WebHCat does not allow values with commas I'd be willing to throw my hat in to help curate/manage such a release. Any thoughts/comments/additional jiras to add to this lot? Thanks, -Sushanth
[jira] [Commented] (HIVE-6835) Reading of partitioned Avro data fails if partition schema does not match table schema
[
https://issues.apache.org/jira/browse/HIVE-6835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981229#comment-13981229
]
Xuefu Zhang commented on HIVE-6835:
---
[~erwaman] If you can confirm that these test failures are unrelated to your
patch, I can commit it in a few hours.
Reading of partitioned Avro data fails if partition schema does not match
table schema
--
Key: HIVE-6835
URL: https://issues.apache.org/jira/browse/HIVE-6835
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0
Reporter: Anthony Hsu
Assignee: Anthony Hsu
Attachments: HIVE-6835.1.patch, HIVE-6835.2.patch, HIVE-6835.3.patch,
HIVE-6835.4.patch, HIVE-6835.5.patch
To reproduce:
{code}
create table testarray (a arraystring);
load data local inpath '/home/ahsu/test/array.txt' into table testarray;
# create partitioned Avro table with one array column
create table avroarray partitioned by (y string) row format serde
'org.apache.hadoop.hive.serde2.avro.AvroSerDe' with serdeproperties
('avro.schema.literal'='{namespace:test,name:avroarray,type:
record, fields: [ { name:a, type:{type:array,items:string}
} ] }') STORED as INPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat';
insert into table avroarray partition(y=1) select * from testarray;
# add an int column with a default value of 0
alter table avroarray set serde
'org.apache.hadoop.hive.serde2.avro.AvroSerDe' with
serdeproperties('avro.schema.literal'='{namespace:test,name:avroarray,type:
record, fields: [ {name:intfield,type:int,default:0},{
name:a, type:{type:array,items:string} } ] }');
# fails with ClassCastException
select * from avroarray;
{code}
The select * fails with:
{code}
Failed with exception java.io.IOException:java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
cannot be cast to
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: [ANNOUNCE] New Hive Committers - Prasanth J and Vaibhav Gumashta
Its a wonderful news. :) Thanks for your wishes guys!! Thanks Prasanth On Apr 25, 2014, at 8:52 AM, Sushanth Sowmyan khorg...@gmail.com wrote: Congrats, guys! :) On Fri, Apr 25, 2014 at 12:33 AM, Lefty Leverenz leftylever...@gmail.com wrote: Congratulations! -- Lefty On Fri, Apr 25, 2014 at 12:10 AM, Hari Subramaniyan hsubramani...@hortonworks.com wrote: Congrats Prasanth and Vaibhav! Thanks Hari On Thu, Apr 24, 2014 at 8:45 PM, Chinna Rao Lalam lalamchinnara...@gmail.com wrote: Congratulations to Prasanth and Vaibhav! On Fri, Apr 25, 2014 at 8:23 AM, Shengjun Xin s...@gopivotal.com wrote: Congratulations ~~ On Fri, Apr 25, 2014 at 10:33 AM, Carl Steinbach cwsteinb...@gmail.com wrote: + Prasanth's correct email address On Thu, Apr 24, 2014 at 7:31 PM, Xuefu Zhang xzh...@cloudera.com wrote: Congratulations to Prasanth and Vaibhav! --Xuefu On Thu, Apr 24, 2014 at 7:26 PM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Prasanth J and Vaibhav Gumashta committers on the Apache Hive Project. Please join me in congratulating Prasanth and Vaibhav! Thanks. - Carl -- Regards Shengjun -- Hope It Helps, Chinna -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Apache Hive 0.13.1
Sushanth, Thanks for bringing this up and volunteering to manage the release! Yes, we need a 0.13.1 to fix some of these critical issues, specially the authorization ones, and the oracle issue you mentioned. I think it will be useful to have a wiki page for 0.13.1 status tracking (mentioning patches to be included, the jira state, and status of inclusion into 0.13 branch.) I think we should try to release it without too much delay. Do you have any target date in mind for getting the 0.13.1 RC out ? People work better with deadlines! Thanks, Thejas On Fri, Apr 25, 2014 at 9:33 AM, Ashutosh Chauhan hashut...@apache.orgwrote: I would like to request: HIVE-6952 : Hive 0.13 HiveOutputFormat breaks backwards compatibility On Fri, Apr 25, 2014 at 9:09 AM, Sushanth Sowmyan khorg...@gmail.com wrote: Hi Folks, Given the quickly increasing scope (from a perspective of sheer number of jiras) of hive 0.13, it was important to get hive 0.13 out of the door, and stop accepting patches, and move new development off to 0.14, but we should begin discussion of a 0.13.1 release with major bug fixes only (no feature additions, nothing like refactoring) as a stabilization of 0.13. There are some jiras here from talking to a couple of people that I think should definitely be part of such a release : Sql-std auth related: HIVE-6919 - hive sql std auth select query fails on partitioned tables HIVE-6921 - index creation fails with sql std auth turned on HIVE-6957 - SQL auth does not work with HS2 binary mode and Kerberos authentication Metastore HIVE-6945 - issues with dropping partitions with oracle as backing db HIVE-6862 - MsSQL upgrade scripts Other HIVE-6883 - Dynamic partitioning does not honour sort order or order-by HIVE-4576 - WebHCat does not allow values with commas I'd be willing to throw my hat in to help curate/manage such a release. Any thoughts/comments/additional jiras to add to this lot? Thanks, -Sushanth -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [ANNOUNCE] New Hive Committers - Prasanth J and Vaibhav Gumashta
Congrats Prasanth and Vaibhav! On Fri, Apr 25, 2014 at 10:16 AM, Prasanth J j.prasant...@gmail.com wrote: Its a wonderful news. :) Thanks for your wishes guys!! Thanks Prasanth On Apr 25, 2014, at 8:52 AM, Sushanth Sowmyan khorg...@gmail.com wrote: Congrats, guys! :) On Fri, Apr 25, 2014 at 12:33 AM, Lefty Leverenz leftylever...@gmail.com wrote: Congratulations! -- Lefty On Fri, Apr 25, 2014 at 12:10 AM, Hari Subramaniyan hsubramani...@hortonworks.com wrote: Congrats Prasanth and Vaibhav! Thanks Hari On Thu, Apr 24, 2014 at 8:45 PM, Chinna Rao Lalam lalamchinnara...@gmail.com wrote: Congratulations to Prasanth and Vaibhav! On Fri, Apr 25, 2014 at 8:23 AM, Shengjun Xin s...@gopivotal.com wrote: Congratulations ~~ On Fri, Apr 25, 2014 at 10:33 AM, Carl Steinbach cwsteinb...@gmail.com wrote: + Prasanth's correct email address On Thu, Apr 24, 2014 at 7:31 PM, Xuefu Zhang xzh...@cloudera.com wrote: Congratulations to Prasanth and Vaibhav! --Xuefu On Thu, Apr 24, 2014 at 7:26 PM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Prasanth J and Vaibhav Gumashta committers on the Apache Hive Project. Please join me in congratulating Prasanth and Vaibhav! Thanks. - Carl -- Regards Shengjun -- Hope It Helps, Chinna -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-6835) Reading of partitioned Avro data fails if partition schema does not match table schema
[
https://issues.apache.org/jira/browse/HIVE-6835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981278#comment-13981278
]
Anthony Hsu commented on HIVE-6835:
---
I will do some local testing soon and let you know.
Reading of partitioned Avro data fails if partition schema does not match
table schema
--
Key: HIVE-6835
URL: https://issues.apache.org/jira/browse/HIVE-6835
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0
Reporter: Anthony Hsu
Assignee: Anthony Hsu
Attachments: HIVE-6835.1.patch, HIVE-6835.2.patch, HIVE-6835.3.patch,
HIVE-6835.4.patch, HIVE-6835.5.patch
To reproduce:
{code}
create table testarray (a arraystring);
load data local inpath '/home/ahsu/test/array.txt' into table testarray;
# create partitioned Avro table with one array column
create table avroarray partitioned by (y string) row format serde
'org.apache.hadoop.hive.serde2.avro.AvroSerDe' with serdeproperties
('avro.schema.literal'='{namespace:test,name:avroarray,type:
record, fields: [ { name:a, type:{type:array,items:string}
} ] }') STORED as INPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat';
insert into table avroarray partition(y=1) select * from testarray;
# add an int column with a default value of 0
alter table avroarray set serde
'org.apache.hadoop.hive.serde2.avro.AvroSerDe' with
serdeproperties('avro.schema.literal'='{namespace:test,name:avroarray,type:
record, fields: [ {name:intfield,type:int,default:0},{
name:a, type:{type:array,items:string} } ] }');
# fails with ClassCastException
select * from avroarray;
{code}
The select * fails with:
{code}
Failed with exception java.io.IOException:java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
cannot be cast to
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-6976) Show query id only when there's jobs on the cluster
Gunther Hagleitner created HIVE-6976: Summary: Show query id only when there's jobs on the cluster Key: HIVE-6976 URL: https://issues.apache.org/jira/browse/HIVE-6976 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Priority: Minor No need to print the query id for local-only execution. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6957) SQL authorization does not work with HS2 binary mode and Kerberos auth
[ https://issues.apache.org/jira/browse/HIVE-6957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-6957: Attachment: HIVE-6957.04-branch.0.13.patch HIVE-6957.04-branch.0.13.patch - patch for 0.13 branch . SQL authorization does not work with HS2 binary mode and Kerberos auth -- Key: HIVE-6957 URL: https://issues.apache.org/jira/browse/HIVE-6957 Project: Hive Issue Type: Bug Components: Authorization, HiveServer2 Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-6957.04-branch.0.13.patch, HIVE-6957.1.patch, HIVE-6957.2.patch, HIVE-6957.3.patch, HIVE-6957.4.patch In HiveServer2, when Kerberos auth and binary transport modes are used, the user name that gets passed on to authorization is the long kerberos username. The username that is used in grant/revoke statements tend to be the short usernames. This also fails in authorizing statements that involve URI, as the authorization mode checks the file system permissions for given user. It does not recognize that the given long username actually owns the file or belongs to the group that owns the file. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6976) Show query id only when there's jobs on the cluster
[ https://issues.apache.org/jira/browse/HIVE-6976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-6976: - Attachment: (was: HIVE-6976.1.patch) Show query id only when there's jobs on the cluster --- Key: HIVE-6976 URL: https://issues.apache.org/jira/browse/HIVE-6976 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Priority: Minor Attachments: HIVE-6976.1.patch No need to print the query id for local-only execution. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6976) Show query id only when there's jobs on the cluster
[ https://issues.apache.org/jira/browse/HIVE-6976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-6976: - Attachment: HIVE-6976.1.patch Show query id only when there's jobs on the cluster --- Key: HIVE-6976 URL: https://issues.apache.org/jira/browse/HIVE-6976 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Priority: Minor Attachments: HIVE-6976.1.patch No need to print the query id for local-only execution. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [ANNOUNCE] New Hive Committers - Prasanth J and Vaibhav Gumashta
Congratulations !! thanks Prasad On Thu, Apr 24, 2014 at 7:33 PM, Carl Steinbach cwsteinb...@gmail.comwrote: + Prasanth's correct email address On Thu, Apr 24, 2014 at 7:31 PM, Xuefu Zhang xzh...@cloudera.com wrote: Congratulations to Prasanth and Vaibhav! --Xuefu On Thu, Apr 24, 2014 at 7:26 PM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Prasanth J and Vaibhav Gumashta committers on the Apache Hive Project. Please join me in congratulating Prasanth and Vaibhav! Thanks. - Carl
Re: [ANNOUNCE] New Hive Committers - Prasanth J and Vaibhav Gumashta
Congratulations! Selina On 4/25/14, 10:35 AM, Prasad Mujumdar pras...@cloudera.com wrote: Congratulations !! thanks Prasad On Thu, Apr 24, 2014 at 7:33 PM, Carl Steinbach cwsteinb...@gmail.comwrote: + Prasanth's correct email address On Thu, Apr 24, 2014 at 7:31 PM, Xuefu Zhang xzh...@cloudera.com wrote: Congratulations to Prasanth and Vaibhav! --Xuefu On Thu, Apr 24, 2014 at 7:26 PM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Prasanth J and Vaibhav Gumashta committers on the Apache Hive Project. Please join me in congratulating Prasanth and Vaibhav! Thanks. - Carl
[jira] [Updated] (HIVE-6976) Show query id only when there's jobs on the cluster
[ https://issues.apache.org/jira/browse/HIVE-6976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-6976: - Attachment: HIVE-6976.1.patch Show query id only when there's jobs on the cluster --- Key: HIVE-6976 URL: https://issues.apache.org/jira/browse/HIVE-6976 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Priority: Minor Attachments: HIVE-6976.1.patch No need to print the query id for local-only execution. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6901) Explain plan doesn't show operator tree for the fetch operator
[
https://issues.apache.org/jira/browse/HIVE-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-6901:
--
Attachment: HIVE-6901.2.patch
Explain plan doesn't show operator tree for the fetch operator
--
Key: HIVE-6901
URL: https://issues.apache.org/jira/browse/HIVE-6901
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.12.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Priority: Minor
Attachments: HIVE-6901.1.patch, HIVE-6901.2.patch, HIVE-6901.2.patch,
HIVE-6901.2.patch, HIVE-6901.2.patch, HIVE-6901.2.patch, HIVE-6901.patch
Explaining a simple select query that involves a MR phase doesn't show
processor tree for the fetch operator.
{code}
hive explain select d from test;
OK
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
...
Stage: Stage-0
Fetch Operator
limit: -1
{code}
It would be nice if the operator tree is shown even if there is only one node.
Please note that in local execution, the operator tree is complete:
{code}
hive explain select * from test;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: test
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE Column
stats: NONE
Select Operator
expressions: d (type: int)
outputColumnNames: _col0
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE
Column stats: NONE
ListSink
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-5823) Support for DECIMAL primitive type in AvroSerDe
[ https://issues.apache.org/jira/browse/HIVE-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5823: -- Attachment: HIVE-5823.1.patch Support for DECIMAL primitive type in AvroSerDe --- Key: HIVE-5823 URL: https://issues.apache.org/jira/browse/HIVE-5823 Project: Hive Issue Type: New Feature Components: Serializers/Deserializers Affects Versions: 0.12.0 Reporter: Mariano Dominguez Assignee: Xuefu Zhang Labels: avro, serde Attachments: HIVE-5823.1.patch, HIVE-5823.patch This new feature request would be tied to AVRO-1402. Adding DECIMAL support would be particularly interesting when converting types from Avro to Hive, since DECIMAL is already a supported data type in Hive. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: [ANNOUNCE] New Hive Committers - Prasanth J and Vaibhav Gumashta
Congrats! Nice job guys! On Apr 25, 2014, at 10:38 AM, Selina Zhang seli...@yahoo-inc.com wrote: Congratulations! Selina On 4/25/14, 10:35 AM, Prasad Mujumdar pras...@cloudera.com wrote: Congratulations !! thanks Prasad On Thu, Apr 24, 2014 at 7:33 PM, Carl Steinbach cwsteinb...@gmail.comwrote: + Prasanth's correct email address On Thu, Apr 24, 2014 at 7:31 PM, Xuefu Zhang xzh...@cloudera.com wrote: Congratulations to Prasanth and Vaibhav! --Xuefu On Thu, Apr 24, 2014 at 7:26 PM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Prasanth J and Vaibhav Gumashta committers on the Apache Hive Project. Please join me in congratulating Prasanth and Vaibhav! Thanks. - Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-6901) Explain plan doesn't show operator tree for the fetch operator
[
https://issues.apache.org/jira/browse/HIVE-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981402#comment-13981402
]
Hive QA commented on HIVE-6901:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12641974/HIVE-6901.2.patch
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/42/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/42/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n '' ]]
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-42/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
++ egrep -v '^X|^Performing status on external'
++ awk '{print $2}'
++ svn status --no-ignore
+ rm -rf
+ svn update
Fetching external item into 'hcatalog/src/test/e2e/harness'
External at revision 1590092.
At revision 1590092.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12641974
Explain plan doesn't show operator tree for the fetch operator
--
Key: HIVE-6901
URL: https://issues.apache.org/jira/browse/HIVE-6901
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.12.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Priority: Minor
Attachments: HIVE-6901.1.patch, HIVE-6901.2.patch, HIVE-6901.2.patch,
HIVE-6901.2.patch, HIVE-6901.2.patch, HIVE-6901.2.patch, HIVE-6901.patch
Explaining a simple select query that involves a MR phase doesn't show
processor tree for the fetch operator.
{code}
hive explain select d from test;
OK
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
...
Stage: Stage-0
Fetch Operator
limit: -1
{code}
It would be nice if the operator tree is shown even if there is only one node.
Please note that in local execution, the operator tree is complete:
{code}
hive explain select * from test;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: test
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE Column
stats: NONE
Select Operator
expressions: d (type: int)
outputColumnNames: _col0
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE
Column stats: NONE
ListSink
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: Apache Hive 0.13.1
I think it will be useful to have a wiki page for 0.13.1 status tracking (mentioning patches to be included, the jira state, and status of inclusion into 0.13 branch.) Yup, makes sense, will create a wiki page for the same. I think we should try to release it without too much delay. Do you have any target date in mind for getting the 0.13.1 RC out ? People work better with deadlines! Agreed. How about tonight 6pm as the deadline for RC1 ? Okay, okay, I kid. :D I think it's important to get a bugfix/stabilization release reasonably quickly, but it's also important to give people a little time to try out 0.13, discover/report bugs and fix them. So I think about two weeks is a good point? And instead of releasing an RC on a friday, I'm thinking of pushing it out to Monday - does 12th May sound good to everyone?
Re: Apache Hive 0.13.1
HIVE-6952 definitely seems an important candidate for 0.13.1, I'll add it in. Thanks for bringing it up, Ashutosh. On Fri, Apr 25, 2014 at 9:33 AM, Ashutosh Chauhan hashut...@apache.org wrote: I would like to request: HIVE-6952 : Hive 0.13 HiveOutputFormat breaks backwards compatibility On Fri, Apr 25, 2014 at 9:09 AM, Sushanth Sowmyan khorg...@gmail.comwrote: Hi Folks, Given the quickly increasing scope (from a perspective of sheer number of jiras) of hive 0.13, it was important to get hive 0.13 out of the door, and stop accepting patches, and move new development off to 0.14, but we should begin discussion of a 0.13.1 release with major bug fixes only (no feature additions, nothing like refactoring) as a stabilization of 0.13. There are some jiras here from talking to a couple of people that I think should definitely be part of such a release : Sql-std auth related: HIVE-6919 - hive sql std auth select query fails on partitioned tables HIVE-6921 - index creation fails with sql std auth turned on HIVE-6957 - SQL auth does not work with HS2 binary mode and Kerberos authentication Metastore HIVE-6945 - issues with dropping partitions with oracle as backing db HIVE-6862 - MsSQL upgrade scripts Other HIVE-6883 - Dynamic partitioning does not honour sort order or order-by HIVE-4576 - WebHCat does not allow values with commas I'd be willing to throw my hat in to help curate/manage such a release. Any thoughts/comments/additional jiras to add to this lot? Thanks, -Sushanth
[jira] [Updated] (HIVE-6901) Explain plan doesn't show operator tree for the fetch operator
[
https://issues.apache.org/jira/browse/HIVE-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-6901:
--
Attachment: (was: HIVE-6901.2.patch)
Explain plan doesn't show operator tree for the fetch operator
--
Key: HIVE-6901
URL: https://issues.apache.org/jira/browse/HIVE-6901
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.12.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Priority: Minor
Attachments: HIVE-6901.1.patch, HIVE-6901.2.patch, HIVE-6901.3.patch,
HIVE-6901.patch
Explaining a simple select query that involves a MR phase doesn't show
processor tree for the fetch operator.
{code}
hive explain select d from test;
OK
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
...
Stage: Stage-0
Fetch Operator
limit: -1
{code}
It would be nice if the operator tree is shown even if there is only one node.
Please note that in local execution, the operator tree is complete:
{code}
hive explain select * from test;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: test
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE Column
stats: NONE
Select Operator
expressions: d (type: int)
outputColumnNames: _col0
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE
Column stats: NONE
ListSink
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6901) Explain plan doesn't show operator tree for the fetch operator
[
https://issues.apache.org/jira/browse/HIVE-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-6901:
--
Attachment: (was: HIVE-6901.2.patch)
Explain plan doesn't show operator tree for the fetch operator
--
Key: HIVE-6901
URL: https://issues.apache.org/jira/browse/HIVE-6901
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.12.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Priority: Minor
Attachments: HIVE-6901.1.patch, HIVE-6901.2.patch, HIVE-6901.3.patch,
HIVE-6901.patch
Explaining a simple select query that involves a MR phase doesn't show
processor tree for the fetch operator.
{code}
hive explain select d from test;
OK
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
...
Stage: Stage-0
Fetch Operator
limit: -1
{code}
It would be nice if the operator tree is shown even if there is only one node.
Please note that in local execution, the operator tree is complete:
{code}
hive explain select * from test;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: test
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE Column
stats: NONE
Select Operator
expressions: d (type: int)
outputColumnNames: _col0
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE
Column stats: NONE
ListSink
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6901) Explain plan doesn't show operator tree for the fetch operator
[
https://issues.apache.org/jira/browse/HIVE-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-6901:
--
Attachment: (was: HIVE-6901.2.patch)
Explain plan doesn't show operator tree for the fetch operator
--
Key: HIVE-6901
URL: https://issues.apache.org/jira/browse/HIVE-6901
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.12.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Priority: Minor
Attachments: HIVE-6901.1.patch, HIVE-6901.2.patch, HIVE-6901.3.patch,
HIVE-6901.patch
Explaining a simple select query that involves a MR phase doesn't show
processor tree for the fetch operator.
{code}
hive explain select d from test;
OK
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
...
Stage: Stage-0
Fetch Operator
limit: -1
{code}
It would be nice if the operator tree is shown even if there is only one node.
Please note that in local execution, the operator tree is complete:
{code}
hive explain select * from test;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: test
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE Column
stats: NONE
Select Operator
expressions: d (type: int)
outputColumnNames: _col0
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE
Column stats: NONE
ListSink
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6901) Explain plan doesn't show operator tree for the fetch operator
[
https://issues.apache.org/jira/browse/HIVE-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-6901:
--
Attachment: (was: HIVE-6901.2.patch)
Explain plan doesn't show operator tree for the fetch operator
--
Key: HIVE-6901
URL: https://issues.apache.org/jira/browse/HIVE-6901
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.12.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Priority: Minor
Attachments: HIVE-6901.1.patch, HIVE-6901.2.patch, HIVE-6901.3.patch,
HIVE-6901.patch
Explaining a simple select query that involves a MR phase doesn't show
processor tree for the fetch operator.
{code}
hive explain select d from test;
OK
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
...
Stage: Stage-0
Fetch Operator
limit: -1
{code}
It would be nice if the operator tree is shown even if there is only one node.
Please note that in local execution, the operator tree is complete:
{code}
hive explain select * from test;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: test
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE Column
stats: NONE
Select Operator
expressions: d (type: int)
outputColumnNames: _col0
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE
Column stats: NONE
ListSink
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6835) Reading of partitioned Avro data fails if partition schema does not match table schema
[
https://issues.apache.org/jira/browse/HIVE-6835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981425#comment-13981425
]
Anthony Hsu commented on HIVE-6835:
---
I tried all the failed union_remove TestCliDriver tests locally and they all
passed. Looking at some of the previous precommit builds, several of them also
have the same test failures, so I believe these test failures are unrelated to
my changes.
Reading of partitioned Avro data fails if partition schema does not match
table schema
--
Key: HIVE-6835
URL: https://issues.apache.org/jira/browse/HIVE-6835
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0
Reporter: Anthony Hsu
Assignee: Anthony Hsu
Attachments: HIVE-6835.1.patch, HIVE-6835.2.patch, HIVE-6835.3.patch,
HIVE-6835.4.patch, HIVE-6835.5.patch
To reproduce:
{code}
create table testarray (a arraystring);
load data local inpath '/home/ahsu/test/array.txt' into table testarray;
# create partitioned Avro table with one array column
create table avroarray partitioned by (y string) row format serde
'org.apache.hadoop.hive.serde2.avro.AvroSerDe' with serdeproperties
('avro.schema.literal'='{namespace:test,name:avroarray,type:
record, fields: [ { name:a, type:{type:array,items:string}
} ] }') STORED as INPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat';
insert into table avroarray partition(y=1) select * from testarray;
# add an int column with a default value of 0
alter table avroarray set serde
'org.apache.hadoop.hive.serde2.avro.AvroSerDe' with
serdeproperties('avro.schema.literal'='{namespace:test,name:avroarray,type:
record, fields: [ {name:intfield,type:int,default:0},{
name:a, type:{type:array,items:string} } ] }');
# fails with ClassCastException
select * from avroarray;
{code}
The select * fails with:
{code}
Failed with exception java.io.IOException:java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
cannot be cast to
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: [ANNOUNCE] New Hive Committers - Prasanth J and Vaibhav Gumashta
This is cool! Thanks a lot guys! --Vaibhav On Fri, Apr 25, 2014 at 10:19 AM, Thejas Nair the...@hortonworks.comwrote: Congrats Prasanth and Vaibhav! On Fri, Apr 25, 2014 at 10:16 AM, Prasanth J j.prasant...@gmail.comwrote: Its a wonderful news. :) Thanks for your wishes guys!! Thanks Prasanth On Apr 25, 2014, at 8:52 AM, Sushanth Sowmyan khorg...@gmail.com wrote: Congrats, guys! :) On Fri, Apr 25, 2014 at 12:33 AM, Lefty Leverenz leftylever...@gmail.com wrote: Congratulations! -- Lefty On Fri, Apr 25, 2014 at 12:10 AM, Hari Subramaniyan hsubramani...@hortonworks.com wrote: Congrats Prasanth and Vaibhav! Thanks Hari On Thu, Apr 24, 2014 at 8:45 PM, Chinna Rao Lalam lalamchinnara...@gmail.com wrote: Congratulations to Prasanth and Vaibhav! On Fri, Apr 25, 2014 at 8:23 AM, Shengjun Xin s...@gopivotal.com wrote: Congratulations ~~ On Fri, Apr 25, 2014 at 10:33 AM, Carl Steinbach cwsteinb...@gmail.com wrote: + Prasanth's correct email address On Thu, Apr 24, 2014 at 7:31 PM, Xuefu Zhang xzh...@cloudera.com wrote: Congratulations to Prasanth and Vaibhav! --Xuefu On Thu, Apr 24, 2014 at 7:26 PM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Prasanth J and Vaibhav Gumashta committers on the Apache Hive Project. Please join me in congratulating Prasanth and Vaibhav! Thanks. - Carl -- Regards Shengjun -- Hope It Helps, Chinna -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Updated] (HIVE-6901) Explain plan doesn't show operator tree for the fetch operator
[
https://issues.apache.org/jira/browse/HIVE-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-6901:
--
Attachment: HIVE-6901.3.patch
Explain plan doesn't show operator tree for the fetch operator
--
Key: HIVE-6901
URL: https://issues.apache.org/jira/browse/HIVE-6901
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.12.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Priority: Minor
Attachments: HIVE-6901.1.patch, HIVE-6901.2.patch, HIVE-6901.2.patch,
HIVE-6901.2.patch, HIVE-6901.2.patch, HIVE-6901.2.patch, HIVE-6901.3.patch,
HIVE-6901.patch
Explaining a simple select query that involves a MR phase doesn't show
processor tree for the fetch operator.
{code}
hive explain select d from test;
OK
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
...
Stage: Stage-0
Fetch Operator
limit: -1
{code}
It would be nice if the operator tree is shown even if there is only one node.
Please note that in local execution, the operator tree is complete:
{code}
hive explain select * from test;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: test
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE Column
stats: NONE
Select Operator
expressions: d (type: int)
outputColumnNames: _col0
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE
Column stats: NONE
ListSink
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6835) Reading of partitioned Avro data fails if partition schema does not match table schema
[
https://issues.apache.org/jira/browse/HIVE-6835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981431#comment-13981431
]
Anthony Hsu commented on HIVE-6835:
---
BTW, I have been doing all my development and testing against Hadoop 1.2.1
(-Phadoop-1).
Reading of partitioned Avro data fails if partition schema does not match
table schema
--
Key: HIVE-6835
URL: https://issues.apache.org/jira/browse/HIVE-6835
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0
Reporter: Anthony Hsu
Assignee: Anthony Hsu
Attachments: HIVE-6835.1.patch, HIVE-6835.2.patch, HIVE-6835.3.patch,
HIVE-6835.4.patch, HIVE-6835.5.patch
To reproduce:
{code}
create table testarray (a arraystring);
load data local inpath '/home/ahsu/test/array.txt' into table testarray;
# create partitioned Avro table with one array column
create table avroarray partitioned by (y string) row format serde
'org.apache.hadoop.hive.serde2.avro.AvroSerDe' with serdeproperties
('avro.schema.literal'='{namespace:test,name:avroarray,type:
record, fields: [ { name:a, type:{type:array,items:string}
} ] }') STORED as INPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat';
insert into table avroarray partition(y=1) select * from testarray;
# add an int column with a default value of 0
alter table avroarray set serde
'org.apache.hadoop.hive.serde2.avro.AvroSerDe' with
serdeproperties('avro.schema.literal'='{namespace:test,name:avroarray,type:
record, fields: [ {name:intfield,type:int,default:0},{
name:a, type:{type:array,items:string} } ] }');
# fails with ClassCastException
select * from avroarray;
{code}
The select * fails with:
{code}
Failed with exception java.io.IOException:java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
cannot be cast to
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: Apache Hive 0.13.1
On Fri, Apr 25, 2014 at 11:33 AM, Sushanth Sowmyan khorg...@gmail.comwrote: I think it's important to get a bugfix/stabilization release reasonably quickly, but it's also important to give people a little time to try out 0.13, discover/report bugs and fix them. So I think about two weeks is a good point? And instead of releasing an RC on a friday, I'm thinking of pushing it out to Monday - does 12th May sound good to everyone? I think we can aim for an earlier date. Most of these issues seem to be already committed to trunk or have patches available. So the remaining ones also might get committed to trunk by early next week. How about shooting for May 5th (Monday) ? By then 0.13 would also have been out for 2 weeks. If we have any new critical bug reported that needs a fix, we can hold off on the RC for few days. What do you think ? Thanks, Thejas -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-6957) SQL authorization does not work with HS2 binary mode and Kerberos auth
[ https://issues.apache.org/jira/browse/HIVE-6957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981433#comment-13981433 ] Vaibhav Gumashta commented on HIVE-6957: +1 SQL authorization does not work with HS2 binary mode and Kerberos auth -- Key: HIVE-6957 URL: https://issues.apache.org/jira/browse/HIVE-6957 Project: Hive Issue Type: Bug Components: Authorization, HiveServer2 Affects Versions: 0.13.0 Reporter: Thejas M Nair Assignee: Thejas M Nair Attachments: HIVE-6957.04-branch.0.13.patch, HIVE-6957.1.patch, HIVE-6957.2.patch, HIVE-6957.3.patch, HIVE-6957.4.patch In HiveServer2, when Kerberos auth and binary transport modes are used, the user name that gets passed on to authorization is the long kerberos username. The username that is used in grant/revoke statements tend to be the short usernames. This also fails in authorizing statements that involve URI, as the authorization mode checks the file system permissions for given user. It does not recognize that the given long username actually owns the file or belongs to the group that owns the file. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6955) ExprNodeColDesc isSame doesn't account for tabAlias: this affects trait Propagation in Joins
[
https://issues.apache.org/jira/browse/HIVE-6955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981440#comment-13981440
]
Gunther Hagleitner commented on HIVE-6955:
--
+1
There seem to be a bunch of unit test failures [~rhbutani] - are they related?
ExprNodeColDesc isSame doesn't account for tabAlias: this affects trait
Propagation in Joins
Key: HIVE-6955
URL: https://issues.apache.org/jira/browse/HIVE-6955
Project: Hive
Issue Type: Bug
Reporter: Harish Butani
Assignee: Harish Butani
Attachments: HIVE-6955.1.patch
For tpcds Q15:
{code}
explain
select ca_zip, sum(cs_sales_price)
from catalog_sales, customer, customer_address, date_dim
where catalog_sales.cs_bill_customer_sk = customer.c_customer_sk
and customer.c_current_addr_sk = customer_address.ca_address_sk
and (substr(ca_zip,1,5) in ('85669', '86197','88274','83405','86475',
'85392', '85460', '80348', '81792')
or ca_state in ('CA','WA','GA')
or cs_sales_price 500)
and catalog_sales.cs_sold_date_sk = date_dim.d_date_sk
and d_qoy = 2 and d_year = 2001
group by ca_zip
order by ca_zip
limit 100;
{code}
The Traits setup for the Operators are:
{code}
FIL[23]: bucketCols=[[]],numBuckets=-1
RS[11]: bucketCols=[[VALUE._col0]],numBuckets=-1
JOIN[12]: bucketCols=[[_col71], [_col71]],numBuckets=-1
FIL[13]: bucketCols=[[_col71], [_col71]],numBuckets=-1
SEL[14]: bucketCols=[[_col71], [_col71]],numBuckets=-1
GBY[15]: bucketCols=[[_col0]],numBuckets=-1
RS[16]: bucketCols=[[KEY._col0]],numBuckets=-1
GBY[17]: bucketCols=[[_col0]],numBuckets=-1
SEL[18]: bucketCols=[[_col0]],numBuckets=-1
LIM[21]: bucketCols=[[_col0]],numBuckets=-1
FS[22]: bucketCols=[[_col0]],numBuckets=-1
TS[3]: bucketCols=[[]],numBuckets=-1
RS[5]: bucketCols=[[VALUE._col0]],numBuckets=-1
JOIN[6]: bucketCols=[[_col3], [_col36]],numBuckets=-1
RS[7]: bucketCols=[[VALUE._col40]],numBuckets=-1
JOIN[9]: bucketCols=[[_col40], [_col0]],numBuckets=-1
RS[10]: bucketCols=[[VALUE._col0]],numBuckets=-1
TS[1]: bucketCols=[[]],numBuckets=-1
RS[8]: bucketCols=[[VALUE._col0]],numBuckets=-1
TS[0]: bucketCols=[[]],numBuckets=-1
RS[4]: bucketCols=[[VALUE._col3]],numBuckets=-1
{code}
This is incorrect:
Join[9] joins ca join (cs join cust). In this case both sides of join have a
'_col0' column. The reverse mapping of trait propagation relies on
ExprNodeColumnDesc.isSame; since this doesn't account for the tabAlias we end
up with Join[9] being bucketed on cs_sold_date_sk; Join[12] has the same
issue, only compounds the error.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6955) ExprNodeColDesc isSame doesn't account for tabAlias: this affects trait Propagation in Joins
[
https://issues.apache.org/jira/browse/HIVE-6955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981443#comment-13981443
]
Harish Butani commented on HIVE-6955:
-
Thanks for reviewing [~hagleitn].
No look like failures because of the move to hadoop-2. Validating by running
failed tests locally.
ExprNodeColDesc isSame doesn't account for tabAlias: this affects trait
Propagation in Joins
Key: HIVE-6955
URL: https://issues.apache.org/jira/browse/HIVE-6955
Project: Hive
Issue Type: Bug
Reporter: Harish Butani
Assignee: Harish Butani
Attachments: HIVE-6955.1.patch
For tpcds Q15:
{code}
explain
select ca_zip, sum(cs_sales_price)
from catalog_sales, customer, customer_address, date_dim
where catalog_sales.cs_bill_customer_sk = customer.c_customer_sk
and customer.c_current_addr_sk = customer_address.ca_address_sk
and (substr(ca_zip,1,5) in ('85669', '86197','88274','83405','86475',
'85392', '85460', '80348', '81792')
or ca_state in ('CA','WA','GA')
or cs_sales_price 500)
and catalog_sales.cs_sold_date_sk = date_dim.d_date_sk
and d_qoy = 2 and d_year = 2001
group by ca_zip
order by ca_zip
limit 100;
{code}
The Traits setup for the Operators are:
{code}
FIL[23]: bucketCols=[[]],numBuckets=-1
RS[11]: bucketCols=[[VALUE._col0]],numBuckets=-1
JOIN[12]: bucketCols=[[_col71], [_col71]],numBuckets=-1
FIL[13]: bucketCols=[[_col71], [_col71]],numBuckets=-1
SEL[14]: bucketCols=[[_col71], [_col71]],numBuckets=-1
GBY[15]: bucketCols=[[_col0]],numBuckets=-1
RS[16]: bucketCols=[[KEY._col0]],numBuckets=-1
GBY[17]: bucketCols=[[_col0]],numBuckets=-1
SEL[18]: bucketCols=[[_col0]],numBuckets=-1
LIM[21]: bucketCols=[[_col0]],numBuckets=-1
FS[22]: bucketCols=[[_col0]],numBuckets=-1
TS[3]: bucketCols=[[]],numBuckets=-1
RS[5]: bucketCols=[[VALUE._col0]],numBuckets=-1
JOIN[6]: bucketCols=[[_col3], [_col36]],numBuckets=-1
RS[7]: bucketCols=[[VALUE._col40]],numBuckets=-1
JOIN[9]: bucketCols=[[_col40], [_col0]],numBuckets=-1
RS[10]: bucketCols=[[VALUE._col0]],numBuckets=-1
TS[1]: bucketCols=[[]],numBuckets=-1
RS[8]: bucketCols=[[VALUE._col0]],numBuckets=-1
TS[0]: bucketCols=[[]],numBuckets=-1
RS[4]: bucketCols=[[VALUE._col3]],numBuckets=-1
{code}
This is incorrect:
Join[9] joins ca join (cs join cust). In this case both sides of join have a
'_col0' column. The reverse mapping of trait propagation relies on
ExprNodeColumnDesc.isSame; since this doesn't account for the tabAlias we end
up with Join[9] being bucketed on cs_sold_date_sk; Join[12] has the same
issue, only compounds the error.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: Apache Hive 0.13.1
True, I was counting two weeks from today, but 0.13 has already been out for a week. I'm amenable to having an RC1 out on May 5th. If any further issues appear that block, then we can deal with them in an RC2/etc modification to that. On Fri, Apr 25, 2014 at 11:45 AM, Thejas Nair the...@hortonworks.com wrote: On Fri, Apr 25, 2014 at 11:33 AM, Sushanth Sowmyan khorg...@gmail.comwrote: I think it's important to get a bugfix/stabilization release reasonably quickly, but it's also important to give people a little time to try out 0.13, discover/report bugs and fix them. So I think about two weeks is a good point? And instead of releasing an RC on a friday, I'm thinking of pushing it out to Monday - does 12th May sound good to everyone? I think we can aim for an earlier date. Most of these issues seem to be already committed to trunk or have patches available. So the remaining ones also might get committed to trunk by early next week. How about shooting for May 5th (Monday) ? By then 0.13 would also have been out for 2 weeks. If we have any new critical bug reported that needs a fix, we can hold off on the RC for few days. What do you think ? Thanks, Thejas -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [ANNOUNCE] New Hive Committers - Prasanth J and Vaibhav Gumashta
Congrats Prasanth and Vaibhav :) Thanks, Rahman On Apr 25, 2014, at 11:41 AM, Vaibhav Gumashta vgumas...@hortonworks.com wrote: This is cool! Thanks a lot guys! --Vaibhav On Fri, Apr 25, 2014 at 10:19 AM, Thejas Nair the...@hortonworks.com wrote: Congrats Prasanth and Vaibhav! On Fri, Apr 25, 2014 at 10:16 AM, Prasanth J j.prasant...@gmail.com wrote: Its a wonderful news. :) Thanks for your wishes guys!! Thanks Prasanth On Apr 25, 2014, at 8:52 AM, Sushanth Sowmyan khorg...@gmail.com wrote: Congrats, guys! :) On Fri, Apr 25, 2014 at 12:33 AM, Lefty Leverenz leftylever...@gmail.com wrote: Congratulations! -- Lefty On Fri, Apr 25, 2014 at 12:10 AM, Hari Subramaniyan hsubramani...@hortonworks.com wrote: Congrats Prasanth and Vaibhav! Thanks Hari On Thu, Apr 24, 2014 at 8:45 PM, Chinna Rao Lalam lalamchinnara...@gmail.com wrote: Congratulations to Prasanth and Vaibhav! On Fri, Apr 25, 2014 at 8:23 AM, Shengjun Xin s...@gopivotal.com wrote: Congratulations ~~ On Fri, Apr 25, 2014 at 10:33 AM, Carl Steinbach cwsteinb...@gmail.com wrote: + Prasanth's correct email address On Thu, Apr 24, 2014 at 7:31 PM, Xuefu Zhang xzh...@cloudera.com wrote: Congratulations to Prasanth and Vaibhav! --Xuefu On Thu, Apr 24, 2014 at 7:26 PM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Prasanth J and Vaibhav Gumashta committers on the Apache Hive Project. Please join me in congratulating Prasanth and Vaibhav! Thanks. - Carl -- Regards Shengjun -- Hope It Helps, Chinna -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-6955) ExprNodeColDesc isSame doesn't account for tabAlias: this affects trait Propagation in Joins
[
https://issues.apache.org/jira/browse/HIVE-6955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981468#comment-13981468
]
Harish Butani commented on HIVE-6955:
-
Ran tests locally on hadoop-1. Tests pass. These are related to hadoop-2 switch.
See similar failures in HIVE-6934
ExprNodeColDesc isSame doesn't account for tabAlias: this affects trait
Propagation in Joins
Key: HIVE-6955
URL: https://issues.apache.org/jira/browse/HIVE-6955
Project: Hive
Issue Type: Bug
Reporter: Harish Butani
Assignee: Harish Butani
Attachments: HIVE-6955.1.patch
For tpcds Q15:
{code}
explain
select ca_zip, sum(cs_sales_price)
from catalog_sales, customer, customer_address, date_dim
where catalog_sales.cs_bill_customer_sk = customer.c_customer_sk
and customer.c_current_addr_sk = customer_address.ca_address_sk
and (substr(ca_zip,1,5) in ('85669', '86197','88274','83405','86475',
'85392', '85460', '80348', '81792')
or ca_state in ('CA','WA','GA')
or cs_sales_price 500)
and catalog_sales.cs_sold_date_sk = date_dim.d_date_sk
and d_qoy = 2 and d_year = 2001
group by ca_zip
order by ca_zip
limit 100;
{code}
The Traits setup for the Operators are:
{code}
FIL[23]: bucketCols=[[]],numBuckets=-1
RS[11]: bucketCols=[[VALUE._col0]],numBuckets=-1
JOIN[12]: bucketCols=[[_col71], [_col71]],numBuckets=-1
FIL[13]: bucketCols=[[_col71], [_col71]],numBuckets=-1
SEL[14]: bucketCols=[[_col71], [_col71]],numBuckets=-1
GBY[15]: bucketCols=[[_col0]],numBuckets=-1
RS[16]: bucketCols=[[KEY._col0]],numBuckets=-1
GBY[17]: bucketCols=[[_col0]],numBuckets=-1
SEL[18]: bucketCols=[[_col0]],numBuckets=-1
LIM[21]: bucketCols=[[_col0]],numBuckets=-1
FS[22]: bucketCols=[[_col0]],numBuckets=-1
TS[3]: bucketCols=[[]],numBuckets=-1
RS[5]: bucketCols=[[VALUE._col0]],numBuckets=-1
JOIN[6]: bucketCols=[[_col3], [_col36]],numBuckets=-1
RS[7]: bucketCols=[[VALUE._col40]],numBuckets=-1
JOIN[9]: bucketCols=[[_col40], [_col0]],numBuckets=-1
RS[10]: bucketCols=[[VALUE._col0]],numBuckets=-1
TS[1]: bucketCols=[[]],numBuckets=-1
RS[8]: bucketCols=[[VALUE._col0]],numBuckets=-1
TS[0]: bucketCols=[[]],numBuckets=-1
RS[4]: bucketCols=[[VALUE._col3]],numBuckets=-1
{code}
This is incorrect:
Join[9] joins ca join (cs join cust). In this case both sides of join have a
'_col0' column. The reverse mapping of trait propagation relies on
ExprNodeColumnDesc.isSame; since this doesn't account for the tabAlias we end
up with Join[9] being bucketed on cs_sold_date_sk; Join[12] has the same
issue, only compounds the error.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6469) skipTrash option in hive command line
[ https://issues.apache.org/jira/browse/HIVE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayesh updated HIVE-6469: - Attachment: HIVE-6469.1.patch skipTrash option in hive command line - Key: HIVE-6469 URL: https://issues.apache.org/jira/browse/HIVE-6469 Project: Hive Issue Type: New Feature Components: CLI Affects Versions: 0.12.0 Reporter: Jayesh Fix For: 0.12.1 Attachments: HIVE-6469.1.patch, HIVE-6469.patch hive drop table command deletes the data from HDFS warehouse and puts it into Trash. Currently there is no way to provide flag to tell warehouse to skip trash while deleting table data. This ticket is to add skipTrash feature in hive command-line, that looks as following. hive -e drop table skipTrash testTable This would be good feature to add, so that user can specify when not to put data into trash directory and thus not to fill hdfs space instead of relying on trash interval and policy configuration to take care of disk filling issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6469) skipTrash option in hive command line
[ https://issues.apache.org/jira/browse/HIVE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981512#comment-13981512 ] Jayesh commented on HIVE-6469: -- Hi [~xuefuz], I have uploaded HIVE-6469.1.patch that exposes this functionality by configuration. Appreciate your guidance and feedback to drive this ticket. skipTrash option in hive command line - Key: HIVE-6469 URL: https://issues.apache.org/jira/browse/HIVE-6469 Project: Hive Issue Type: New Feature Components: CLI Affects Versions: 0.12.0 Reporter: Jayesh Fix For: 0.12.1 Attachments: HIVE-6469.1.patch, HIVE-6469.patch hive drop table command deletes the data from HDFS warehouse and puts it into Trash. Currently there is no way to provide flag to tell warehouse to skip trash while deleting table data. This ticket is to add skipTrash feature in hive command-line, that looks as following. hive -e drop table skipTrash testTable This would be good feature to add, so that user can specify when not to put data into trash directory and thus not to fill hdfs space instead of relying on trash interval and policy configuration to take care of disk filling issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6469) skipTrash option in hive command line
[
https://issues.apache.org/jira/browse/HIVE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981567#comment-13981567
]
Xuefu Zhang commented on HIVE-6469:
---
[~jhsenjaliya] Thanks for the patch. It looks good to me. Minor nit: Hive code
style expects space between ){ in the following if/else block.
{code}
+ if (skipTrash){
+LOG.info(Not moving+ f + to trash due to configuration +
+ HiveConf.ConfVars.HIVE_WAREHOUSE_DATA_SKIPTRASH + is set to
true.);
+ }else if (hadoopShim.moveToAppropriateTrash(fs, f, conf)) {
{code}
Would you mind fixing the format?
skipTrash option in hive command line
-
Key: HIVE-6469
URL: https://issues.apache.org/jira/browse/HIVE-6469
Project: Hive
Issue Type: New Feature
Components: CLI
Affects Versions: 0.12.0
Reporter: Jayesh
Fix For: 0.12.1
Attachments: HIVE-6469.1.patch, HIVE-6469.patch
hive drop table command deletes the data from HDFS warehouse and puts it into
Trash.
Currently there is no way to provide flag to tell warehouse to skip trash
while deleting table data.
This ticket is to add skipTrash feature in hive command-line, that looks as
following.
hive -e drop table skipTrash testTable
This would be good feature to add, so that user can specify when not to put
data into trash directory and thus not to fill hdfs space instead of relying
on trash interval and policy configuration to take care of disk filling issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6835) Reading of partitioned Avro data fails if partition schema does not match table schema
[
https://issues.apache.org/jira/browse/HIVE-6835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-6835:
--
Resolution: Fixed
Fix Version/s: 0.14.0
Status: Resolved (was: Patch Available)
Patch committed to trunk. Thanks Anthony for the contribution.
Reading of partitioned Avro data fails if partition schema does not match
table schema
--
Key: HIVE-6835
URL: https://issues.apache.org/jira/browse/HIVE-6835
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0
Reporter: Anthony Hsu
Assignee: Anthony Hsu
Fix For: 0.14.0
Attachments: HIVE-6835.1.patch, HIVE-6835.2.patch, HIVE-6835.3.patch,
HIVE-6835.4.patch, HIVE-6835.5.patch
To reproduce:
{code}
create table testarray (a arraystring);
load data local inpath '/home/ahsu/test/array.txt' into table testarray;
# create partitioned Avro table with one array column
create table avroarray partitioned by (y string) row format serde
'org.apache.hadoop.hive.serde2.avro.AvroSerDe' with serdeproperties
('avro.schema.literal'='{namespace:test,name:avroarray,type:
record, fields: [ { name:a, type:{type:array,items:string}
} ] }') STORED as INPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat';
insert into table avroarray partition(y=1) select * from testarray;
# add an int column with a default value of 0
alter table avroarray set serde
'org.apache.hadoop.hive.serde2.avro.AvroSerDe' with
serdeproperties('avro.schema.literal'='{namespace:test,name:avroarray,type:
record, fields: [ {name:intfield,type:int,default:0},{
name:a, type:{type:array,items:string} } ] }');
# fails with ClassCastException
select * from avroarray;
{code}
The select * fails with:
{code}
Failed with exception java.io.IOException:java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
cannot be cast to
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6945) issues with dropping partitions on Oracle
[ https://issues.apache.org/jira/browse/HIVE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981595#comment-13981595 ] Sergey Shelukhin commented on HIVE-6945: committed to trunk issues with dropping partitions on Oracle - Key: HIVE-6945 URL: https://issues.apache.org/jira/browse/HIVE-6945 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.14.0 Attachments: HIVE-6945.01.patch, HIVE-6945.02.patch, HIVE-6945.patch 1) Direct SQL is broken on Oracle due to the usage of NUMBER type which is translated by DN into decimal rather than long. This appears to be specific to some cases because it seemed to have worked before (different version of Oracle? JDBC? DN? Maybe depends on whether db was auto-created). 2) When partition dropping code falls back to JDO, it creates objects to return, then drops partitions. It appears that dropping makes DN objects invalid. We create metastore partition objects out of DN objects before drop, however the list of partition column values is re-used, rather than copied, into these. DN appears to clear this list during drop, so the returned object becomes invalid and the exception is thrown. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6945) issues with dropping partitions on Oracle
[ https://issues.apache.org/jira/browse/HIVE-6945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-6945: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) issues with dropping partitions on Oracle - Key: HIVE-6945 URL: https://issues.apache.org/jira/browse/HIVE-6945 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.14.0 Attachments: HIVE-6945.01.patch, HIVE-6945.02.patch, HIVE-6945.patch 1) Direct SQL is broken on Oracle due to the usage of NUMBER type which is translated by DN into decimal rather than long. This appears to be specific to some cases because it seemed to have worked before (different version of Oracle? JDBC? DN? Maybe depends on whether db was auto-created). 2) When partition dropping code falls back to JDO, it creates objects to return, then drops partitions. It appears that dropping makes DN objects invalid. We create metastore partition objects out of DN objects before drop, however the list of partition column values is re-used, rather than copied, into these. DN appears to clear this list during drop, so the returned object becomes invalid and the exception is thrown. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Apache Hive 0.13.1
I'd like to request to include these Tez fixes: HIVE-6824, HIVE-6826, HIVE-6828, HIVE-6898 Thanks, Gunther. On Fri, Apr 25, 2014 at 11:59 AM, Sushanth Sowmyan khorg...@gmail.comwrote: True, I was counting two weeks from today, but 0.13 has already been out for a week. I'm amenable to having an RC1 out on May 5th. If any further issues appear that block, then we can deal with them in an RC2/etc modification to that. On Fri, Apr 25, 2014 at 11:45 AM, Thejas Nair the...@hortonworks.com wrote: On Fri, Apr 25, 2014 at 11:33 AM, Sushanth Sowmyan khorg...@gmail.com wrote: I think it's important to get a bugfix/stabilization release reasonably quickly, but it's also important to give people a little time to try out 0.13, discover/report bugs and fix them. So I think about two weeks is a good point? And instead of releasing an RC on a friday, I'm thinking of pushing it out to Monday - does 12th May sound good to everyone? I think we can aim for an earlier date. Most of these issues seem to be already committed to trunk or have patches available. So the remaining ones also might get committed to trunk by early next week. How about shooting for May 5th (Monday) ? By then 0.13 would also have been out for 2 weeks. If we have any new critical bug reported that needs a fix, we can hold off on the RC for few days. What do you think ? Thanks, Thejas -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Created] (HIVE-6977) Delete Hiveserver1
Ashutosh Chauhan created HIVE-6977: -- Summary: Delete Hiveserver1 Key: HIVE-6977 URL: https://issues.apache.org/jira/browse/HIVE-6977 Project: Hive Issue Type: Task Components: JDBC, Server Infrastructure Reporter: Ashutosh Chauhan See mailing list discussion. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Remove HiveServer1
Seems like there is an agreement in principle. I created https://issues.apache.org/jira/browse/HIVE-6977 for it. I dont have cycles at the moment for it, so if anyone is interested feel free to take it up, else I will get to it later. On Thu, Apr 17, 2014 at 12:04 PM, Xuefu Zhang xzh...@cloudera.com wrote: +1 removing server1 and related. However, +1 on keeping Hive CLI. On Thu, Apr 17, 2014 at 11:34 AM, Vaibhav Gumashta vgumas...@hortonworks.com wrote: I am +1 on it. I'd also add that we removed JDBC-1 which was supposed to work with HiveServer1. Thanks, --Vaibhav On Thu, Apr 17, 2014 at 11:26 AM, Ashutosh Chauhan hashut...@apache.org wrote: HiveServer2 was introduced in Hive 0.10 since than we have 3 releases 0.11, 0.12 soon to be 0.13. I think its a high time we remove HS1 from our trunk. Thoughts? Thanks, Ashutosh -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-6826) Hive-tez has issues when different partitions work off of different input types
[ https://issues.apache.org/jira/browse/HIVE-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981634#comment-13981634 ] Gunther Hagleitner commented on HIVE-6826: -- Some comments on rb. The code says this is a stop gap fix, and I agree. Can you open a follow up jira for the fix using custom input initializer? Hive-tez has issues when different partitions work off of different input types --- Key: HIVE-6826 URL: https://issues.apache.org/jira/browse/HIVE-6826 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.13.0, 0.14.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: HIVE-6826.1.patch create table test (key int, value string) partitioned by (p int) stored as textfile; insert into table test partition (p=1) select * from src limit 10; alter table test set fileformat orc; insert into table test partition (p=2) select * from src limit 10; describe test; select * from test where p=1 and key 0; select * from test where p=2 and key 0; select * from test where key 0; throws a classcast exception -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6976) Show query id only when there's jobs on the cluster
[ https://issues.apache.org/jira/browse/HIVE-6976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981638#comment-13981638 ] Sergey Shelukhin commented on HIVE-6976: +1... perhaps printInfos can be combined Show query id only when there's jobs on the cluster --- Key: HIVE-6976 URL: https://issues.apache.org/jira/browse/HIVE-6976 Project: Hive Issue Type: Bug Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Priority: Minor Attachments: HIVE-6976.1.patch No need to print the query id for local-only execution. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6828) Hive tez bucket map join conversion interferes with map join conversion
[ https://issues.apache.org/jira/browse/HIVE-6828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981640#comment-13981640 ] Gunther Hagleitner commented on HIVE-6828: -- +1 Hive tez bucket map join conversion interferes with map join conversion --- Key: HIVE-6828 URL: https://issues.apache.org/jira/browse/HIVE-6828 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.13.0, 0.14.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: HIVE-6828.1.patch The issue is that bucket count is used for checking the scaled down size of the hash tables but is used later on to convert to the map join as well which may be incorrect in cases where the entire hash table does not fit in the specified size. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6824) Hive HBase query fails on Tez due to missing jars - part 2
[ https://issues.apache.org/jira/browse/HIVE-6824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-6824: --- Description: Follow-up from HIVE-6739. We cannot wait for Tez 0.4 (or even be sure that it will have TEZ-1004 and TEZ-1005), so I will split the patch into two. Original jira will have the straightforward (but less efficient) fix. This jira will use new relocalize APIs. -Depending on relative timing of Tez 0.4 release and Hive 0.13 release, this will go into 0.13 or 0.14- blocked on Tez 0.5 (was: Follow-up from HIVE-6739. We cannot wait for Tez 0.4 (or even be sure that it will have TEZ-1004 and TEZ-1005), so I will split the patch into two. Original jira will have the straightforward (but less efficient) fix. This jira will use new relocalize APIs. Depending on relative timing of Tez 0.4 release and Hive 0.13 release, this will go into 0.13 or 0.14) Hive HBase query fails on Tez due to missing jars - part 2 -- Key: HIVE-6824 URL: https://issues.apache.org/jira/browse/HIVE-6824 Project: Hive Issue Type: Bug Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Fix For: 0.14.0 Attachments: HIVE-6824.patch Follow-up from HIVE-6739. We cannot wait for Tez 0.4 (or even be sure that it will have TEZ-1004 and TEZ-1005), so I will split the patch into two. Original jira will have the straightforward (but less efficient) fix. This jira will use new relocalize APIs. -Depending on relative timing of Tez 0.4 release and Hive 0.13 release, this will go into 0.13 or 0.14- blocked on Tez 0.5 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6469) skipTrash option in hive command line
[ https://issues.apache.org/jira/browse/HIVE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayesh updated HIVE-6469: - Attachment: HIVE-6469.2.patch sure, updated patch. Thanks! skipTrash option in hive command line - Key: HIVE-6469 URL: https://issues.apache.org/jira/browse/HIVE-6469 Project: Hive Issue Type: New Feature Components: CLI Affects Versions: 0.12.0 Reporter: Jayesh Fix For: 0.12.1 Attachments: HIVE-6469.1.patch, HIVE-6469.2.patch, HIVE-6469.patch hive drop table command deletes the data from HDFS warehouse and puts it into Trash. Currently there is no way to provide flag to tell warehouse to skip trash while deleting table data. This ticket is to add skipTrash feature in hive command-line, that looks as following. hive -e drop table skipTrash testTable This would be good feature to add, so that user can specify when not to put data into trash directory and thus not to fill hdfs space instead of relying on trash interval and policy configuration to take care of disk filling issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Apache Hive 0.13.1
Sorry - HIVE-6824 isn't needed. Just the other 3. My bad. Thanks, Gunther. On Fri, Apr 25, 2014 at 2:10 PM, Gunther Hagleitner ghagleit...@hortonworks.com wrote: I'd like to request to include these Tez fixes: HIVE-6824, HIVE-6826, HIVE-6828, HIVE-6898 Thanks, Gunther. On Fri, Apr 25, 2014 at 11:59 AM, Sushanth Sowmyan khorg...@gmail.comwrote: True, I was counting two weeks from today, but 0.13 has already been out for a week. I'm amenable to having an RC1 out on May 5th. If any further issues appear that block, then we can deal with them in an RC2/etc modification to that. On Fri, Apr 25, 2014 at 11:45 AM, Thejas Nair the...@hortonworks.com wrote: On Fri, Apr 25, 2014 at 11:33 AM, Sushanth Sowmyan khorg...@gmail.com wrote: I think it's important to get a bugfix/stabilization release reasonably quickly, but it's also important to give people a little time to try out 0.13, discover/report bugs and fix them. So I think about two weeks is a good point? And instead of releasing an RC on a friday, I'm thinking of pushing it out to Monday - does 12th May sound good to everyone? I think we can aim for an earlier date. Most of these issues seem to be already committed to trunk or have patches available. So the remaining ones also might get committed to trunk by early next week. How about shooting for May 5th (Monday) ? By then 0.13 would also have been out for 2 weeks. If we have any new critical bug reported that needs a fix, we can hold off on the RC for few days. What do you think ? Thanks, Thejas -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-6469) skipTrash option in hive command line
[
https://issues.apache.org/jira/browse/HIVE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981660#comment-13981660
]
Xuefu Zhang commented on HIVE-6469:
---
+1.
There is on more instance at {code} }else if {code}. I can fix that when I
commit this.
skipTrash option in hive command line
-
Key: HIVE-6469
URL: https://issues.apache.org/jira/browse/HIVE-6469
Project: Hive
Issue Type: New Feature
Components: CLI
Affects Versions: 0.12.0
Reporter: Jayesh
Fix For: 0.12.1
Attachments: HIVE-6469.1.patch, HIVE-6469.2.patch, HIVE-6469.patch
hive drop table command deletes the data from HDFS warehouse and puts it into
Trash.
Currently there is no way to provide flag to tell warehouse to skip trash
while deleting table data.
This ticket is to add skipTrash feature in hive command-line, that looks as
following.
hive -e drop table skipTrash testTable
This would be good feature to add, so that user can specify when not to put
data into trash directory and thus not to fill hdfs space instead of relying
on trash interval and policy configuration to take care of disk filling issue.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6469) skipTrash option in hive command line
[ https://issues.apache.org/jira/browse/HIVE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981666#comment-13981666 ] Xuefu Zhang commented on HIVE-6469: --- BTW, you might want to be included in the contributor list. For that, you can send an email to dev@hive.apache.org. Right now i cannot assign this JIRA under your name. skipTrash option in hive command line - Key: HIVE-6469 URL: https://issues.apache.org/jira/browse/HIVE-6469 Project: Hive Issue Type: New Feature Components: CLI Affects Versions: 0.12.0 Reporter: Jayesh Fix For: 0.12.1 Attachments: HIVE-6469.1.patch, HIVE-6469.2.patch, HIVE-6469.patch hive drop table command deletes the data from HDFS warehouse and puts it into Trash. Currently there is no way to provide flag to tell warehouse to skip trash while deleting table data. This ticket is to add skipTrash feature in hive command-line, that looks as following. hive -e drop table skipTrash testTable This would be good feature to add, so that user can specify when not to put data into trash directory and thus not to fill hdfs space instead of relying on trash interval and policy configuration to take care of disk filling issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
join dev channel
join dev channel
[jira] [Commented] (HIVE-6835) Reading of partitioned Avro data fails if partition schema does not match table schema
[
https://issues.apache.org/jira/browse/HIVE-6835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981677#comment-13981677
]
Anthony Hsu commented on HIVE-6835:
---
Thanks, [~xuefuz], for all your help and guidance.
Reading of partitioned Avro data fails if partition schema does not match
table schema
--
Key: HIVE-6835
URL: https://issues.apache.org/jira/browse/HIVE-6835
Project: Hive
Issue Type: Bug
Affects Versions: 0.12.0
Reporter: Anthony Hsu
Assignee: Anthony Hsu
Fix For: 0.14.0
Attachments: HIVE-6835.1.patch, HIVE-6835.2.patch, HIVE-6835.3.patch,
HIVE-6835.4.patch, HIVE-6835.5.patch
To reproduce:
{code}
create table testarray (a arraystring);
load data local inpath '/home/ahsu/test/array.txt' into table testarray;
# create partitioned Avro table with one array column
create table avroarray partitioned by (y string) row format serde
'org.apache.hadoop.hive.serde2.avro.AvroSerDe' with serdeproperties
('avro.schema.literal'='{namespace:test,name:avroarray,type:
record, fields: [ { name:a, type:{type:array,items:string}
} ] }') STORED as INPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat' OUTPUTFORMAT
'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat';
insert into table avroarray partition(y=1) select * from testarray;
# add an int column with a default value of 0
alter table avroarray set serde
'org.apache.hadoop.hive.serde2.avro.AvroSerDe' with
serdeproperties('avro.schema.literal'='{namespace:test,name:avroarray,type:
record, fields: [ {name:intfield,type:int,default:0},{
name:a, type:{type:array,items:string} } ] }');
# fails with ClassCastException
select * from avroarray;
{code}
The select * fails with:
{code}
Failed with exception java.io.IOException:java.lang.ClassCastException:
org.apache.hadoop.hive.serde2.objectinspector.StandardListObjectInspector
cannot be cast to
org.apache.hadoop.hive.serde2.objectinspector.PrimitiveObjectInspector
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6952) Hive 0.13 HiveOutputFormat breaks backwards compatibility
[ https://issues.apache.org/jira/browse/HIVE-6952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6952: --- Attachment: HIVE-6952_branch-13.patch patch for branch 0.13 Hive 0.13 HiveOutputFormat breaks backwards compatibility - Key: HIVE-6952 URL: https://issues.apache.org/jira/browse/HIVE-6952 Project: Hive Issue Type: Bug Components: File Formats, Serializers/Deserializers Affects Versions: 0.13.0 Reporter: Costin Leau Assignee: Ashutosh Chauhan Priority: Blocker Fix For: 0.14.0 Attachments: HIVE-6952.patch, HIVE-6952_branch-13.patch Hive 0.13 changed the signature of HiveOutputFormat (through commit r1527149) breaking backwards compatibility with previous releases; the return type of getHiveRecordWriter has been changed from RecordWriter to FSRecordWriter. FSRecordWriter introduces one new method on top of RecordWriter however it does not extend the previous interface and it lives in a completely new package. Thus code running fine on Hive 0.12 breaks on Hive 0.13. After the upgrade, code running on HIve 0.13, will break on anything lower than this. This could have easily been avoided by extending the existing interface or introducing a new one that RecordWriter could have extended going forward. By changing the signature, the existing contract (and compatibility) has been voided. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Assigned] (HIVE-5969) SQL std auth - authorize create database
[ https://issues.apache.org/jira/browse/HIVE-5969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair reassigned HIVE-5969: --- Assignee: Thejas M Nair SQL std auth - authorize create database Key: HIVE-5969 URL: https://issues.apache.org/jira/browse/HIVE-5969 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Thejas M Nair Original Estimate: 48h Remaining Estimate: 48h Permission to create a database must be given to any user, with the following caveats - The user must have HDFS rights to create a directory in the Hive warehouse directory, if no database location is specified; - or the user must own the directory provided in the database location specification (in this case it is recommended but not required that the group of the directory be hive, so that the hive user has access to the data in the database); - and if the database is owned by a role (rather than a user) the directory must be owned by the hive user. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HIVE-5969) SQL std auth - authorize create database
[ https://issues.apache.org/jira/browse/HIVE-5969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair resolved HIVE-5969. - Resolution: Implemented This is already implemented as part of HIVE-5958 . SQL std auth - authorize create database Key: HIVE-5969 URL: https://issues.apache.org/jira/browse/HIVE-5969 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Original Estimate: 48h Remaining Estimate: 48h Permission to create a database must be given to any user, with the following caveats - The user must have HDFS rights to create a directory in the Hive warehouse directory, if no database location is specified; - or the user must own the directory provided in the database location specification (in this case it is recommended but not required that the group of the directory be hive, so that the hive user has access to the data in the database); - and if the database is owned by a role (rather than a user) the directory must be owned by the hive user. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6469) skipTrash option in hive command line
[ https://issues.apache.org/jira/browse/HIVE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981675#comment-13981675 ] Jayesh commented on HIVE-6469: -- uploaded HIVE-6469.3.patch with syntax update and emailed dev@hive.apache.org skipTrash option in hive command line - Key: HIVE-6469 URL: https://issues.apache.org/jira/browse/HIVE-6469 Project: Hive Issue Type: New Feature Components: CLI Affects Versions: 0.12.0 Reporter: Jayesh Fix For: 0.12.1 Attachments: HIVE-6469.1.patch, HIVE-6469.2.patch, HIVE-6469.patch hive drop table command deletes the data from HDFS warehouse and puts it into Trash. Currently there is no way to provide flag to tell warehouse to skip trash while deleting table data. This ticket is to add skipTrash feature in hive command-line, that looks as following. hive -e drop table skipTrash testTable This would be good feature to add, so that user can specify when not to put data into trash directory and thus not to fill hdfs space instead of relying on trash interval and policy configuration to take care of disk filling issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-4965) Add support so that PTFs can stream their output; Windowing PTF should do this
[ https://issues.apache.org/jira/browse/HIVE-4965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Butani updated HIVE-4965: Status: Patch Available (was: Open) Add support so that PTFs can stream their output; Windowing PTF should do this -- Key: HIVE-4965 URL: https://issues.apache.org/jira/browse/HIVE-4965 Project: Hive Issue Type: Bug Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-4965.4.patch, HIVE-4965.D12033.1.patch, HIVE-4965.D12615.1.patch There is no need to create an output PTF Partition for the last PTF in a chain. For the Windowing PTF this should give a perf. boost; we avoid creating temporary results for each UDAF; avoid populating an output Partition. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5969) SQL std auth - authorize create database
[ https://issues.apache.org/jira/browse/HIVE-5969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981679#comment-13981679 ] Thejas M Nair commented on HIVE-5969: - Regarding the statement in description: bq. and if the database is owned by a role (rather than a user) the directory must be owned by the hive user. The create database statement does not allow specifying the owner role. So this is not currently applicable. The ownership can be changed to another using or role only using admin role privileges. SQL std auth - authorize create database Key: HIVE-5969 URL: https://issues.apache.org/jira/browse/HIVE-5969 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Thejas M Nair Original Estimate: 48h Remaining Estimate: 48h Permission to create a database must be given to any user, with the following caveats - The user must have HDFS rights to create a directory in the Hive warehouse directory, if no database location is specified; - or the user must own the directory provided in the database location specification (in this case it is recommended but not required that the group of the directory be hive, so that the hive user has access to the data in the database); - and if the database is owned by a role (rather than a user) the directory must be owned by the hive user. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-4965) Add support so that PTFs can stream their output; Windowing PTF should do this
[ https://issues.apache.org/jira/browse/HIVE-4965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Butani updated HIVE-4965: Attachment: HIVE-4965.4.patch Add support so that PTFs can stream their output; Windowing PTF should do this -- Key: HIVE-4965 URL: https://issues.apache.org/jira/browse/HIVE-4965 Project: Hive Issue Type: Bug Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-4965.4.patch, HIVE-4965.D12033.1.patch, HIVE-4965.D12615.1.patch There is no need to create an output PTF Partition for the last PTF in a chain. For the Windowing PTF this should give a perf. boost; we avoid creating temporary results for each UDAF; avoid populating an output Partition. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6469) skipTrash option in hive command line
[ https://issues.apache.org/jira/browse/HIVE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6469: --- Assignee: Jayesh skipTrash option in hive command line - Key: HIVE-6469 URL: https://issues.apache.org/jira/browse/HIVE-6469 Project: Hive Issue Type: New Feature Components: CLI Affects Versions: 0.12.0 Reporter: Jayesh Assignee: Jayesh Fix For: 0.12.1 Attachments: HIVE-6469.1.patch, HIVE-6469.2.patch, HIVE-6469.3.patch, HIVE-6469.patch hive drop table command deletes the data from HDFS warehouse and puts it into Trash. Currently there is no way to provide flag to tell warehouse to skip trash while deleting table data. This ticket is to add skipTrash feature in hive command-line, that looks as following. hive -e drop table skipTrash testTable This would be good feature to add, so that user can specify when not to put data into trash directory and thus not to fill hdfs space instead of relying on trash interval and policy configuration to take care of disk filling issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6469) skipTrash option in hive command line
[ https://issues.apache.org/jira/browse/HIVE-6469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayesh updated HIVE-6469: - Attachment: HIVE-6469.3.patch skipTrash option in hive command line - Key: HIVE-6469 URL: https://issues.apache.org/jira/browse/HIVE-6469 Project: Hive Issue Type: New Feature Components: CLI Affects Versions: 0.12.0 Reporter: Jayesh Assignee: Jayesh Fix For: 0.12.1 Attachments: HIVE-6469.1.patch, HIVE-6469.2.patch, HIVE-6469.3.patch, HIVE-6469.patch hive drop table command deletes the data from HDFS warehouse and puts it into Trash. Currently there is no way to provide flag to tell warehouse to skip trash while deleting table data. This ticket is to add skipTrash feature in hive command-line, that looks as following. hive -e drop table skipTrash testTable This would be good feature to add, so that user can specify when not to put data into trash directory and thus not to fill hdfs space instead of relying on trash interval and policy configuration to take care of disk filling issue. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6898) Functions in hive are failing with java.lang.ClassNotFoundException on Tez
[
https://issues.apache.org/jira/browse/HIVE-6898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-6898:
---
Resolution: Fixed
Fix Version/s: 0.14.0
Status: Resolved (was: Patch Available)
committed to trunk
Functions in hive are failing with java.lang.ClassNotFoundException on Tez
--
Key: HIVE-6898
URL: https://issues.apache.org/jira/browse/HIVE-6898
Project: Hive
Issue Type: Bug
Components: Tez
Affects Versions: 0.13.0, 0.14.0
Reporter: Vikram Dixit K
Assignee: Vikram Dixit K
Fix For: 0.14.0
Attachments: HIVE-6898.1.patch, HIVE-6898.2.patch
{code}
CREATE TABLE T1(key int, val STRING) STORED AS TEXTFILE;
LOAD DATA LOCAL INPATH '../../data/files/T1.txt' INTO TABLE T1;
add jar /tmp/testudf.jar;
create temporary function square as 'org.apache.hive.udf.UDFSquare';
select square(key) from T1 limit 3;
{code}
Fails with
{code}
Vertex failed, vertexName=Map 1, vertexId=vertex_1397230190905_0590_1_00,
diagnostics=[Task failed, taskId=task_1397230190905_0590_1_00_00,
diagnostics=[AttemptID:attempt_1397230190905_0590_1_00_00_0 Info:Error:
java.lang.RuntimeException: Map operator initialization failed
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:145)
at
org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:163)
at
org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:307)
at
org.apache.hadoop.mapred.YarnTezDagChild$5.run(YarnTezDagChild.java:564)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
at
org.apache.hadoop.mapred.YarnTezDagChild.main(YarnTezDagChild.java:553)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.hive.udf.UDFSquare
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.getUdfClass(GenericUDFBridge.java:133)
at
org.apache.hadoop.hive.ql.exec.FunctionRegistry.isStateful(FunctionRegistry.java:1636)
at
org.apache.hadoop.hive.ql.exec.FunctionRegistry.isDeterministic(FunctionRegistry.java:1599)
at
org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.isDeterministic(ExprNodeGenericFuncEvaluator.java:132)
at
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.iterate(ExprNodeEvaluatorFactory.java:83)
at
org.apache.hadoop.hive.ql.exec.ExprNodeEvaluatorFactory.toCachedEval(ExprNodeEvaluatorFactory.java:73)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:59)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:460)
at
org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:416)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:189)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at
org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:425)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:376)
at
org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:121)
... 7 more
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6884) HiveLockObject and enclosed HiveLockObjectData override equal() method but didn't do so for hashcode()
[ https://issues.apache.org/jira/browse/HIVE-6884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-6884: -- Attachment: HIVE-6884.patch HiveLockObject and enclosed HiveLockObjectData override equal() method but didn't do so for hashcode() -- Key: HIVE-6884 URL: https://issues.apache.org/jira/browse/HIVE-6884 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.12.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang Attachments: HIVE-6884.patch This breaches the JAVA contact that equal objects should have equal hash code, thus may cause unexpected results. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6884) HiveLockObject and enclosed HiveLockObjectData override equal() method but didn't do so for hashcode()
[ https://issues.apache.org/jira/browse/HIVE-6884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-6884: -- Status: Patch Available (was: Open) HiveLockObject and enclosed HiveLockObjectData override equal() method but didn't do so for hashcode() -- Key: HIVE-6884 URL: https://issues.apache.org/jira/browse/HIVE-6884 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.12.0 Reporter: Xuefu Zhang Assignee: Xuefu Zhang Attachments: HIVE-6884.patch This breaches the JAVA contact that equal objects should have equal hash code, thus may cause unexpected results. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5823) Support for DECIMAL primitive type in AvroSerDe
[
https://issues.apache.org/jira/browse/HIVE-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981696#comment-13981696
]
Hive QA commented on HIVE-5823:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12641975/HIVE-5823.1.patch
{color:red}ERROR:{color} -1 due to 47 failed/errored test(s), 5423 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_avro_decimal
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_filter_numeric
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby_sort_skew_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_infer_bucket_sort_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_6
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_list_bucket_dml_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_test_outer
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_orc_createas1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_truncate_column_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_1
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_10
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_12
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_13
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_14
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_19
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_2
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_20
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_21
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_22
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_24
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_5
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_7
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_8
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_union_remove_9
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testListPartitions
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testNameMethods
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testPartition
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testListPartitions
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testNameMethods
org.apache.hadoop.hive.metastore.TestSetUGIOnBothClientServer.testPartition
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/43/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/43/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 47 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12641975
Support for DECIMAL primitive type in AvroSerDe
---
Key: HIVE-5823
URL: https://issues.apache.org/jira/browse/HIVE-5823
Project: Hive
Issue Type: New Feature
Components: Serializers/Deserializers
Affects Versions: 0.12.0
Reporter: Mariano Dominguez
Assignee:
[jira] [Commented] (HIVE-6968) list bucketing feature does not update the location map for unpartitioned tables
[
https://issues.apache.org/jira/browse/HIVE-6968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981700#comment-13981700
]
Prasanth J commented on HIVE-6968:
--
test failures are not related.
list bucketing feature does not update the location map for unpartitioned
tables
Key: HIVE-6968
URL: https://issues.apache.org/jira/browse/HIVE-6968
Project: Hive
Issue Type: Bug
Affects Versions: 0.11.0, 0.12.0, 0.13.0, 0.14.0
Reporter: Prasanth J
Assignee: Prasanth J
Attachments: HIVE-6968.1.patch
list bucketing feature maintains a map of skewed columns/values to location
in metastore. This map is not getting updated for unpartitioned tables. For
partitioned tables the location map gets updated properly. To reproduce the
issue
{code}
hiveset hive.mapred.supports.subdirectories=true;
hiveset mapred.input.dir.recursive=true;
hivecreate table t(col1 string, col2 string);
hiveload data local inpath '/home/hadoop/a.txt' into table t;
hive select * from t;
OK
1 a
2 b
3 c
4 a
5 b
6 a
hivecreate tablet1(r1 string, r2 string) skewed by (r2) on (‘a’) stored as
directories;
hiveinsert into table t1 select * from t;
hivedesc extended t1;
OK
r1string
r2string
Detailed Table InformationTable(tableName:t1, dbName:default,
owner:pjayachandran, createTime:1398295903, lastAccessTime:0, retention:0,
sd:StorageDescriptor(cols:[FieldSchema(name:r1, type:string, comment:null),
FieldSchema(name:r2, type:string, comment:null)],
location:file:/app/warehouse/t1,
inputFormat:org.apache.hadoop.mapred.TextInputFormat,
outputFormat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat,
compressed:false, numBuckets:-1, serdeInfo:SerDeInfo(name:null,
serializationLib:org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe,
parameters:{serialization.format=1}), bucketCols:[], sortCols:[],
parameters:{}, skewedInfo:SkewedInfo(skewedColNames:[r2],
skewedColValues:[[a]], skewedColValueLocationMaps:{}),
storedAsSubDirectories:true), partitionKeys:[], parameters:{numFiles=6,
COLUMN_STATS_ACCURATE=true, transient_lastDdlTime=1398297887, numRows=6,
totalSize=72, rawDataSize=18}, viewOriginalText:null, viewExpandedText:null,
tableType:MANAGED_TABLE)
Time taken: 0.119 seconds, Fetched: 4 row(s)
{code}
as seen from describe output *skewedColValueLocationMaps* is empty
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-6978) beeline always exits with 0 status, should exit with non-zero status on error
Gwen Shapira created HIVE-6978: -- Summary: beeline always exits with 0 status, should exit with non-zero status on error Key: HIVE-6978 URL: https://issues.apache.org/jira/browse/HIVE-6978 Project: Hive Issue Type: Bug Components: HiveServer2 Affects Versions: 0.12.0 Reporter: Gwen Shapira Was supposed to be fixed in Hive 0.12 (HIVE-4364). Doesn't look fixed from here. [i@p sqoop]$ beeline -u 'jdbc:hive2://p:1/k;principal=hive/p@L' -e select * from MEMBERS --outputformat=vertical scan complete in 3ms Connecting to jdbc:hive2://p:1/k;principal=hive/p@L SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/zookeeper/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-5.0.0-1.cdh5.0.0.p0.47/lib/avro/avro-tools-1.7.5-cdh5.0.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] Connected to: Apache Hive (version 0.12.0-cdh5.0.0) Driver: Hive JDBC (version 0.12.0-cdh5.0.0) Transaction isolation: TRANSACTION_REPEATABLE_READ -hiveconf (No such file or directory) hive.aux.jars.path=[redacted] Error: Error while compiling statement: FAILED: SemanticException [Error 10001]: Line 1:14 Table not found 'MEMBERS' (state=42S02,code=10001) Beeline version 0.12.0-cdh5.0.0 by Apache Hive Closing: org.apache.hive.jdbc.HiveConnection [inter@p sqoop]$ echo $? 0 -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5072) [WebHCat]Enable directly invoke Sqoop job through Templeton
[ https://issues.apache.org/jira/browse/HIVE-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5072: Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Patch committed to trunk. Thanks for the contribution [~shuainie]. Thanks for the reviews [~ekoifman] [WebHCat]Enable directly invoke Sqoop job through Templeton --- Key: HIVE-5072 URL: https://issues.apache.org/jira/browse/HIVE-5072 Project: Hive Issue Type: Improvement Components: WebHCat Affects Versions: 0.12.0 Reporter: Shuaishuai Nie Assignee: Shuaishuai Nie Fix For: 0.14.0 Attachments: HIVE-5072.1.patch, HIVE-5072.2.patch, HIVE-5072.3.patch, HIVE-5072.4.patch, HIVE-5072.5.patch, Templeton-Sqoop-Action.pdf Now it is hard to invoke a Sqoop job through templeton. The only way is to use the classpath jar generated by a sqoop job and use the jar delegator in Templeton. We should implement Sqoop Delegator to enable directly invoke Sqoop job through Templeton. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 20736: HIVE-4965 Add support so that PTFs can stream their output; Windowing PTF should do this
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20736/ --- (Updated April 25, 2014, 10:55 p.m.) Review request for hive and Ashutosh Chauhan. Changes --- Bugs: HIVE-4965 https://issues.apache.org/jira/browse/HIVE-4965 Repository: hive-git Description --- There is no need to create an output PTF Partition for the last PTF in a chain. For the Windowing PTF this should give a perf. boost; we avoid creating temporary results for each UDAF; avoid populating an output Partition. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/PTFOperator.java af25dc8 ql/src/java/org/apache/hadoop/hive/ql/parse/PTFTranslator.java ac052cd ql/src/java/org/apache/hadoop/hive/ql/plan/PTFDeserializer.java 154f29a ql/src/java/org/apache/hadoop/hive/ql/udf/ptf/TableFunctionEvaluator.java 080fd44 ql/src/java/org/apache/hadoop/hive/ql/udf/ptf/WindowingTableFunction.java 110ef27 Diff: https://reviews.apache.org/r/20736/diff/ Testing --- ran existing tests Thanks, Harish Butani
Review Request 20736: HIVE-4965 Add support so that PTFs can stream their output; Windowing PTF should do this
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20736/ --- Review request for hive and Ashutosh Chauhan. Bugs: HIVE-4965 https://issues.apache.org/jira/browse/HIVE-4965 Repository: hive-git Description --- There is no need to create an output PTF Partition for the last PTF in a chain. For the Windowing PTF this should give a perf. boost; we avoid creating temporary results for each UDAF; avoid populating an output Partition. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/PTFOperator.java af25dc8 ql/src/java/org/apache/hadoop/hive/ql/parse/PTFTranslator.java ac052cd ql/src/java/org/apache/hadoop/hive/ql/plan/PTFDeserializer.java 154f29a ql/src/java/org/apache/hadoop/hive/ql/udf/ptf/TableFunctionEvaluator.java 080fd44 ql/src/java/org/apache/hadoop/hive/ql/udf/ptf/WindowingTableFunction.java 110ef27 Diff: https://reviews.apache.org/r/20736/diff/ Testing --- ran existing tests Thanks, Harish Butani
Review Request 20737: HIVE-6031: explain subquery rewrite for where clause predicates
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20737/ --- Review request for hive and Ashutosh Chauhan. Bugs: HIVE-6031 https://issues.apache.org/jira/browse/HIVE-6031 Repository: hive-git Description --- explain subquery rewrite for where clause predicates Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainSQRewriteTask.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/TaskFactory.java 679c6ec ql/src/java/org/apache/hadoop/hive/ql/parse/ExplainSQRewriteSemanticAnalyzer.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g 3e673ca ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 13bbf0a ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g 864e692 ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java a8b436e ql/src/java/org/apache/hadoop/hive/ql/parse/QBSubQuery.java b7c9e65 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 3b33dc2 ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java e7d0359 ql/src/java/org/apache/hadoop/hive/ql/parse/SubQueryDiagnostic.java PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/parse/SubQueryUtils.java 07d32ed ql/src/java/org/apache/hadoop/hive/ql/plan/ExplainSQRewriteWork.java PRE-CREATION ql/src/test/queries/clientpositive/subquery_exists_explain_rewrite.q PRE-CREATION ql/src/test/queries/clientpositive/subquery_in_explain_rewrite.q PRE-CREATION ql/src/test/results/clientpositive/subquery_exists_explain_rewrite.q.out PRE-CREATION ql/src/test/results/clientpositive/subquery_in_explain_rewrite.q.out PRE-CREATION Diff: https://reviews.apache.org/r/20737/diff/ Testing --- new tests added Thanks, Harish Butani
Re: [ANNOUNCE] New Hive Committers - Prasanth J and Vaibhav Gumashta
Congratulations folks! +Vinod On Thu, Apr 24, 2014 at 7:26 PM, Carl Steinbach c...@apache.org wrote: The Apache Hive PMC has voted to make Prasanth J and Vaibhav Gumashta committers on the Apache Hive Project. Please join me in congratulating Prasanth and Vaibhav! Thanks. - Carl -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: Apache Hive 0.13.1
I've created the following wiki link : https://cwiki.apache.org/confluence/display/Hive/Hive+0.13.1+Release+tracking People should be able to request additional jiras by adding it to the list. I think it might make sense to halt addition of requests to the list 3 days before the RC is cut, so as to prevent an endless-tail scenario, unless the bug in question is a breaking severe issue, where, yes, after discussion, we can vote to add it to the list. That also gives us time to run a full suite of tests on a stable build before we cut the RC. I propose that the first RC (RC0) be built on Monday May 5th at 6pm PDT, and the jira list on the wiki be closed to open/easy additions at 6pm PDT on Friday May 2nd. On Fri, Apr 25, 2014 at 2:40 PM, Gunther Hagleitner ghagleit...@hortonworks.com wrote: Sorry - HIVE-6824 isn't needed. Just the other 3. My bad. Thanks, Gunther. On Fri, Apr 25, 2014 at 2:10 PM, Gunther Hagleitner ghagleit...@hortonworks.com wrote: I'd like to request to include these Tez fixes: HIVE-6824, HIVE-6826, HIVE-6828, HIVE-6898 Thanks, Gunther. On Fri, Apr 25, 2014 at 11:59 AM, Sushanth Sowmyan khorg...@gmail.comwrote: True, I was counting two weeks from today, but 0.13 has already been out for a week. I'm amenable to having an RC1 out on May 5th. If any further issues appear that block, then we can deal with them in an RC2/etc modification to that. On Fri, Apr 25, 2014 at 11:45 AM, Thejas Nair the...@hortonworks.com wrote: On Fri, Apr 25, 2014 at 11:33 AM, Sushanth Sowmyan khorg...@gmail.com wrote: I think it's important to get a bugfix/stabilization release reasonably quickly, but it's also important to give people a little time to try out 0.13, discover/report bugs and fix them. So I think about two weeks is a good point? And instead of releasing an RC on a friday, I'm thinking of pushing it out to Monday - does 12th May sound good to everyone? I think we can aim for an earlier date. Most of these issues seem to be already committed to trunk or have patches available. So the remaining ones also might get committed to trunk by early next week. How about shooting for May 5th (Monday) ? By then 0.13 would also have been out for 2 weeks. If we have any new critical bug reported that needs a fix, we can hold off on the RC for few days. What do you think ? Thanks, Thejas -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
[jira] [Commented] (HIVE-6900) HostUtil.getTaskLogUrl signature change causes compilation to fail
[ https://issues.apache.org/jira/browse/HIVE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981794#comment-13981794 ] Jason Dere commented on HIVE-6900: -- Would like to get this fixed sooner than later, as I'd like for Hive to be able to use 2.4.0. Using Hadoop 2.4.0 in the Hive build will fix a number of unit test failures that we've been seeing. I'm thinking to remove use of the HostUtil call since based on Vinod's comment it sounds like the URL we're generating isn't supposed to work anymore. When MAPREDUCE-5857 is fixed we can add this functionality back to Hadoop23Shims. HostUtil.getTaskLogUrl signature change causes compilation to fail -- Key: HIVE-6900 URL: https://issues.apache.org/jira/browse/HIVE-6900 Project: Hive Issue Type: Bug Components: Shims Affects Versions: 0.13.0, 0.14.0 Reporter: Chris Drome Attachments: HIVE-6900.1.patch.txt The signature for HostUtil.getTaskLogUrl has changed between Hadoop-2.3 and Hadoop-2.4. Code in shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java works with Hadoop-2.3 method and causes compilation failure with Hadoop-2.4. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6031) explain subquery rewrite for where clause predicates
[ https://issues.apache.org/jira/browse/HIVE-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Butani updated HIVE-6031: Status: Open (was: Patch Available) explain subquery rewrite for where clause predicates - Key: HIVE-6031 URL: https://issues.apache.org/jira/browse/HIVE-6031 Project: Hive Issue Type: Sub-task Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-6031.1.patch, HIVE-6031.2.patch, HIVE-6031.3.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6031) explain subquery rewrite for where clause predicates
[ https://issues.apache.org/jira/browse/HIVE-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Butani updated HIVE-6031: Status: Patch Available (was: Open) explain subquery rewrite for where clause predicates - Key: HIVE-6031 URL: https://issues.apache.org/jira/browse/HIVE-6031 Project: Hive Issue Type: Sub-task Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-6031.1.patch, HIVE-6031.2.patch, HIVE-6031.3.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6031) explain subquery rewrite for where clause predicates
[ https://issues.apache.org/jira/browse/HIVE-6031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Harish Butani updated HIVE-6031: Attachment: HIVE-6031.3.patch fix .q.out files for hadoop 2 explain subquery rewrite for where clause predicates - Key: HIVE-6031 URL: https://issues.apache.org/jira/browse/HIVE-6031 Project: Hive Issue Type: Sub-task Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-6031.1.patch, HIVE-6031.2.patch, HIVE-6031.3.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
Review Request 20738: HIVE-2584: Alter table should accept database name
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20738/ --- Review request for hive. Bugs: HIVE-2584 https://issues.apache.org/jira/browse/HIVE-2584 Repository: hive-git Description --- It would be nice if alter table accepts database name. For example: This would be more useful in certain usecases: alter table DB.Tbl set location location; rather than 2 statements. use DB; alter table Tbl set location location; Diffs - metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 8345d70 ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java ec68e7c ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 947b65c ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 448dae2 ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java 0f60fcb ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g b34f53b ql/src/test/queries/clientnegative/alter_table_rename.q PRE-CREATION ql/src/test/queries/clientpositive/alter6.q PRE-CREATION ql/src/test/results/clientnegative/alter_partition_coltype_2columns.q.out e1f9a27 ql/src/test/results/clientnegative/alter_table_rename.q.out PRE-CREATION ql/src/test/results/clientnegative/archive_partspec3.q.out c85e9a2 ql/src/test/results/clientpositive/alter6.q.out PRE-CREATION ql/src/test/results/clientpositive/drop_multi_partitions.q.out 31cd197 ql/src/test/results/clientpositive/input3.q.out 58231b1 ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out fcc551e ql/src/test/results/clientpositive/show_create_table_db_table.q.out d36e8b0 Diff: https://reviews.apache.org/r/20738/diff/ Testing --- Thanks, Harish Butani
[jira] [Created] (HIVE-6979) Hadoop-2 test failures related to quick stats not being populated correctly
Prasanth J created HIVE-6979:
Summary: Hadoop-2 test failures related to quick stats not being
populated correctly
Key: HIVE-6979
URL: https://issues.apache.org/jira/browse/HIVE-6979
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Reporter: Prasanth J
Assignee: Prasanth J
The test failures that are currently reported by Hive QA running on hadoop-2
(https://issues.apache.org/jira/browse/HIVE-6968?focusedCommentId=13980570page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13980570)
are related to difference in the way hadoop FileSystem.globStatus() api
behaves. For a directory structure like below
{code}
dir1/file1
dir1/file2
{code}
Two level of path pattern like dir1/*/* will return both files in hadoop 1.x
but will return empty result in hadoop 2.x (in fact it will say no such file or
directory and return empty file status array). Hadoop 2.x seems to be compliant
to linux behaviour (ls dir1/*/*) but hadoop 1.x is not.
As a result of this, the fast statistics (NUM_FILES and TOTAL_SIZE) are
populated wrongly causing diffs in qfile tests for hadoop-1 and hadoop-2.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6900) HostUtil.getTaskLogUrl signature change causes compilation to fail
[ https://issues.apache.org/jira/browse/HIVE-6900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-6900: - Attachment: HIVE-6900.2.patch Patch v2 simply removes use of the HostUtil API. If folks would still prefer to call into that API, we can use Navis' patch. HostUtil.getTaskLogUrl signature change causes compilation to fail -- Key: HIVE-6900 URL: https://issues.apache.org/jira/browse/HIVE-6900 Project: Hive Issue Type: Bug Components: Shims Affects Versions: 0.13.0, 0.14.0 Reporter: Chris Drome Attachments: HIVE-6900.1.patch.txt, HIVE-6900.2.patch The signature for HostUtil.getTaskLogUrl has changed between Hadoop-2.3 and Hadoop-2.4. Code in shims/0.23/src/main/java/org/apache/hadoop/hive/shims/Hadoop23Shims.java works with Hadoop-2.3 method and causes compilation failure with Hadoop-2.4. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6979) Hadoop-2 test failures related to quick stats not being populated correctly
[
https://issues.apache.org/jira/browse/HIVE-6979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prasanth J updated HIVE-6979:
-
Attachment: HIVE-6979.1.patch
Hadoop-2 test failures related to quick stats not being populated correctly
---
Key: HIVE-6979
URL: https://issues.apache.org/jira/browse/HIVE-6979
Project: Hive
Issue Type: Bug
Affects Versions: 0.14.0
Reporter: Prasanth J
Assignee: Prasanth J
Attachments: HIVE-6979.1.patch
The test failures that are currently reported by Hive QA running on hadoop-2
(https://issues.apache.org/jira/browse/HIVE-6968?focusedCommentId=13980570page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13980570)
are related to difference in the way hadoop FileSystem.globStatus() api
behaves. For a directory structure like below
{code}
dir1/file1
dir1/file2
{code}
Two level of path pattern like dir1/*/* will return both files in hadoop 1.x
but will return empty result in hadoop 2.x (in fact it will say no such file
or directory and return empty file status array). Hadoop 2.x seems to be
compliant to linux behaviour (ls dir1/*/*) but hadoop 1.x is not.
As a result of this, the fast statistics (NUM_FILES and TOTAL_SIZE) are
populated wrongly causing diffs in qfile tests for hadoop-1 and hadoop-2.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
