[jira] [Commented] (HIVE-6430) MapJoin hash table has large memory overhead
[ https://issues.apache.org/jira/browse/HIVE-6430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987458#comment-13987458 ] Lefty Leverenz commented on HIVE-6430: -- Thanks for the parameter descriptions in hive-default.xml.template. But patch 12 has a duplicate description for hive.mapjoin.optimized.hashtable. MapJoin hash table has large memory overhead Key: HIVE-6430 URL: https://issues.apache.org/jira/browse/HIVE-6430 Project: Hive Issue Type: Improvement Reporter: Sergey Shelukhin Assignee: Sergey Shelukhin Attachments: HIVE-6430.01.patch, HIVE-6430.02.patch, HIVE-6430.03.patch, HIVE-6430.04.patch, HIVE-6430.05.patch, HIVE-6430.06.patch, HIVE-6430.07.patch, HIVE-6430.08.patch, HIVE-6430.09.patch, HIVE-6430.10.patch, HIVE-6430.11.patch, HIVE-6430.12.patch, HIVE-6430.12.patch, HIVE-6430.patch Right now, in some queries, I see that storing e.g. 4 ints (2 for key and 2 for row) can take several hundred bytes, which is ridiculous. I am reducing the size of MJKey and MJRowContainer in other jiras, but in general we don't need to have java hash table there. We can either use primitive-friendly hashtable like the one from HPPC (Apache-licenced), or some variation, to map primitive keys to single row storage structure without an object per row (similar to vectorization). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6828) Hive tez bucket map join conversion interferes with map join conversion
[
https://issues.apache.org/jira/browse/HIVE-6828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987519#comment-13987519
]
Hive QA commented on HIVE-6828:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12642970/HIVE-6828.2.patch
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 5427 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/101/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/101/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12642970
Hive tez bucket map join conversion interferes with map join conversion
---
Key: HIVE-6828
URL: https://issues.apache.org/jira/browse/HIVE-6828
Project: Hive
Issue Type: Bug
Components: Tez
Affects Versions: 0.13.0, 0.14.0
Reporter: Vikram Dixit K
Assignee: Vikram Dixit K
Attachments: HIVE-6828.1.patch, HIVE-6828.2.patch
The issue is that bucket count is used for checking the scaled down size of
the hash tables but is used later on to convert to the map join as well which
may be incorrect in cases where the entire hash table does not fit in the
specified size.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-5092) Fix hiveserver2 mapreduce local job on Windows
[
https://issues.apache.org/jira/browse/HIVE-5092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987650#comment-13987650
]
Hive QA commented on HIVE-5092:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12642884/HIVE-5092.4.patch
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 5428 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/102/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/102/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12642884
Fix hiveserver2 mapreduce local job on Windows
--
Key: HIVE-5092
URL: https://issues.apache.org/jira/browse/HIVE-5092
Project: Hive
Issue Type: Bug
Components: HiveServer2, Windows
Affects Versions: 0.14.0
Reporter: Daniel Dai
Assignee: Hari Sankar Sivarama Subramaniyan
Fix For: 0.14.0
Attachments: HIVE-5092-1.patch, HIVE-5092.2.patch, HIVE-5092.3.patch,
HIVE-5092.4.patch
Hiveserver2 fail on Mapreduce local job fail. For example:
{code}
select /*+ MAPJOIN(v) */ registration from studenttab10k s join votertab10k v
on (s.name = v.name);
{code}
The root cause is class not found in the local hadoop job
(MapredLocalTask.execute). HADOOP_CLASSPATH does not include $HIVE_HOME/lib.
Set HADOOP_CLASSPATH correctly will fix the issue.
However, there is one complexity in Windows. We start Hiveserver2 using
Windows service console (services.msc), which takes hiveserver2.xml generated
by hive.cmd. There is no way to pass environment variable in hiveserver2.xml
(weird but reality). I attach a patch which pass it through command line
arguments and relay to HADOOP_CLASSPATH in Hive code.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6995) GenericUDFBridge should log exception when it is unable to instantiate UDF object
[ https://issues.apache.org/jira/browse/HIVE-6995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6995: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Jason! GenericUDFBridge should log exception when it is unable to instantiate UDF object - Key: HIVE-6995 URL: https://issues.apache.org/jira/browse/HIVE-6995 Project: Hive Issue Type: Bug Components: UDF Reporter: Jason Dere Assignee: Jason Dere Fix For: 0.14.0 Attachments: HIVE-6995.1.patch If GenericUDFBridge.initialize() is unable to create an instance of the UDF class, it logs a blanket error message of The UDF implementation class udf is not present in the class path. This can make it difficult to determine what the actual error is as there can be other issues besides the class not being in the class path. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6868) Create table in HCatalog sets different SerDe defaults than what is set through the CLI
[ https://issues.apache.org/jira/browse/HIVE-6868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6868: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks, Harish! Create table in HCatalog sets different SerDe defaults than what is set through the CLI --- Key: HIVE-6868 URL: https://issues.apache.org/jira/browse/HIVE-6868 Project: Hive Issue Type: Bug Components: HCatalog Reporter: Harish Butani Assignee: Harish Butani Fix For: 0.14.0 Attachments: HIVE-6868.1.patch, HIVE-6868.2.patch, HIVE-6868.3.patch HCatCreateTableDesc doesn't invoke the getEmptyTable function on org.apache.hadoop.hive.ql.metadata.Table -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7006) Fix ql_rewrite_gbtoidx.q output file
[ https://issues.apache.org/jira/browse/HIVE-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987828#comment-13987828 ] Ashutosh Chauhan commented on HIVE-7006: +1 Fix ql_rewrite_gbtoidx.q output file Key: HIVE-7006 URL: https://issues.apache.org/jira/browse/HIVE-7006 Project: Hive Issue Type: Bug Reporter: Harish Butani Assignee: Harish Butani Attachments: HIVE-7006.1.patch HIVE-4904 moved GroupBy Optimization to happen before ColumnPruner Now, the RewriteGBUsingIndex happens after GroupByOptimization. So setting of bucketGroup flag doesn't happen for Index tables. Temporary fix is to update .q.out file. Will file a bug to look into moving RewriteGBUsingIndex before GroupByOptimizer. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6960) Set Hive pom to use Hadoop-2.4
[
https://issues.apache.org/jira/browse/HIVE-6960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13987877#comment-13987877
]
Hive QA commented on HIVE-6960:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12642785/HIVE-6960.1.patch
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 5428 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_auto_join32
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_mapjoin_test_outer
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup3
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/103/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/103/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12642785
Set Hive pom to use Hadoop-2.4
--
Key: HIVE-6960
URL: https://issues.apache.org/jira/browse/HIVE-6960
Project: Hive
Issue Type: Bug
Components: Build Infrastructure
Affects Versions: 0.14.0
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-6960.1.patch, HIVE-6960.1.patch
A number of the hadoop-2 unit test failures are due to HADOOP-10425, fixed in
Hadoop 2.4. Perhaps we should move onto that version.
- org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup3
- org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4
- org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source
-
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format
- org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database
-
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6960) Set Hive pom to use Hadoop-2.4
[ https://issues.apache.org/jira/browse/HIVE-6960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-6960: - Status: Open (was: Patch Available) Will take a look at the q file failures, it's possible that these golden files were changed in HIVE-6979 to have stats values that reflected the bug in HADOOP-10425. Set Hive pom to use Hadoop-2.4 -- Key: HIVE-6960 URL: https://issues.apache.org/jira/browse/HIVE-6960 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.14.0 Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6960.1.patch, HIVE-6960.1.patch A number of the hadoop-2 unit test failures are due to HADOOP-10425, fixed in Hadoop 2.4. Perhaps we should move onto that version. - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup3 - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4 - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database - org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-2777) ability to add and drop partitions atomically
[ https://issues.apache.org/jira/browse/HIVE-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988029#comment-13988029 ] Steven Wong commented on HIVE-2777: --- ASF.LICENSE.NOT.GRANTED--HIVE-2777.D2271.1.patch was created by Netflix. We at Netflix intend it to be freely used according to the Apache license. ability to add and drop partitions atomically - Key: HIVE-2777 URL: https://issues.apache.org/jira/browse/HIVE-2777 Project: Hive Issue Type: New Feature Components: Metastore Reporter: Aniket Mokashi Assignee: Aniket Mokashi Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2777.D2271.1.patch Hive should have ability to atomically add and drop partitions. This way admins can change partitions atomically without breaking the running jobs. It allows admin to merge several partitions into one. Essentially, we would like to have an api- add_drop_partitions(String db, String tbl_name, ListPartition addParts, ListListString dropParts, boolean deleteData); This jira covers changes required for metastore and thrift. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-7008) Clean-up some old dead code
Ashutosh Chauhan created HIVE-7008: -- Summary: Clean-up some old dead code Key: HIVE-7008 URL: https://issues.apache.org/jira/browse/HIVE-7008 Project: Hive Issue Type: Task Components: Tests Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan There is some code to workaround limitations in historic Hadoop (hadoop-17 earlier). Lets get rid of those. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7008) Clean-up some old dead code
[ https://issues.apache.org/jira/browse/HIVE-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-7008: --- Attachment: HIVE-7008.patch Clean-up some old dead code --- Key: HIVE-7008 URL: https://issues.apache.org/jira/browse/HIVE-7008 Project: Hive Issue Type: Task Components: Tests Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-7008.patch There is some code to workaround limitations in historic Hadoop (hadoop-17 earlier). Lets get rid of those. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7008) Clean-up some old dead code
[ https://issues.apache.org/jira/browse/HIVE-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-7008: --- Status: Patch Available (was: Open) Clean-up some old dead code --- Key: HIVE-7008 URL: https://issues.apache.org/jira/browse/HIVE-7008 Project: Hive Issue Type: Task Components: Tests Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-7008.patch There is some code to workaround limitations in historic Hadoop (hadoop-17 earlier). Lets get rid of those. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7008) Clean-up some old dead code
[ https://issues.apache.org/jira/browse/HIVE-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988040#comment-13988040 ] Ashutosh Chauhan commented on HIVE-7008: As a first step, mark now-obsolete classes as Deprecated and delete tests. Next release we should delete classes as well. Clean-up some old dead code --- Key: HIVE-7008 URL: https://issues.apache.org/jira/browse/HIVE-7008 Project: Hive Issue Type: Task Components: Tests Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-7008.patch There is some code to workaround limitations in historic Hadoop (hadoop-17 earlier). Lets get rid of those. -- This message was sent by Atlassian JIRA (v6.2#6252)
Review Request 21016: Deprecate class and remove tests.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21016/ --- Review request for hive and Jason Dere. Bugs: HIVE-7008 https://issues.apache.org/jira/browse/HIVE-7008 Repository: hive-git Description --- Marked class as Deprecated. Removed tests for now obsolete functionality. Diffs - ql/src/java/org/apache/hadoop/hive/ql/io/FlatFileInputFormat.java b9b151a ql/src/test/org/apache/hadoop/hive/ql/io/RecordTestObj.java 1bb846b ql/src/test/org/apache/hadoop/hive/ql/io/TestFlatFileInputFormat.java c46d754 Diff: https://reviews.apache.org/r/21016/diff/ Testing --- Thanks, Ashutosh Chauhan
Re: SMB join bug
It is possible that you hit this issue -
https://issues.apache.org/jira/browse/HIVE-5973
It is fixed in apache hive 0.13 release.
On Thu, May 1, 2014 at 7:10 PM, Sukhendu Chakraborty
sukhendu.chakrabo...@gmail.com wrote:
I am seeing very different number of rows in this query output depending on
whether I enable SMB join:
select count(*)
from dss.hist_hshld_profl_mc a
join
dss.hshld_summary_mc b
on a.hh_key = b.hh_key
where ('2012-02-27' between a.hshld_profl_eff_dt and a.hshld_profl_exp_dt)
and a.hshld_exp_dt='-12-31'
and trim(a.cntry_id) = 'USA'
The SMB join returns 60 rows (wrong value) while the regular join returns
30million plus rows (correct value).
Is there a known issue/jira for this? We are using CDH5.0/hive-0.12.
-Sukhendu
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Review Request 21016: Deprecate class and remove tests.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21016/#review42045 --- Also delete JavaTestObjFlatFileInputFormat which is being used by the deleted test. - Jason Dere On May 2, 2014, 6:19 p.m., Ashutosh Chauhan wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21016/ --- (Updated May 2, 2014, 6:19 p.m.) Review request for hive and Jason Dere. Bugs: HIVE-7008 https://issues.apache.org/jira/browse/HIVE-7008 Repository: hive-git Description --- Marked class as Deprecated. Removed tests for now obsolete functionality. Diffs - ql/src/java/org/apache/hadoop/hive/ql/io/FlatFileInputFormat.java b9b151a ql/src/test/org/apache/hadoop/hive/ql/io/RecordTestObj.java 1bb846b ql/src/test/org/apache/hadoop/hive/ql/io/TestFlatFileInputFormat.java c46d754 Diff: https://reviews.apache.org/r/21016/diff/ Testing --- Thanks, Ashutosh Chauhan
[jira] [Commented] (HIVE-7001) fs.permissions.umask-mode is getting unset when Session is started
[
https://issues.apache.org/jira/browse/HIVE-7001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988089#comment-13988089
]
Thejas M Nair commented on HIVE-7001:
-
Ran parquet_ctas locally and it passed, rest of the test failures are unrelated.
fs.permissions.umask-mode is getting unset when Session is started
--
Key: HIVE-7001
URL: https://issues.apache.org/jira/browse/HIVE-7001
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Attachments: HIVE-7001.1.patch, HIVE-7001.2.patch, HIVE-7001.3.patch
{code}
hive set fs.permissions.umask-mode;
fs.permissions.umask-mode=022
hive show tables;
OK
t1
Time taken: 0.301 seconds, Fetched: 1 row(s)
hive set fs.permissions.umask-mode;
fs.permissions.umask-mode is undefined
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7004) Fix more unit test failures on hadoop-2
[
https://issues.apache.org/jira/browse/HIVE-7004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988098#comment-13988098
]
Hive QA commented on HIVE-7004:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12642963/HIVE-7004.1.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 5495 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_load_dyn_part1
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_dml
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_tez_union
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/105/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/105/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12642963
Fix more unit test failures on hadoop-2
---
Key: HIVE-7004
URL: https://issues.apache.org/jira/browse/HIVE-7004
Project: Hive
Issue Type: Bug
Components: Tests
Reporter: Jason Dere
Assignee: Jason Dere
Attachments: HIVE-7004.1.patch
Still a number of precommit failures with hadoop-2, will try to fix some of
them.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6901) Explain plan doesn't show operator tree for the fetch operator
[
https://issues.apache.org/jira/browse/HIVE-6901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xuefu Zhang updated HIVE-6901:
--
Attachment: HIVE-6109.10.patch
Explain plan doesn't show operator tree for the fetch operator
--
Key: HIVE-6901
URL: https://issues.apache.org/jira/browse/HIVE-6901
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.12.0
Reporter: Xuefu Zhang
Assignee: Xuefu Zhang
Priority: Minor
Attachments: HIVE-6109.10.patch, HIVE-6901.1.patch,
HIVE-6901.2.patch, HIVE-6901.3.patch, HIVE-6901.4.patch, HIVE-6901.5.patch,
HIVE-6901.6.patch, HIVE-6901.7.patch, HIVE-6901.8.patch, HIVE-6901.9.patch,
HIVE-6901.patch
Explaining a simple select query that involves a MR phase doesn't show
processor tree for the fetch operator.
{code}
hive explain select d from test;
OK
STAGE DEPENDENCIES:
Stage-1 is a root stage
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-1
Map Reduce
Map Operator Tree:
...
Stage: Stage-0
Fetch Operator
limit: -1
{code}
It would be nice if the operator tree is shown even if there is only one node.
Please note that in local execution, the operator tree is complete:
{code}
hive explain select * from test;
OK
STAGE DEPENDENCIES:
Stage-0 is a root stage
STAGE PLANS:
Stage: Stage-0
Fetch Operator
limit: -1
Processor Tree:
TableScan
alias: test
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE Column
stats: NONE
Select Operator
expressions: d (type: int)
outputColumnNames: _col0
Statistics: Num rows: 8 Data size: 34 Basic stats: COMPLETE
Column stats: NONE
ListSink
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-7001) fs.permissions.umask-mode is getting unset when Session is started
[
https://issues.apache.org/jira/browse/HIVE-7001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejas M Nair updated HIVE-7001:
Resolution: Fixed
Fix Version/s: 0.14.0
Status: Resolved (was: Patch Available)
Patch committed to trunk.
Thanks for the reviews Ashutosh Vikram!
fs.permissions.umask-mode is getting unset when Session is started
--
Key: HIVE-7001
URL: https://issues.apache.org/jira/browse/HIVE-7001
Project: Hive
Issue Type: Bug
Affects Versions: 0.13.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
Fix For: 0.14.0
Attachments: HIVE-7001.1.patch, HIVE-7001.2.patch, HIVE-7001.3.patch
{code}
hive set fs.permissions.umask-mode;
fs.permissions.umask-mode=022
hive show tables;
OK
t1
Time taken: 0.301 seconds, Fetched: 1 row(s)
hive set fs.permissions.umask-mode;
fs.permissions.umask-mode is undefined
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6946) Make it easier to run WebHCat e2e tests
[ https://issues.apache.org/jira/browse/HIVE-6946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-6946: Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Patch committed to trunk. Thanks for the contribution Eugene! Make it easier to run WebHCat e2e tests --- Key: HIVE-6946 URL: https://issues.apache.org/jira/browse/HIVE-6946 Project: Hive Issue Type: Bug Components: WebHCat Affects Versions: 0.14.0 Reporter: Eugene Koifman Assignee: Eugene Koifman Fix For: 0.14.0 Attachments: HIVE-6946.2.patch, HIVE-6946.3.patch, HIVE-6946.patch Right now hcatalog/src/test/e2e/templeton/README.txt explains the steps to set up WebHCat e2e tests but it's cumbersome and error prone. Need to make some improvements here. The high level goal here is to simplify code-compile-test loop for WebHCat. NO PRECOMMIT TESTS -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6960) Set Hive pom to use Hadoop-2.4
[ https://issues.apache.org/jira/browse/HIVE-6960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-6960: - Attachment: HIVE-6960.2.patch patch v2 updates a number of diffs that were changed by HIVE-6979 Set Hive pom to use Hadoop-2.4 -- Key: HIVE-6960 URL: https://issues.apache.org/jira/browse/HIVE-6960 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.14.0 Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6960.1.patch, HIVE-6960.1.patch, HIVE-6960.2.patch A number of the hadoop-2 unit test failures are due to HADOOP-10425, fixed in Hadoop 2.4. Perhaps we should move onto that version. - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup3 - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4 - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database - org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6960) Set Hive pom to use Hadoop-2.4
[ https://issues.apache.org/jira/browse/HIVE-6960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Dere updated HIVE-6960: - Status: Patch Available (was: Open) Set Hive pom to use Hadoop-2.4 -- Key: HIVE-6960 URL: https://issues.apache.org/jira/browse/HIVE-6960 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.14.0 Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6960.1.patch, HIVE-6960.1.patch, HIVE-6960.2.patch A number of the hadoop-2 unit test failures are due to HADOOP-10425, fixed in Hadoop 2.4. Perhaps we should move onto that version. - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup3 - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4 - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database - org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-7009) HIVE_USER_INSTALL_DIR could not bet set to non-HDFS filesystem
Chuan Liu created HIVE-7009:
---
Summary: HIVE_USER_INSTALL_DIR could not bet set to non-HDFS
filesystem
Key: HIVE-7009
URL: https://issues.apache.org/jira/browse/HIVE-7009
Project: Hive
Issue Type: Bug
Components: Tez
Affects Versions: 0.13.0
Reporter: Chuan Liu
In {{hive/ql/exec/tez/DagUtils.java}}, we enforce the user path get from
{{HIVE_USER_INSTALL_DIR}} to be HDFS. This makes it impossible to run Hive+Tez
jobs on non-HDFS filesystem, e.g. WASB. Relevant code are as follows:
{noformat}
public Path getDefaultDestDir(Configuration conf) throws LoginException,
IOException {
UserGroupInformation ugi = ShimLoader.getHadoopShims().getUGIForConf(conf);
String userName = ShimLoader.getHadoopShims().getShortUserName(ugi);
String userPathStr = HiveConf.getVar(conf,
HiveConf.ConfVars.HIVE_USER_INSTALL_DIR);
Path userPath = new Path(userPathStr);
FileSystem fs = userPath.getFileSystem(conf);
if (!(fs instanceof DistributedFileSystem)) {
throw new IOException(ErrorMsg.INVALID_HDFS_URI.format(userPathStr));
}
{noformat}
Exceptions running jobs with defaultFs configured to WASB.
{noformat}
2014-05-01 00:21:39,847 ERROR exec.Task (TezTask.java:execute(192)) - Failed to
execute tez graph.
java.io.IOException:
wasb://hdi31-chuan...@clhdistorage.blob.core.windows.net/user is not a hdfs uri
at
org.apache.hadoop.hive.ql.exec.tez.DagUtils.getDefaultDestDir(DagUtils.java:662)
at
org.apache.hadoop.hive.ql.exec.tez.DagUtils.getHiveJarDirectory(DagUtils.java:759)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.createJarLocalResource(TezSessionState.java:321)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:159)
at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:154)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1504)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1271)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1089)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:912)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HIVE-7010) templeton/v1/queue REST method has been removed
Eugene Koifman created HIVE-7010: Summary: templeton/v1/queue REST method has been removed Key: HIVE-7010 URL: https://issues.apache.org/jira/browse/HIVE-7010 Project: Hive Issue Type: Bug Components: Documentation, WebHCat Affects Versions: 0.14.0 Reporter: Eugene Koifman deprecated queue REST method was removed from WebHCat in HIVE-6432. jobs is the replacement. https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference needs to be updated -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7004) Fix more unit test failures on hadoop-2
[ https://issues.apache.org/jira/browse/HIVE-7004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988250#comment-13988250 ] Ashutosh Chauhan commented on HIVE-7004: +1 Fix more unit test failures on hadoop-2 --- Key: HIVE-7004 URL: https://issues.apache.org/jira/browse/HIVE-7004 Project: Hive Issue Type: Bug Components: Tests Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-7004.1.patch Still a number of precommit failures with hadoop-2, will try to fix some of them. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6960) Set Hive pom to use Hadoop-2.4
[ https://issues.apache.org/jira/browse/HIVE-6960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988256#comment-13988256 ] Ashutosh Chauhan commented on HIVE-6960: +1 Set Hive pom to use Hadoop-2.4 -- Key: HIVE-6960 URL: https://issues.apache.org/jira/browse/HIVE-6960 Project: Hive Issue Type: Bug Components: Build Infrastructure Affects Versions: 0.14.0 Reporter: Jason Dere Assignee: Jason Dere Attachments: HIVE-6960.1.patch, HIVE-6960.1.patch, HIVE-6960.2.patch A number of the hadoop-2 unit test failures are due to HADOOP-10425, fixed in Hadoop 2.4. Perhaps we should move onto that version. - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_nullgroup3 - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ppd_join4 - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_select_dummy_source - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_symlink_text_input_format - org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_udf_current_database - org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketizedhiveinputformat -- This message was sent by Atlassian JIRA (v6.2#6252)
Review Request 21026: Remove HDFS check in tez/DagUtils
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21026/ --- Review request for hive. Bugs: HIVE-7009 https://issues.apache.org/jira/browse/HIVE-7009 Repository: hive-git Description --- Removed all the DFS checks in the change. They should not impact any functionality. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java abbf38f Diff: https://reviews.apache.org/r/21026/diff/ Testing --- Manually tested on a cluster deployed in Windows Azure environment. Thanks, Chuan Liu
[jira] [Updated] (HIVE-7009) HIVE_USER_INSTALL_DIR could not bet set to non-HDFS filesystem
[
https://issues.apache.org/jira/browse/HIVE-7009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chuan Liu updated HIVE-7009:
Attachment: HIVE-7009.patch
Attach a patch that remove HDFS checks in tez/DagUtils.java.
HIVE_USER_INSTALL_DIR could not bet set to non-HDFS filesystem
--
Key: HIVE-7009
URL: https://issues.apache.org/jira/browse/HIVE-7009
Project: Hive
Issue Type: Bug
Components: Tez
Affects Versions: 0.13.0
Reporter: Chuan Liu
Attachments: HIVE-7009.patch
In {{hive/ql/exec/tez/DagUtils.java}}, we enforce the user path get from
{{HIVE_USER_INSTALL_DIR}} to be HDFS. This makes it impossible to run
Hive+Tez jobs on non-HDFS filesystem, e.g. WASB. Relevant code are as follows:
{noformat}
public Path getDefaultDestDir(Configuration conf) throws LoginException,
IOException {
UserGroupInformation ugi =
ShimLoader.getHadoopShims().getUGIForConf(conf);
String userName = ShimLoader.getHadoopShims().getShortUserName(ugi);
String userPathStr = HiveConf.getVar(conf,
HiveConf.ConfVars.HIVE_USER_INSTALL_DIR);
Path userPath = new Path(userPathStr);
FileSystem fs = userPath.getFileSystem(conf);
if (!(fs instanceof DistributedFileSystem)) {
throw new IOException(ErrorMsg.INVALID_HDFS_URI.format(userPathStr));
}
{noformat}
Exceptions running jobs with defaultFs configured to WASB.
{noformat}
2014-05-01 00:21:39,847 ERROR exec.Task (TezTask.java:execute(192)) - Failed
to execute tez graph.
java.io.IOException:
wasb://hdi31-chuan...@clhdistorage.blob.core.windows.net/user is not a hdfs
uri
at
org.apache.hadoop.hive.ql.exec.tez.DagUtils.getDefaultDestDir(DagUtils.java:662)
at
org.apache.hadoop.hive.ql.exec.tez.DagUtils.getHiveJarDirectory(DagUtils.java:759)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.createJarLocalResource(TezSessionState.java:321)
at
org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:159)
at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:154)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1504)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1271)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1089)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:912)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:268)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:220)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:423)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:793)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:686)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6684) Beeline does not accept comments that are preceded by spaces
[ https://issues.apache.org/jira/browse/HIVE-6684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Beard updated HIVE-6684: --- Attachment: HIVE-6684.1.patch Added patch with unit test - please review. Beeline does not accept comments that are preceded by spaces Key: HIVE-6684 URL: https://issues.apache.org/jira/browse/HIVE-6684 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.10.0 Reporter: Jeremy Beard Attachments: HIVE-6684.1.patch Beeline throws an error if single-line comments are indented with spaces. This works in the embedded Hive CLI. For example: SELECT -- this is the field we want field FROM table; Error: Error while processing statement: FAILED: ParseException line 1:71 cannot recognize input near 'EOF' 'EOF' 'EOF' in select clause (state=42000,code=4) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6684) Beeline does not accept comments that are preceded by spaces
[ https://issues.apache.org/jira/browse/HIVE-6684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Beard updated HIVE-6684: --- Fix Version/s: 0.14.0 Status: Patch Available (was: Open) Beeline does not accept comments that are preceded by spaces Key: HIVE-6684 URL: https://issues.apache.org/jira/browse/HIVE-6684 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.10.0 Reporter: Jeremy Beard Fix For: 0.14.0 Attachments: HIVE-6684.1.patch Beeline throws an error if single-line comments are indented with spaces. This works in the embedded Hive CLI. For example: SELECT -- this is the field we want field FROM table; Error: Error while processing statement: FAILED: ParseException line 1:71 cannot recognize input near 'EOF' 'EOF' 'EOF' in select clause (state=42000,code=4) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6915) Hive Hbase queries fail on secure Tez cluster
[
https://issues.apache.org/jira/browse/HIVE-6915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988328#comment-13988328
]
Craig Condit commented on HIVE-6915:
Patch applied here, get a different error now:
{noformat}
Vertex failed, vertexName=Map 1, vertexId=vertex_1392942637536_9375_1_00,
diagnostics=[Vertex Input: hosting_scheduled_jobs initializer failed.,
org.apache.hadoop.hbase.security.AccessDeniedException: Token generation only
allowed for Kerberos authenticated clients
at
org.apache.hadoop.hbase.security.token.TokenProvider.getAuthenticationToken(TokenProvider.java:122)
at
org.apache.hadoop.hbase.protobuf.generated.AuthenticationProtos$AuthenticationService$1.getAuthenticationToken(AuthenticationProtos.java:4267)
at
org.apache.hadoop.hbase.protobuf.generated.AuthenticationProtos$AuthenticationService.callMethod(AuthenticationProtos.java:4387)
at
org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:5088)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3197)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26933)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
at
org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
]
14/05/02 15:06:21 ERROR tez.TezJobMonitor: Vertex failed, vertexName=Map 1,
vertexId=vertex_1392942637536_9375_1_00, diagnostics=[Vertex Input:
hosting_scheduled_jobs initializer failed.,
org.apache.hadoop.hbase.security.AccessDeniedException: Token generation only
allowed for Kerberos authenticated clients
at
org.apache.hadoop.hbase.security.token.TokenProvider.getAuthenticationToken(TokenProvider.java:122)
at
org.apache.hadoop.hbase.protobuf.generated.AuthenticationProtos$AuthenticationService$1.getAuthenticationToken(AuthenticationProtos.java:4267)
at
org.apache.hadoop.hbase.protobuf.generated.AuthenticationProtos$AuthenticationService.callMethod(AuthenticationProtos.java:4387)
at
org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:5088)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3197)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26933)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
at
org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
]
DAG failed due to vertex failure. failedVertices:1 killedVertices:0
14/05/02 15:06:21 ERROR tez.TezJobMonitor: DAG failed due to vertex failure.
failedVertices:1 killedVertices:0
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.tez.TezTask
14/05/02 15:06:21 ERROR ql.Driver: FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.tez.TezTask
{noformat}
Hive Hbase queries fail on secure Tez cluster
-
Key: HIVE-6915
URL: https://issues.apache.org/jira/browse/HIVE-6915
Project: Hive
Issue Type: Bug
Components: Tez
Affects Versions: 0.13.0
Environment: Kerberos secure Tez cluster
Reporter: Deepesh Khandelwal
Assignee: Siddharth Seth
Attachments: HIVE-6915.1.patch
Hive queries reading and writing to HBase are currently failing with the
following exception in a secure Tez cluster:
{noformat}
2014-04-14 13:47:05,644 FATAL [InputInitializer [Map 1] #0]
org.apache.hadoop.ipc.RpcClient: SASL authentication failed. The most likely
cause is missing or invalid credentials. Consider 'kinit'.
javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Failed to find
any Kerberos tgt)]
at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at
org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:152)
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupSaslConnection(RpcClient.java:792)
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection.access$800(RpcClient.java:349)
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection$2.run(RpcClient.java:918)
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection$2.run(RpcClient.java:915)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupIOstreams(RpcClient.java:915)
at
[jira] [Updated] (HIVE-6684) Beeline does not accept comments that are preceded by spaces
[ https://issues.apache.org/jira/browse/HIVE-6684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jeremy Beard updated HIVE-6684: --- Fix Version/s: (was: 0.14.0) Beeline does not accept comments that are preceded by spaces Key: HIVE-6684 URL: https://issues.apache.org/jira/browse/HIVE-6684 Project: Hive Issue Type: Bug Components: CLI Affects Versions: 0.10.0 Reporter: Jeremy Beard Attachments: HIVE-6684.1.patch Beeline throws an error if single-line comments are indented with spaces. This works in the embedded Hive CLI. For example: SELECT -- this is the field we want field FROM table; Error: Error while processing statement: FAILED: ParseException line 1:71 cannot recognize input near 'EOF' 'EOF' 'EOF' in select clause (state=42000,code=4) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7006) Fix ql_rewrite_gbtoidx.q output file
[
https://issues.apache.org/jira/browse/HIVE-7006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988368#comment-13988368
]
Hive QA commented on HIVE-7006:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12642980/HIVE-7006.1.patch
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 5429 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/107/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/107/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12642980
Fix ql_rewrite_gbtoidx.q output file
Key: HIVE-7006
URL: https://issues.apache.org/jira/browse/HIVE-7006
Project: Hive
Issue Type: Bug
Reporter: Harish Butani
Assignee: Harish Butani
Attachments: HIVE-7006.1.patch
HIVE-4904 moved GroupBy Optimization to happen before ColumnPruner
Now, the RewriteGBUsingIndex happens after GroupByOptimization.
So setting of bucketGroup flag doesn't happen for Index tables.
Temporary fix is to update .q.out file.
Will file a bug to look into moving RewriteGBUsingIndex before
GroupByOptimizer.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6828) Hive tez bucket map join conversion interferes with map join conversion
[ https://issues.apache.org/jira/browse/HIVE-6828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988376#comment-13988376 ] Gunther Hagleitner commented on HIVE-6828: -- No new test failures (build 100 had the same errors - plus one more). Hive tez bucket map join conversion interferes with map join conversion --- Key: HIVE-6828 URL: https://issues.apache.org/jira/browse/HIVE-6828 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.13.0, 0.14.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: HIVE-6828.1.patch, HIVE-6828.2.patch The issue is that bucket count is used for checking the scaled down size of the hash tables but is used later on to convert to the map join as well which may be incorrect in cases where the entire hash table does not fit in the specified size. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-2777) ability to add and drop partitions atomically
[ https://issues.apache.org/jira/browse/HIVE-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinyu Wang updated HIVE-2777: - Affects Version/s: 0.13.0 Status: Patch Available (was: Open) This is a rebased patch on top of hive branch-0.13. Please review. ability to add and drop partitions atomically - Key: HIVE-2777 URL: https://issues.apache.org/jira/browse/HIVE-2777 Project: Hive Issue Type: New Feature Components: Metastore Affects Versions: 0.13.0 Reporter: Aniket Mokashi Assignee: Aniket Mokashi Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2777.D2271.1.patch Hive should have ability to atomically add and drop partitions. This way admins can change partitions atomically without breaking the running jobs. It allows admin to merge several partitions into one. Essentially, we would like to have an api- add_drop_partitions(String db, String tbl_name, ListPartition addParts, ListListString dropParts, boolean deleteData); This jira covers changes required for metastore and thrift. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6996) FS based stats broken with indexed tables
[ https://issues.apache.org/jira/browse/HIVE-6996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6996: --- Status: Open (was: Patch Available) FS based stats broken with indexed tables - Key: HIVE-6996 URL: https://issues.apache.org/jira/browse/HIVE-6996 Project: Hive Issue Type: Bug Components: Indexing, Statistics Affects Versions: 0.13.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-6996.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-2777) ability to add and drop partitions atomically
[ https://issues.apache.org/jira/browse/HIVE-2777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinyu Wang updated HIVE-2777: - Attachment: hive-2777.patch ability to add and drop partitions atomically - Key: HIVE-2777 URL: https://issues.apache.org/jira/browse/HIVE-2777 Project: Hive Issue Type: New Feature Components: Metastore Affects Versions: 0.13.0 Reporter: Aniket Mokashi Assignee: Aniket Mokashi Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2777.D2271.1.patch, hive-2777.patch Hive should have ability to atomically add and drop partitions. This way admins can change partitions atomically without breaking the running jobs. It allows admin to merge several partitions into one. Essentially, we would like to have an api- add_drop_partitions(String db, String tbl_name, ListPartition addParts, ListListString dropParts, boolean deleteData); This jira covers changes required for metastore and thrift. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6996) FS based stats broken with indexed tables
[ https://issues.apache.org/jira/browse/HIVE-6996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6996: --- Attachment: HIVE-6996.2.patch Fixed failing tests. FS based stats broken with indexed tables - Key: HIVE-6996 URL: https://issues.apache.org/jira/browse/HIVE-6996 Project: Hive Issue Type: Bug Components: Indexing, Statistics Affects Versions: 0.13.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-6996.2.patch, HIVE-6996.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6996) FS based stats broken with indexed tables
[ https://issues.apache.org/jira/browse/HIVE-6996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6996: --- Status: Patch Available (was: Open) FS based stats broken with indexed tables - Key: HIVE-6996 URL: https://issues.apache.org/jira/browse/HIVE-6996 Project: Hive Issue Type: Bug Components: Indexing, Statistics Affects Versions: 0.13.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-6996.2.patch, HIVE-6996.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6828) Hive tez bucket map join conversion interferes with map join conversion
[ https://issues.apache.org/jira/browse/HIVE-6828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-6828: - Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. Thanks Vikram! Hive tez bucket map join conversion interferes with map join conversion --- Key: HIVE-6828 URL: https://issues.apache.org/jira/browse/HIVE-6828 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.13.0, 0.14.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Fix For: 0.14.0 Attachments: HIVE-6828.1.patch, HIVE-6828.2.patch The issue is that bucket count is used for checking the scaled down size of the hash tables but is used later on to convert to the map join as well which may be incorrect in cases where the entire hash table does not fit in the specified size. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6915) Hive Hbase queries fail on secure Tez cluster
[
https://issues.apache.org/jira/browse/HIVE-6915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988400#comment-13988400
]
Craig Condit commented on HIVE-6915:
Better stack trace:
{noformat}
org.apache.hadoop.hbase.security.AccessDeniedException:
org.apache.hadoop.hbase.security.AccessDeniedException: Token generation only
allowed for Kerberos authenticated clients
at
org.apache.hadoop.hbase.security.token.TokenProvider.getAuthenticationToken(TokenProvider.java:122)
at
org.apache.hadoop.hbase.protobuf.generated.AuthenticationProtos$AuthenticationService$1.getAuthenticationToken(AuthenticationProtos.java:4267)
at
org.apache.hadoop.hbase.protobuf.generated.AuthenticationProtos$AuthenticationService.callMethod(AuthenticationProtos.java:4387)
at
org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:5088)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3197)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26933)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2146)
at
org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1851)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at
org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:235)
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1348)
at
org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:87)
at
org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:84)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:116)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:94)
at
org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:90)
at
org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callBlockingMethod(CoprocessorRpcChannel.java:67)
at
org.apache.hadoop.hbase.protobuf.generated.AuthenticationProtos$AuthenticationService$BlockingStub.getAuthenticationToken(AuthenticationProtos.java:4512)
at
org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:60)
at
org.apache.hadoop.hbase.security.token.TokenUtil$3.run(TokenUtil.java:174)
at
org.apache.hadoop.hbase.security.token.TokenUtil$3.run(TokenUtil.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at
org.apache.hadoop.hbase.security.token.TokenUtil.obtainTokenForJob(TokenUtil.java:171)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.hbase.util.Methods.call(Methods.java:39)
at
org.apache.hadoop.hbase.security.User$SecureHadoopUser.obtainAuthTokenForJob(User.java:314)
at
org.apache.hadoop.hbase.mapred.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:181)
at
org.apache.hadoop.hive.hbase.HiveHBaseTableInputFormat.getSplits(HiveHBaseTableInputFormat.java:416)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.addSplitsForGroup(HiveInputFormat.java:291)
at
org.apache.hadoop.hive.ql.io.HiveInputFormat.getSplits(HiveInputFormat.java:372)
at
org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat.getSplits(TezGroupedSplitsInputFormat.java:68)
at
org.apache.tez.mapreduce.hadoop.MRHelpers.generateOldSplits(MRHelpers.java:263)
at
org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:139)
at
org.apache.tez.dag.app.dag.RootInputInitializerRunner$InputInitializerCallable$1.run(RootInputInitializerRunner.java:146)
at
Re: Review Request 21016: Deprecate class and remove tests.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21016/ --- (Updated May 2, 2014, 11:24 p.m.) Review request for hive and Jason Dere. Changes --- Incorporated Jason feedback to delete another test file. Bugs: HIVE-7008 https://issues.apache.org/jira/browse/HIVE-7008 Repository: hive-git Description --- Marked class as Deprecated. Removed tests for now obsolete functionality. Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/io/FlatFileInputFormat.java b9b151a ql/src/test/org/apache/hadoop/hive/ql/io/JavaTestObjFlatFileInputFormat.java 830ecef ql/src/test/org/apache/hadoop/hive/ql/io/RecordTestObj.java 1bb846b ql/src/test/org/apache/hadoop/hive/ql/io/TestFlatFileInputFormat.java c46d754 Diff: https://reviews.apache.org/r/21016/diff/ Testing --- Thanks, Ashutosh Chauhan
[jira] [Updated] (HIVE-7008) Clean-up some old dead code
[ https://issues.apache.org/jira/browse/HIVE-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-7008: --- Status: Open (was: Patch Available) Clean-up some old dead code --- Key: HIVE-7008 URL: https://issues.apache.org/jira/browse/HIVE-7008 Project: Hive Issue Type: Task Components: Tests Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-7008.1.patch, HIVE-7008.patch There is some code to workaround limitations in historic Hadoop (hadoop-17 earlier). Lets get rid of those. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7008) Clean-up some old dead code
[ https://issues.apache.org/jira/browse/HIVE-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-7008: --- Attachment: HIVE-7008.1.patch Incorporated Jason's RB feedback. Clean-up some old dead code --- Key: HIVE-7008 URL: https://issues.apache.org/jira/browse/HIVE-7008 Project: Hive Issue Type: Task Components: Tests Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-7008.1.patch, HIVE-7008.patch There is some code to workaround limitations in historic Hadoop (hadoop-17 earlier). Lets get rid of those. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7008) Clean-up some old dead code
[ https://issues.apache.org/jira/browse/HIVE-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-7008: --- Status: Patch Available (was: Open) Clean-up some old dead code --- Key: HIVE-7008 URL: https://issues.apache.org/jira/browse/HIVE-7008 Project: Hive Issue Type: Task Components: Tests Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-7008.1.patch, HIVE-7008.patch There is some code to workaround limitations in historic Hadoop (hadoop-17 earlier). Lets get rid of those. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5823) Support for DECIMAL primitive type in AvroSerDe
[ https://issues.apache.org/jira/browse/HIVE-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5823: -- Attachment: HIVE-5823.4.patch Patch #4 updated with review comment. Support for DECIMAL primitive type in AvroSerDe --- Key: HIVE-5823 URL: https://issues.apache.org/jira/browse/HIVE-5823 Project: Hive Issue Type: New Feature Components: Serializers/Deserializers Affects Versions: 0.12.0 Reporter: Mariano Dominguez Assignee: Xuefu Zhang Labels: avro, serde Attachments: HIVE-5823.1.patch, HIVE-5823.2.patch, HIVE-5823.3.patch, HIVE-5823.4.patch, HIVE-5823.patch, dec.avro This new feature request would be tied to AVRO-1402. Adding DECIMAL support would be particularly interesting when converting types from Avro to Hive, since DECIMAL is already a supported data type in Hive. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7008) Clean-up some old dead code
[ https://issues.apache.org/jira/browse/HIVE-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988430#comment-13988430 ] Jason Dere commented on HIVE-7008: -- +1 Clean-up some old dead code --- Key: HIVE-7008 URL: https://issues.apache.org/jira/browse/HIVE-7008 Project: Hive Issue Type: Task Components: Tests Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-7008.1.patch, HIVE-7008.patch There is some code to workaround limitations in historic Hadoop (hadoop-17 earlier). Lets get rid of those. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6915) Hive Hbase queries fail on secure Tez cluster
[
https://issues.apache.org/jira/browse/HIVE-6915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Craig Condit updated HIVE-6915:
---
Attachment: HIVE-6915.2.patch
New patch version. This version calls TableMapReduceUtil.initCredentials() only
in the case where the current user is logged in via Kerberos.
Hive Hbase queries fail on secure Tez cluster
-
Key: HIVE-6915
URL: https://issues.apache.org/jira/browse/HIVE-6915
Project: Hive
Issue Type: Bug
Components: Tez
Affects Versions: 0.13.0
Environment: Kerberos secure Tez cluster
Reporter: Deepesh Khandelwal
Assignee: Siddharth Seth
Attachments: HIVE-6915.1.patch, HIVE-6915.2.patch
Hive queries reading and writing to HBase are currently failing with the
following exception in a secure Tez cluster:
{noformat}
2014-04-14 13:47:05,644 FATAL [InputInitializer [Map 1] #0]
org.apache.hadoop.ipc.RpcClient: SASL authentication failed. The most likely
cause is missing or invalid credentials. Consider 'kinit'.
javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Failed to find
any Kerberos tgt)]
at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at
org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:152)
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupSaslConnection(RpcClient.java:792)
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection.access$800(RpcClient.java:349)
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection$2.run(RpcClient.java:918)
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection$2.run(RpcClient.java:915)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupIOstreams(RpcClient.java:915)
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection.writeRequest(RpcClient.java:1065)
at
org.apache.hadoop.hbase.ipc.RpcClient$Connection.tracedWriteRequest(RpcClient.java:1032)
at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1474)
at
org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1684)
at
org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1737)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.execService(ClientProtos.java:29288)
at
org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1562)
at
org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:87)
at
org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:84)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:121)
at
org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:97)
at
org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:90)
at
org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callBlockingMethod(CoprocessorRpcChannel.java:67)
at
org.apache.hadoop.hbase.protobuf.generated.AuthenticationProtos$AuthenticationService$BlockingStub.getAuthenticationToken(AuthenticationProtos.java:4512)
at
org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:60)
at
org.apache.hadoop.hbase.security.token.TokenUtil$3.run(TokenUtil.java:174)
at
org.apache.hadoop.hbase.security.token.TokenUtil$3.run(TokenUtil.java:172)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1557)
at
org.apache.hadoop.hbase.security.token.TokenUtil.obtainTokenForJob(TokenUtil.java:171)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.hbase.util.Methods.call(Methods.java:39)
at
org.apache.hadoop.hbase.security.User$SecureHadoopUser.obtainAuthTokenForJob(User.java:334)
at
org.apache.hadoop.hbase.mapred.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:201)
at
[jira] [Updated] (HIVE-6826) Hive-tez has issues when different partitions work off of different input types
[ https://issues.apache.org/jira/browse/HIVE-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-6826: - Status: Open (was: Patch Available) Hive-tez has issues when different partitions work off of different input types --- Key: HIVE-6826 URL: https://issues.apache.org/jira/browse/HIVE-6826 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.13.0, 0.14.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: HIVE-6826.1.patch, HIVE-6826.2.patch create table test (key int, value string) partitioned by (p int) stored as textfile; insert into table test partition (p=1) select * from src limit 10; alter table test set fileformat orc; insert into table test partition (p=2) select * from src limit 10; describe test; select * from test where p=1 and key 0; select * from test where p=2 and key 0; select * from test where key 0; throws a classcast exception -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6826) Hive-tez has issues when different partitions work off of different input types
[ https://issues.apache.org/jira/browse/HIVE-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-6826: - Attachment: HIVE-6826.2.patch Address Gunther's comments. Hive-tez has issues when different partitions work off of different input types --- Key: HIVE-6826 URL: https://issues.apache.org/jira/browse/HIVE-6826 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.13.0, 0.14.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: HIVE-6826.1.patch, HIVE-6826.2.patch create table test (key int, value string) partitioned by (p int) stored as textfile; insert into table test partition (p=1) select * from src limit 10; alter table test set fileformat orc; insert into table test partition (p=2) select * from src limit 10; describe test; select * from test where p=1 and key 0; select * from test where p=2 and key 0; select * from test where key 0; throws a classcast exception -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6826) Hive-tez has issues when different partitions work off of different input types
[ https://issues.apache.org/jira/browse/HIVE-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-6826: - Status: Patch Available (was: Open) Hive-tez has issues when different partitions work off of different input types --- Key: HIVE-6826 URL: https://issues.apache.org/jira/browse/HIVE-6826 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.13.0, 0.14.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: HIVE-6826.1.patch, HIVE-6826.2.patch create table test (key int, value string) partitioned by (p int) stored as textfile; insert into table test partition (p=1) select * from src limit 10; alter table test set fileformat orc; insert into table test partition (p=2) select * from src limit 10; describe test; select * from test where p=1 and key 0; select * from test where p=2 and key 0; select * from test where key 0; throws a classcast exception -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6826) Hive-tez has issues when different partitions work off of different input types
[ https://issues.apache.org/jira/browse/HIVE-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-6826: - Status: Open (was: Patch Available) Hive-tez has issues when different partitions work off of different input types --- Key: HIVE-6826 URL: https://issues.apache.org/jira/browse/HIVE-6826 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.13.0, 0.14.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: HIVE-6826.1.patch, HIVE-6826.2.patch create table test (key int, value string) partitioned by (p int) stored as textfile; insert into table test partition (p=1) select * from src limit 10; alter table test set fileformat orc; insert into table test partition (p=2) select * from src limit 10; describe test; select * from test where p=1 and key 0; select * from test where p=2 and key 0; select * from test where key 0; throws a classcast exception -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6826) Hive-tez has issues when different partitions work off of different input types
[ https://issues.apache.org/jira/browse/HIVE-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-6826: - Attachment: (was: HIVE-6826.2.patch) Hive-tez has issues when different partitions work off of different input types --- Key: HIVE-6826 URL: https://issues.apache.org/jira/browse/HIVE-6826 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.13.0, 0.14.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: HIVE-6826.1.patch create table test (key int, value string) partitioned by (p int) stored as textfile; insert into table test partition (p=1) select * from src limit 10; alter table test set fileformat orc; insert into table test partition (p=2) select * from src limit 10; describe test; select * from test where p=1 and key 0; select * from test where p=2 and key 0; select * from test where key 0; throws a classcast exception -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6826) Hive-tez has issues when different partitions work off of different input types
[ https://issues.apache.org/jira/browse/HIVE-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-6826: - Status: Patch Available (was: Open) Hive-tez has issues when different partitions work off of different input types --- Key: HIVE-6826 URL: https://issues.apache.org/jira/browse/HIVE-6826 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.13.0, 0.14.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: HIVE-6826.1.patch, HIVE-6826.2.patch create table test (key int, value string) partitioned by (p int) stored as textfile; insert into table test partition (p=1) select * from src limit 10; alter table test set fileformat orc; insert into table test partition (p=2) select * from src limit 10; describe test; select * from test where p=1 and key 0; select * from test where p=2 and key 0; select * from test where key 0; throws a classcast exception -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6826) Hive-tez has issues when different partitions work off of different input types
[ https://issues.apache.org/jira/browse/HIVE-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vikram Dixit K updated HIVE-6826: - Attachment: HIVE-6826.2.patch Hive-tez has issues when different partitions work off of different input types --- Key: HIVE-6826 URL: https://issues.apache.org/jira/browse/HIVE-6826 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.13.0, 0.14.0 Reporter: Vikram Dixit K Assignee: Vikram Dixit K Attachments: HIVE-6826.1.patch, HIVE-6826.2.patch create table test (key int, value string) partitioned by (p int) stored as textfile; insert into table test partition (p=1) select * from src limit 10; alter table test set fileformat orc; insert into table test partition (p=2) select * from src limit 10; describe test; select * from test where p=1 and key 0; select * from test where p=2 and key 0; select * from test where key 0; throws a classcast exception -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 21026: Remove HDFS check in tez/DagUtils
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21026/#review42080 --- ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java https://reviews.apache.org/r/21026/#comment75826 Just a proposal: in place of throwing exception, you could LOG info/debug something telling what type of FS is utilized. This might help to debug if someone used (intentionally or mistakenly) unsupported FS . - Mohammad Islam On May 2, 2014, 9:16 p.m., Chuan Liu wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/21026/ --- (Updated May 2, 2014, 9:16 p.m.) Review request for hive. Bugs: HIVE-7009 https://issues.apache.org/jira/browse/HIVE-7009 Repository: hive-git Description --- Removed all the DFS checks in the change. They should not impact any functionality. Diffs - ql/src/java/org/apache/hadoop/hive/ql/exec/tez/DagUtils.java abbf38f Diff: https://reviews.apache.org/r/21026/diff/ Testing --- Manually tested on a cluster deployed in Windows Azure environment. Thanks, Chuan Liu
Re: Apache Hive 0.13.1
Hi Folks, It is now past 6pm PDT May 2nd 2014, so I have locked down the table of jiras for inclusion into 0.13.1. The jiras included are as listed in : https://cwiki.apache.org/confluence/display/Hive/Hive+0.13.1+Release+tracking The jiras being included are : HIVE-6919, HIVE-6921, HIVE-6957, HIVE-6945, HIVE-6862, HIVE-6883, HIVE-6961, HIVE-4576, HIVE-6952, HIVE-6828 HIVE-6898, HIVE-6783, HIVE-6955, HIVE-6877, HIVE-6985 HIVE-7001 The inclusion requestors for the above are : thejas, ashutoshc, sershe, ekoifman, prasanth_j, hagleitn, szehon, rhbutani and amareshwari. If you are in this list, I might be pinging you in the next couple of days for help if the patch does not cleanly apply, or causes additional failures, and I appreciate any help you can tend. There is one jira on that list, HIVE-6826, which is in the list, and will still be included if it moves from Patch-available to Committed in 0.14 by Monday when I roll out the RC, which I have assurances from vikram that it will, given that it's waiting on a test result. If anyone has any further breaking jiras that they absolutely want in this release, please do still bring it up, I've created a new table for them in the release tracking wiki page. I do, however, raise the bar for inclusion by asking two other committers to also +1 such an inclusion request at this stage. Hopefully, we'll see an RC and a release pretty soon! :) Thanks all! -Sushanth On Thu, May 1, 2014 at 2:34 PM, Sushanth Sowmyan khorg...@gmail.com wrote: Hi All, A quick reminder - per the timeline discussed so far, the list of jiras for inclusion in 0.13.1 RC0 freezes tomorrow (May 2nd) evening at 6pm PDT. Please make sure you ask for inclusion or edit the wiki with any further requests so that we can get this release done in a timely manner. Thanks, -Sushanth On Mon, Apr 28, 2014 at 1:39 PM, Sushanth Sowmyan khorg...@gmail.com wrote: Good question - I think that if the number of patches is small ( ~20 which I expect because this is a stabilization release soon after 0.13.0), I'd prefer to commit them myself, so that I can retain order of patch application and make sure they all apply cleanly. I'm also actually using this as an opportunity towards testing out an semi-automated script that reads the requested jiras to do this - I figure it's useful to test something like that out on a smaller release that'll make further releases easier. On Mon, Apr 28, 2014 at 11:42 AM, Thejas Nair the...@hortonworks.com wrote: Sushanth, How do you want to go about committing the patches into 0.13 branch ? Will you be able to commit the requested patches into 0.13.1 yourself, or do you want requestors to discuss it with you and then commit the patches themselves (if the requestor is a committer). Thanks, Thejas On Sat, Apr 26, 2014 at 2:19 PM, Sushanth Sowmyan khorg...@gmail.comwrote: Added. If others have difficulty editing( I can't figure out how to change editing privileges, but it seems to indicate that others can edit) the page, I'll accept replies to this thread as well and can add it in. On Apr 25, 2014 6:25 PM, Sergey Shelukhin ser...@hortonworks.com wrote: I don't have access to edit this page (or cannot figure out the UI). Username sershe. Can you add HIVE-6961 : Drop partitions treats partition columns as strings (area - metastore) On Fri, Apr 25, 2014 at 4:20 PM, Sushanth Sowmyan khorg...@gmail.com wrote: I've created the following wiki link : https://cwiki.apache.org/confluence/display/Hive/Hive+0.13.1+Release+tracking People should be able to request additional jiras by adding it to the list. I think it might make sense to halt addition of requests to the list 3 days before the RC is cut, so as to prevent an endless-tail scenario, unless the bug in question is a breaking severe issue, where, yes, after discussion, we can vote to add it to the list. That also gives us time to run a full suite of tests on a stable build before we cut the RC. I propose that the first RC (RC0) be built on Monday May 5th at 6pm PDT, and the jira list on the wiki be closed to open/easy additions at 6pm PDT on Friday May 2nd. On Fri, Apr 25, 2014 at 2:40 PM, Gunther Hagleitner ghagleit...@hortonworks.com wrote: Sorry - HIVE-6824 isn't needed. Just the other 3. My bad. Thanks, Gunther. On Fri, Apr 25, 2014 at 2:10 PM, Gunther Hagleitner ghagleit...@hortonworks.com wrote: I'd like to request to include these Tez fixes: HIVE-6824, HIVE-6826, HIVE-6828, HIVE-6898 Thanks, Gunther. On Fri, Apr 25, 2014 at 11:59 AM, Sushanth Sowmyan khorg...@gmail.com wrote: True, I was counting two weeks from today, but 0.13 has already been out for a week. I'm amenable to having an RC1 out on May 5th. If any further issues appear that block, then we can deal with them in an RC2/etc modification to
[jira] [Created] (HIVE-7011) HiveInputFormat's split generation isn't thread safe
Gunther Hagleitner created HIVE-7011: Summary: HiveInputFormat's split generation isn't thread safe Key: HIVE-7011 URL: https://issues.apache.org/jira/browse/HIVE-7011 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.13.0, 0.14.0 Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Tez will do split generation in parallel. Need to protect the inputformat cache against concurrent access. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7011) HiveInputFormat's split generation isn't thread safe
[ https://issues.apache.org/jira/browse/HIVE-7011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-7011: - Attachment: HIVE-7011.1.patch HiveInputFormat's split generation isn't thread safe Key: HIVE-7011 URL: https://issues.apache.org/jira/browse/HIVE-7011 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.13.0, 0.14.0 Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-7011.1.patch Tez will do split generation in parallel. Need to protect the inputformat cache against concurrent access. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-7011) HiveInputFormat's split generation isn't thread safe
[ https://issues.apache.org/jira/browse/HIVE-7011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-7011: - Status: Patch Available (was: Open) HiveInputFormat's split generation isn't thread safe Key: HIVE-7011 URL: https://issues.apache.org/jira/browse/HIVE-7011 Project: Hive Issue Type: Bug Components: Tez Affects Versions: 0.13.0, 0.14.0 Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-7011.1.patch Tez will do split generation in parallel. Need to protect the inputformat cache against concurrent access. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-7008) Clean-up some old dead code
[
https://issues.apache.org/jira/browse/HIVE-7008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988535#comment-13988535
]
Hive QA commented on HIVE-7008:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12643087/HIVE-7008.patch
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 5428 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_groupby2_map_skew
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_list_bucket
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_stats_partscan_1_23
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_root_dir_external_table
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_dynamic_partitions_with_whitelist
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_partialscan_autogether
{noformat}
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/108/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-Build/108/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12643087
Clean-up some old dead code
---
Key: HIVE-7008
URL: https://issues.apache.org/jira/browse/HIVE-7008
Project: Hive
Issue Type: Task
Components: Tests
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
Attachments: HIVE-7008.1.patch, HIVE-7008.patch
There is some code to workaround limitations in historic Hadoop (hadoop-17
earlier). Lets get rid of those.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6684) Beeline does not accept comments that are preceded by spaces
[
https://issues.apache.org/jira/browse/HIVE-6684?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13988547#comment-13988547
]
Xuefu Zhang commented on HIVE-6684:
---
Patch looks good. Minor nit though, is that in the line
{code}
+return line.trim().startsWith(#) || line.trim().startsWith(--);
{code}
line.trim() may be called twice, which is unnecessary. Better if just to trim
once.
Beeline does not accept comments that are preceded by spaces
Key: HIVE-6684
URL: https://issues.apache.org/jira/browse/HIVE-6684
Project: Hive
Issue Type: Bug
Components: CLI
Affects Versions: 0.10.0
Reporter: Jeremy Beard
Attachments: HIVE-6684.1.patch
Beeline throws an error if single-line comments are indented with spaces.
This works in the embedded Hive CLI.
For example:
SELECT
-- this is the field we want
field
FROM
table;
Error: Error while processing statement: FAILED: ParseException line 1:71
cannot recognize input near 'EOF' 'EOF' 'EOF' in select clause
(state=42000,code=4)
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-5823) Support for DECIMAL primitive type in AvroSerDe
[ https://issues.apache.org/jira/browse/HIVE-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-5823: -- Attachment: HIVE-5823.5.patch Support for DECIMAL primitive type in AvroSerDe --- Key: HIVE-5823 URL: https://issues.apache.org/jira/browse/HIVE-5823 Project: Hive Issue Type: New Feature Components: Serializers/Deserializers Affects Versions: 0.12.0 Reporter: Mariano Dominguez Assignee: Xuefu Zhang Labels: avro, serde Attachments: HIVE-5823.1.patch, HIVE-5823.2.patch, HIVE-5823.3.patch, HIVE-5823.4.patch, HIVE-5823.5.patch, HIVE-5823.patch, dec.avro This new feature request would be tied to AVRO-1402. Adding DECIMAL support would be particularly interesting when converting types from Avro to Hive, since DECIMAL is already a supported data type in Hive. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 20826: HIVE-5823: Support for DECIMAL primitive type in AvroSerDe
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/20826/ --- (Updated May 3, 2014, 3:44 a.m.) Review request for hive. Bugs: HIVE-5823 https://issues.apache.org/jira/browse/HIVE-5823 Repository: hive-git Description --- Support decimal type in Avro storage. The implemenation is based on specifications detailed in AVRO-1402. Diffs (updated) - data/files/dec.txt PRE-CREATION ql/src/java/org/apache/hadoop/hive/ql/exec/TableScanOperator.java ecce21e ql/src/java/org/apache/hadoop/hive/ql/io/avro/AvroGenericRecordReader.java 52a22e5 ql/src/test/queries/clientpositive/avro_decimal.q PRE-CREATION ql/src/test/queries/clientpositive/avro_schema_literal.q d77f310 ql/src/test/results/clientpositive/avro_decimal.q.out PRE-CREATION ql/src/test/results/clientpositive/avro_schema_literal.q.out ca945d5 serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroDeserializer.java a28861f serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroGenericRecordWritable.java 8beffd7 serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerDe.java 92799ed serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerdeUtils.java 9d58d13 serde/src/java/org/apache/hadoop/hive/serde2/avro/AvroSerializer.java b2c58c7 serde/src/java/org/apache/hadoop/hive/serde2/avro/SchemaToTypeInfo.java 251f04f serde/src/test/org/apache/hadoop/hive/serde2/avro/TestAvroSerializer.java b3559ea serde/src/test/org/apache/hadoop/hive/serde2/avro/TestGenericAvroRecordWritable.java a0e5018 serde/src/test/org/apache/hadoop/hive/serde2/avro/Utils.java d5730fa Diff: https://reviews.apache.org/r/20826/diff/ Testing --- Unit tests are added. Test suite passed. Thanks, Xuefu Zhang
