[jira] [Updated] (HIVE-6541) Need to write documentation for ACID work
[ https://issues.apache.org/jira/browse/HIVE-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lefty Leverenz updated HIVE-6541: - Attachment: hive-6541-firstEdit.rtf Attaching an rtf file with color-coded edits. (Hope you like Mac orchid.) Looks good to me, just a few suggestions and confusions. Need to write documentation for ACID work - Key: HIVE-6541 URL: https://issues.apache.org/jira/browse/HIVE-6541 Project: Hive Issue Type: Sub-task Components: Documentation Affects Versions: 0.13.0 Reporter: Alan Gates Assignee: Alan Gates Fix For: 0.13.0 Attachments: hive-6541-firstEdit.rtf, hive-6541.txt ACID introduces a number of new config file options, tables in the metastore, keywords in the grammar, and a new interface for use of tools like storm and flume. These need to be documented. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6531) Runtime errors in vectorized execution.
[ https://issues.apache.org/jira/browse/HIVE-6531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13923692#comment-13923692 ] Remus Rusanu commented on HIVE-6531: +1 Runtime errors in vectorized execution. --- Key: HIVE-6531 URL: https://issues.apache.org/jira/browse/HIVE-6531 Project: Hive Issue Type: Bug Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Attachments: HIVE-6531.1.patch, HIVE-6531.2.patch, HIVE-6531.3.patch There are a few runtime errors observed in some of the tpcds queries for following reasons: 1) VectorFileSinkOperator fails with LazyBinarySerde. 2) Decimal128 and Unsigned128 don't serialize correctly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6531) Runtime errors in vectorized execution.
[ https://issues.apache.org/jira/browse/HIVE-6531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HIVE-6531: --- Status: Patch Available (was: Open) Runtime errors in vectorized execution. --- Key: HIVE-6531 URL: https://issues.apache.org/jira/browse/HIVE-6531 Project: Hive Issue Type: Bug Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Attachments: HIVE-6531.1.patch, HIVE-6531.2.patch, HIVE-6531.3.patch There are a few runtime errors observed in some of the tpcds queries for following reasons: 1) VectorFileSinkOperator fails with LazyBinarySerde. 2) Decimal128 and Unsigned128 don't serialize correctly. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6581) vectorization_decimal_date.q is dependent on local host timezone
Remus Rusanu created HIVE-6581: -- Summary: vectorization_decimal_date.q is dependent on local host timezone Key: HIVE-6581 URL: https://issues.apache.org/jira/browse/HIVE-6581 Project: Hive Issue Type: Test Reporter: Remus Rusanu Assignee: Teddy Choi Running vectorization_decimal_date.q fails in eastern hemisphere due to result diffs. In my GMT+2 zone, the first column returns 1970-01-07. This comes from the CAST (CAST (cint AS TIMESTAMP) AS DATE) AS cdate expression in the CTAS, this returns different dates on depending on local timezone (in row-mode, this is not a vectorization issue). -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18065: HIVE-6024 Load data local inpath unnecessarily creates a copy task
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18065/ --- (Updated March 7, 2014, 10:21 a.m.) Review request for hive and Ashutosh Chauhan. Changes --- Addressed unit test failures. Please check 3 .q.out changes. Those were required because this patch removes one extra copy phase. Bugs: HIVE-6024 https://issues.apache.org/jira/browse/HIVE-6024 Repository: hive-git Description --- Excerpt from the JIRA: Load data command creates an additional copy task only when its loading from local It doesn't create this additional copy task while loading from DFS though. Diffs (updated) - itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/history/TestHiveHistory.java 8beef09 ql/src/java/org/apache/hadoop/hive/ql/exec/MoveTask.java a190155 ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 05a2da7 ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java 8318be1 ql/src/java/org/apache/hadoop/hive/ql/parse/LoadSemanticAnalyzer.java 9ec22ca ql/src/java/org/apache/hadoop/hive/ql/plan/MoveWork.java 407450e ql/src/test/org/apache/hadoop/hive/ql/exec/TestExecDriver.java 5991aae ql/src/test/queries/clientpositive/load_local_dir_test.q PRE-CREATION ql/src/test/results/clientpositive/input4.q.out 9b169f9 ql/src/test/results/clientpositive/load_local_dir_test.q.out PRE-CREATION ql/src/test/results/clientpositive/stats11.q.out ce1197e ql/src/test/results/clientpositive/stats3.q.out a14e449 Diff: https://reviews.apache.org/r/18065/diff/ Testing --- Ran some existing q tests with LOAD DATA LOCAL INPATH. Thanks, Mohammad Islam
[jira] [Updated] (HIVE-6024) Load data local inpath unnecessarily creates a copy task
[
https://issues.apache.org/jira/browse/HIVE-6024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mohammad Kamrul Islam updated HIVE-6024:
Attachment: HIVE-6024.4.patch
Addressed unit test failures.
Please check 3 .q.out changes. Those were required because this patch removes
one extra copy phase.
Load data local inpath unnecessarily creates a copy task
Key: HIVE-6024
URL: https://issues.apache.org/jira/browse/HIVE-6024
Project: Hive
Issue Type: Improvement
Components: Query Processor
Reporter: Ashutosh Chauhan
Assignee: Mohammad Kamrul Islam
Attachments: HIVE-6024.1.patch, HIVE-6024.2.patch, HIVE-6024.3.patch,
HIVE-6024.4.patch
Load data command creates an additional copy task only when its loading from
{{local}} It doesn't create this additional copy task while loading from DFS
though.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: Review Request 18793: Bug fix.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18793/#review36507 --- Ship it! Ship It! - Thejas Nair On March 5, 2014, 8:05 p.m., Ashutosh Chauhan wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18793/ --- (Updated March 5, 2014, 8:05 p.m.) Review request for hive and Thejas Nair. Bugs: HIVE-6417 https://issues.apache.org/jira/browse/HIVE-6417 Repository: hive-git Description --- Bug fix. Diffs - itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestAdminUser.java PRE-CREATION metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java ba51b96 Diff: https://reviews.apache.org/r/18793/diff/ Testing --- Added new test. Thanks, Ashutosh Chauhan
[jira] [Commented] (HIVE-6553) hive CLI shell use of `dirname $0` prevents it from being linked from other directories
[
https://issues.apache.org/jira/browse/HIVE-6553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13923787#comment-13923787
]
Remus Rusanu commented on HIVE-6553:
I'm not comfortable submitting a patch that uses readlink because readlink is
not available on all platforms and I think readlink -f is not available on OS
X. Given that no one complained about this issue before, destabilizing the main
CLI seems too risky.
hive CLI shell use of `dirname $0` prevents it from being linked from other
directories
-
Key: HIVE-6553
URL: https://issues.apache.org/jira/browse/HIVE-6553
Project: Hive
Issue Type: Bug
Reporter: Remus Rusanu
Priority: Minor
I tried making /usr/bin/hive a link to /usr/lib/hive-0.14/bin/hive . But the
launch fails because:
{code}
bin=`dirname $0`
bin=`cd $bin; pwd`
. $bin/hive-config.sh
{code}
So I used a readlink -f first, to locate the proper script home, and then it
works fine.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6581) vectorization_decimal_date.q is dependent on local host timezone
[ https://issues.apache.org/jira/browse/HIVE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13923788#comment-13923788 ] Thejas M Nair commented on HIVE-6581: - AFAIK, the build/test scripts are supposed to set the timezone to US/Pacific for consistent results. I see that pom.xml setsTZUS/Pacific/TZ. vectorization_decimal_date.q is dependent on local host timezone Key: HIVE-6581 URL: https://issues.apache.org/jira/browse/HIVE-6581 Project: Hive Issue Type: Test Reporter: Remus Rusanu Assignee: Teddy Choi Running vectorization_decimal_date.q fails in eastern hemisphere due to result diffs. In my GMT+2 zone, the first column returns 1970-01-07. This comes from the CAST (CAST (cint AS TIMESTAMP) AS DATE) AS cdate expression in the CTAS, this returns different dates on depending on local timezone (in row-mode, this is not a vectorization issue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6581) vectorization_decimal_date.q is dependent on local host timezone
[ https://issues.apache.org/jira/browse/HIVE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13923792#comment-13923792 ] Remus Rusanu commented on HIVE-6581: [~thejas] It has no effect if you run the tests on Windows. vectorization_decimal_date.q is dependent on local host timezone Key: HIVE-6581 URL: https://issues.apache.org/jira/browse/HIVE-6581 Project: Hive Issue Type: Test Reporter: Remus Rusanu Assignee: Teddy Choi Running vectorization_decimal_date.q fails in eastern hemisphere due to result diffs. In my GMT+2 zone, the first column returns 1970-01-07. This comes from the CAST (CAST (cint AS TIMESTAMP) AS DATE) AS cdate expression in the CTAS, this returns different dates on depending on local timezone (in row-mode, this is not a vectorization issue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (HIVE-6581) vectorization_decimal_date.q is dependent on local host timezone
[ https://issues.apache.org/jira/browse/HIVE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13923810#comment-13923810 ] Thejas M Nair edited comment on HIVE-6581 at 3/7/14 11:48 AM: -- This is seems to be related to HIVE-5175 . This is probably not the only unit test that would fail on windows machine with non pacific timezone. was (Author: thejas): This is probably related to HIVE-5175 . This is probably not the only unit test that would fail on windows machine with non pacific timezone. vectorization_decimal_date.q is dependent on local host timezone Key: HIVE-6581 URL: https://issues.apache.org/jira/browse/HIVE-6581 Project: Hive Issue Type: Test Reporter: Remus Rusanu Assignee: Teddy Choi Running vectorization_decimal_date.q fails in eastern hemisphere due to result diffs. In my GMT+2 zone, the first column returns 1970-01-07. This comes from the CAST (CAST (cint AS TIMESTAMP) AS DATE) AS cdate expression in the CTAS, this returns different dates on depending on local timezone (in row-mode, this is not a vectorization issue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6581) vectorization_decimal_date.q is dependent on local host timezone
[ https://issues.apache.org/jira/browse/HIVE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13923810#comment-13923810 ] Thejas M Nair commented on HIVE-6581: - This is probably related to HIVE-5175 . This is probably not the only unit test that would fail on windows machine with non pacific timezone. vectorization_decimal_date.q is dependent on local host timezone Key: HIVE-6581 URL: https://issues.apache.org/jira/browse/HIVE-6581 Project: Hive Issue Type: Test Reporter: Remus Rusanu Assignee: Teddy Choi Running vectorization_decimal_date.q fails in eastern hemisphere due to result diffs. In my GMT+2 zone, the first column returns 1970-01-07. This comes from the CAST (CAST (cint AS TIMESTAMP) AS DATE) AS cdate expression in the CTAS, this returns different dates on depending on local timezone (in row-mode, this is not a vectorization issue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6363) IllegalArgumentException is thrown instead of SQLException
[
https://issues.apache.org/jira/browse/HIVE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13923834#comment-13923834
]
chandra sekhar gunturi commented on HIVE-6363:
--
I think the issue is resolved by
[HIVE-4194|https://issues.apache.org/jira/browse/HIVE-4194].
The parseURL is called from two places.
https://github.com/apache/hive/blob/trunk/jdbc/src/java/org/apache/hive/jdbc/HiveDriver.java#L233
(from parseURLforPropertyInfo() function)
https://github.com/apache/hive/blob/trunk/jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java#L122
(Constructor of HiveConnection)
Both are having the condition check if(URI.startsWith(jdbc:hive2://), hence
the parseURL() never gets called when the condition
uri.startsWith(URL_PREFIX) is false.
But it is good to have fix so that if some one calls the function
unconditionally in future, the issue will resurface.
I think [HIVE-4802|https://issues.apache.org/jira/browse/HIVE-4802] did not
affect changes made by
[HIVE-4194|https://issues.apache.org/jira/browse/HIVE-4194] as conditional call
is still in place.
I was testing this on Hive 0.11 saw the code for parseURL was not changed
from throwing IllegalArgumentException. I think the JIRA can be changed to
lower priority.
IllegalArgumentException is thrown instead of SQLException
--
Key: HIVE-6363
URL: https://issues.apache.org/jira/browse/HIVE-6363
Project: Hive
Issue Type: Bug
Components: JDBC
Affects Versions: 0.10.0, 0.11.0, 0.12.0
Reporter: chandra sekhar gunturi
Priority: Critical
Labels: JDBC, unit-test
Attachments: HIVE-6363.patch.1.txt, HIVE-6363.patch.2.txt
Original Estimate: 24h
Remaining Estimate: 24h
parseURL in the following code is throwing IllegalArgumentException
http://svn.apache.org/viewvc/hive/trunk/jdbc/src/java/org/apache/hive/jdbc/Utils.java?view=markup
This is going to break other JDBC based connectors because
java.sql.DriverManager doesn’t catch IllegalArgumentException while probing
for correct Driver for a given URL.
A simple test case can have class.forName(org.apache.hive.jdbc.HiveDriver)
(Loading hiveserver2 JDBC driver) followed by
class.forName(org.apache.hadoop.hive.jdbc.HiveDriver)(Loading hiveserver JDBC
driver).
In this case hiveserver connection will fail with BAD URL format for
hiveserver. If you reverse the driver loading to hiveserver followed by
hiveserver2, both the connections will be successful.
Following code in java.sql.DriverManager is causing the issue
[[
// Worker method called by the public getConnection() methods.
private static Connection getConnection(
// Walk through the loaded registeredDrivers attempting to make a connection.
// Remember the first exception that gets raised so we can reraise it.
for(DriverInfo aDriver : registeredDrivers) {
// If the caller does not have permission to load the driver then
// skip it.
if(isDriverAllowed(aDriver.driver, callerCL)) {
try {
Connection con = aDriver.driver.connect(url, info);
if (con != null) {
// Success!
println(getConnection returning + aDriver.driver.getClass().getName());
return (con);
}
} catch (SQLException ex) {
if (reason == null) {
reason = ex;
}
}
} else {
println( skipping: + aDriver.getClass().getName());
}
}
}
]]
Marking it as critical because this is going to restrict consuming JDBC
driver in production environment where many drivers are loaded on requirement
rather than statically loading all drivers.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6569) HCatalog still has references to deprecated property hive.metastore.local
[
https://issues.apache.org/jira/browse/HIVE-6569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13923836#comment-13923836
]
Hive QA commented on HIVE-6569:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633230/HIVE-6569.2.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 5357 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_mapreduce_stack_trace_hadoop20
org.apache.hive.beeline.TestSchemaTool.testSchemaInit
org.apache.hive.beeline.TestSchemaTool.testSchemaUpgrade
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1644/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1644/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633230
HCatalog still has references to deprecated property hive.metastore.local
-
Key: HIVE-6569
URL: https://issues.apache.org/jira/browse/HIVE-6569
Project: Hive
Issue Type: Bug
Reporter: Sushanth Sowmyan
Assignee: Sushanth Sowmyan
Priority: Minor
Labels: cleanup, hcatalog
Attachments: HIVE-6569.2.patch, HIVE-6569.patch
HIVE-2585 removed the conf parameter hive.metastore.local, but HCatalog still
has references to it. Most of it is in tests, but one is in PigHCatUtil,
which leads to HCatLoader/HCatStorer jobs giving warnings. We need to remove
them.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (HIVE-6581) vectorization_decimal_date.q is dependent on local host timezone
[ https://issues.apache.org/jira/browse/HIVE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu resolved HIVE-6581. Resolution: Fixed vectorization_decimal_date.q is dependent on local host timezone Key: HIVE-6581 URL: https://issues.apache.org/jira/browse/HIVE-6581 Project: Hive Issue Type: Test Reporter: Remus Rusanu Assignee: Teddy Choi Priority: Minor Running vectorization_decimal_date.q fails in eastern hemisphere due to result diffs. In my GMT+2 zone, the first column returns 1970-01-07. This comes from the CAST (CAST (cint AS TIMESTAMP) AS DATE) AS cdate expression in the CTAS, this returns different dates on depending on local timezone (in row-mode, this is not a vectorization issue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6581) vectorization_decimal_date.q is dependent on local host timezone
[ https://issues.apache.org/jira/browse/HIVE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Remus Rusanu updated HIVE-6581: --- Priority: Minor (was: Major) vectorization_decimal_date.q is dependent on local host timezone Key: HIVE-6581 URL: https://issues.apache.org/jira/browse/HIVE-6581 Project: Hive Issue Type: Test Reporter: Remus Rusanu Assignee: Teddy Choi Priority: Minor Running vectorization_decimal_date.q fails in eastern hemisphere due to result diffs. In my GMT+2 zone, the first column returns 1970-01-07. This comes from the CAST (CAST (cint AS TIMESTAMP) AS DATE) AS cdate expression in the CTAS, this returns different dates on depending on local timezone (in row-mode, this is not a vectorization issue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6581) vectorization_decimal_date.q is dependent on local host timezone
[ https://issues.apache.org/jira/browse/HIVE-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13923854#comment-13923854 ] Remus Rusanu commented on HIVE-6581: Agreed, I reolved as duplicate of HIVE-5175 vectorization_decimal_date.q is dependent on local host timezone Key: HIVE-6581 URL: https://issues.apache.org/jira/browse/HIVE-6581 Project: Hive Issue Type: Test Reporter: Remus Rusanu Assignee: Teddy Choi Priority: Minor Running vectorization_decimal_date.q fails in eastern hemisphere due to result diffs. In my GMT+2 zone, the first column returns 1970-01-07. This comes from the CAST (CAST (cint AS TIMESTAMP) AS DATE) AS cdate expression in the CTAS, this returns different dates on depending on local timezone (in row-mode, this is not a vectorization issue). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6582) missing ; in HTML entities like lt; in conf file
Pierre Nerzic created HIVE-6582: --- Summary: missing ; in HTML entities like lt; in conf file Key: HIVE-6582 URL: https://issues.apache.org/jira/browse/HIVE-6582 Project: Hive Issue Type: Bug Components: Configuration Affects Versions: 0.14.0 Reporter: Pierre Nerzic Priority: Trivial In conf/hive-default.xml.template, line 2392, the description of the property is malformed : lthive... instead of lt;hive... and 3 other like this : gt This causes an error when launching hive : org.xml.sax.SAXParseException (translated from french) reference to entity lthive.user.install.directory must end with ';'. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6582) missing ; in HTML entities like lt; in conf file
[ https://issues.apache.org/jira/browse/HIVE-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Nerzic updated HIVE-6582: Status: Patch Available (was: Open) missing ; in HTML entities like lt; in conf file - Key: HIVE-6582 URL: https://issues.apache.org/jira/browse/HIVE-6582 Project: Hive Issue Type: Bug Components: Configuration Affects Versions: 0.14.0 Reporter: Pierre Nerzic Priority: Trivial Attachments: HIVE-6582.1.patch.txt In conf/hive-default.xml.template, line 2392, the description of the property is malformed : lthive... instead of lt;hive... and 3 other like this : gt This causes an error when launching hive : org.xml.sax.SAXParseException (translated from french) reference to entity lthive.user.install.directory must end with ';'. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6582) missing ; in HTML entities like lt; in conf file
[ https://issues.apache.org/jira/browse/HIVE-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Nerzic updated HIVE-6582: Attachment: HIVE-6582.1.patch.txt missing ; in HTML entities like lt; in conf file - Key: HIVE-6582 URL: https://issues.apache.org/jira/browse/HIVE-6582 Project: Hive Issue Type: Bug Components: Configuration Affects Versions: 0.14.0 Reporter: Pierre Nerzic Priority: Trivial Attachments: HIVE-6582.1.patch.txt In conf/hive-default.xml.template, line 2392, the description of the property is malformed : lthive... instead of lt;hive... and 3 other like this : gt This causes an error when launching hive : org.xml.sax.SAXParseException (translated from french) reference to entity lthive.user.install.directory must end with ';'. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6582) missing ; in HTML entities like lt; in conf file
[
https://issues.apache.org/jira/browse/HIVE-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Nerzic updated HIVE-6582:
Description:
In conf/hive-default.xml.template, line 2392, the description of the property
is malformed : lthive... instead of {{lt;}}hive... and 3 other like this :
gt
This causes an error when launching hive : org.xml.sax.SAXParseException
(translated from french) reference to entity lthive.user.install.directory
must end with ';'.
was:
In conf/hive-default.xml.template, line 2392, the description of the property
is malformed : {{lt}}hive... instead of {{lt;}}hive... and 3 other like this
: {{gt}}
This causes an error when launching hive : org.xml.sax.SAXParseException
(translated from french) reference to entity lthive.user.install.directory
must end with ';'.
missing ; in HTML entities like lt; in conf file
-
Key: HIVE-6582
URL: https://issues.apache.org/jira/browse/HIVE-6582
Project: Hive
Issue Type: Bug
Components: Configuration
Affects Versions: 0.14.0
Reporter: Pierre Nerzic
Priority: Trivial
Attachments: HIVE-6582.1.patch.txt
In conf/hive-default.xml.template, line 2392, the description of the property
is malformed : lthive... instead of {{lt;}}hive... and 3 other like this :
gt
This causes an error when launching hive : org.xml.sax.SAXParseException
(translated from french) reference to entity lthive.user.install.directory
must end with ';'.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6582) missing ; in HTML entities like lt; in conf file
[
https://issues.apache.org/jira/browse/HIVE-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Nerzic updated HIVE-6582:
Description:
In conf/hive-default.xml.template, line 2392, the description of the property
is malformed : {{lt}}hive... instead of {{lt;}}hive... and 3 other like this
: {{gt}}
This causes an error when launching hive : org.xml.sax.SAXParseException
(translated from french) reference to entity lthive.user.install.directory
must end with ';'.
was:
In conf/hive-default.xml.template, line 2392, the description of the property
is malformed : lthive... instead of lt;hive... and 3 other like this : gt
This causes an error when launching hive : org.xml.sax.SAXParseException
(translated from french) reference to entity lthive.user.install.directory
must end with ';'.
missing ; in HTML entities like lt; in conf file
-
Key: HIVE-6582
URL: https://issues.apache.org/jira/browse/HIVE-6582
Project: Hive
Issue Type: Bug
Components: Configuration
Affects Versions: 0.14.0
Reporter: Pierre Nerzic
Priority: Trivial
Attachments: HIVE-6582.1.patch.txt
In conf/hive-default.xml.template, line 2392, the description of the property
is malformed : {{lt}}hive... instead of {{lt;}}hive... and 3 other like
this : {{gt}}
This causes an error when launching hive : org.xml.sax.SAXParseException
(translated from french) reference to entity lthive.user.install.directory
must end with ';'.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6582) missing ; in HTML entities like lt; in conf file
[ https://issues.apache.org/jira/browse/HIVE-6582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Nerzic updated HIVE-6582: Description: In conf/hive-default.xml.template, line 2392, the description of the property is malformed : lthive... instead of lt ;hive... and 3 other like this : gt on the same line. (I have problems with wikification to display lt ; and not ) This causes an error when launching hive : org.xml.sax.SAXParseException (translated from french) reference to entity lthive.user.install.directory must end with ';'. was: In conf/hive-default.xml.template, line 2392, the description of the property is malformed : lthive... instead of lt ;hive... and 3 other like this : gt (I have problems with wikification to display lt ; and not ) This causes an error when launching hive : org.xml.sax.SAXParseException (translated from french) reference to entity lthive.user.install.directory must end with ';'. missing ; in HTML entities like lt; in conf file - Key: HIVE-6582 URL: https://issues.apache.org/jira/browse/HIVE-6582 Project: Hive Issue Type: Bug Components: Configuration Affects Versions: 0.14.0 Reporter: Pierre Nerzic Priority: Trivial Attachments: HIVE-6582.1.patch.txt In conf/hive-default.xml.template, line 2392, the description of the property is malformed : lthive... instead of lt ;hive... and 3 other like this : gt on the same line. (I have problems with wikification to display lt ; and not ) This causes an error when launching hive : org.xml.sax.SAXParseException (translated from french) reference to entity lthive.user.install.directory must end with ';'. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5155) Support secure proxy user access to HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13923887#comment-13923887 ] Vaibhav Gumashta commented on HIVE-5155: +1 (non-binding). Latest patch looks good to me. Thanks for the patch [~prasadm]! Support secure proxy user access to HiveServer2 --- Key: HIVE-5155 URL: https://issues.apache.org/jira/browse/HIVE-5155 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.12.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-5155-1-nothrift.patch, HIVE-5155-noThrift.2.patch, HIVE-5155-noThrift.4.patch, HIVE-5155-noThrift.5.patch, HIVE-5155-noThrift.6.patch, HIVE-5155-noThrift.7.patch, HIVE-5155-noThrift.8.patch, HIVE-5155.1.patch, HIVE-5155.2.patch, HIVE-5155.3.patch, ProxyAuth.java, ProxyAuth.out, TestKERBEROS_Hive_JDBC.java The HiveServer2 can authenticate a client using via Kerberos and impersonate the connecting user with underlying secure hadoop. This becomes a gateway for a remote client to access secure hadoop cluster. Now this works fine for when the client obtains Kerberos ticket and directly connects to HiveServer2. There's another big use case for middleware tools where the end user wants to access Hive via another server. For example Oozie action or Hue submitting queries or a BI tool server accessing to HiveServer2. In these cases, the third party server doesn't have end user's Kerberos credentials and hence it can't submit queries to HiveServer2 on behalf of the end user. This ticket is for enabling proxy access to HiveServer2 for third party tools on behalf of end users. There are two parts of the solution proposed in this ticket: 1) Delegation token based connection for Oozie (OOZIE-1457) This is the common mechanism for Hadoop ecosystem components. Hive Remote Metastore and HCatalog already support this. This is suitable for tool like Oozie that submits the MR jobs as actions on behalf of its client. Oozie already uses similar mechanism for Metastore/HCatalog access. 2) Direct proxy access for privileged hadoop users The delegation token implementation can be a challenge for non-hadoop (especially non-java) components. This second part enables a privileged user to directly specify an alternate session user during the connection. If the connecting user has hadoop level privilege to impersonate the requested userid, then HiveServer2 will run the session as that requested user. For example, user Hue is allowed to impersonate user Bob (via core-site.xml proxy user configuration). Then user Hue can connect to HiveServer2 and specify Bob as session user via a session property. HiveServer2 will verify Hue's proxy user privilege and then impersonate user Bob instead of Hue. This will enable any third party tool to impersonate alternate userid without having to implement delegation token connection. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Proposal to switch to pull requests
+1 On Fri, Mar 7, 2014 at 1:05 AM, Thejas Nair the...@hortonworks.com wrote: Should we start with moving our primary source code repository from svn to git ? I feel git is more powerful and easy to use (once you go past the learning curve!). On Wed, Mar 5, 2014 at 7:39 AM, Brock Noland br...@cloudera.com wrote: Personally I prefer the Github workflow, but I believe there have been some challenges with that since the source for apache projects must be stored in apache source control (git or svn). Relevent: https://blogs.apache.org/infra/entry/improved_integration_between_apache_and On Wed, Mar 5, 2014 at 9:19 AM, kulkarni.swar...@gmail.com kulkarni.swar...@gmail.com wrote: Hello, Since we have a nice mirrored git repository for hive[1], any specific reason why we can't switch to doing pull requests instead of patches? IMHO pull requests are awesome for peer review plus it is also very easy to keep track of JIRAs with open pull requests instead of looking for JIRAs in a Patch Available state. Also since they get updated automatically, it is also very easy to see if a review comment made by a reviewer was addressed properly or not. Thoughts? Thanks, [1] https://github.com/apache/hive -- Swarnim -- Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Swarnim
Re: Hung Precommit Jenkins Jobs
AFAICT this version/installation of Jenkins does not support the time limits.
Ultimately it'd be nice to perhaps run Jenkins on the current master
host we have. That would give us dedicated executors and allow us full
control over the installation.
On Fri, Mar 7, 2014 at 1:07 AM, Thejas Nair the...@hortonworks.com wrote:
Would it be possible/help to set a time limit for the job in this jenkins job
?
On Thu, Mar 6, 2014 at 10:17 AM, Brock Noland br...@cloudera.com wrote:
Ashutosh informed me that the precommit build was hung. Long story
short, PTest2 had completed. For example, see this job which took 8
hours:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1636/
That correpsonds to:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-Build-1637/execution.txt
which if you scroll all the way to the bottom, you will note it
finished long before that:
2014-03-06 05:48:22,194 INFO PTest.run:207 Executed 5358 tests
2014-03-06 05:48:22,194 INFO PTest.run:209 PERF: Phase ExecutionPhase
took 101 minutes
2014-03-06 05:48:22,194 INFO PTest.run:209 PERF: Phase PrepPhase took 5
minutes
2014-03-06 05:48:22,194 INFO PTest.run:209 PERF: Phase ReportingPhase
took 0 minutes
2014-03-06 05:48:22,194 INFO JIRAService.postComment:136 Comment:
{color:red}Overall{color}: -1 at least one tests failed
Long story short, it looks like Bigtop Jenkins is not as reliable as
we would like.
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
[jira] [Created] (HIVE-6583) wrong sql comments : ----... instead of -- ---...
Pierre Nerzic created HIVE-6583: --- Summary: wrong sql comments : ... instead of -- ---... Key: HIVE-6583 URL: https://issues.apache.org/jira/browse/HIVE-6583 Project: Hive Issue Type: Bug Components: Database/Schema Affects Versions: 0.14.0 Reporter: Pierre Nerzic Priority: Minor Attachments: HIVE-6583.1.patch.txt In file metastore/scripts/upgrade/mysql/hive-schema-0.13.0.mysql.sql, lines 799 and 801, a comment is written as -- (uninterrupted line of dashes) and should be -- -- (a space after 2 dashes) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6583) wrong sql comments : ----... instead of -- ---...
[ https://issues.apache.org/jira/browse/HIVE-6583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Nerzic updated HIVE-6583: Fix Version/s: 0.14.0 Status: Patch Available (was: Open) wrong sql comments : ... instead of -- ---... - Key: HIVE-6583 URL: https://issues.apache.org/jira/browse/HIVE-6583 Project: Hive Issue Type: Bug Components: Database/Schema Affects Versions: 0.14.0 Reporter: Pierre Nerzic Priority: Minor Fix For: 0.14.0 Attachments: HIVE-6583.1.patch.txt In file metastore/scripts/upgrade/mysql/hive-schema-0.13.0.mysql.sql, lines 799 and 801, a comment is written as -- (uninterrupted line of dashes) and should be -- -- (a space after 2 dashes) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6583) wrong sql comments : ----... instead of -- ---...
[ https://issues.apache.org/jira/browse/HIVE-6583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Nerzic updated HIVE-6583: Attachment: HIVE-6583.1.patch.txt wrong sql comments : ... instead of -- ---... - Key: HIVE-6583 URL: https://issues.apache.org/jira/browse/HIVE-6583 Project: Hive Issue Type: Bug Components: Database/Schema Affects Versions: 0.14.0 Reporter: Pierre Nerzic Priority: Minor Fix For: 0.14.0 Attachments: HIVE-6583.1.patch.txt In file metastore/scripts/upgrade/mysql/hive-schema-0.13.0.mysql.sql, lines 799 and 801, a comment is written as -- (uninterrupted line of dashes) and should be -- -- (a space after 2 dashes) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6583) wrong sql comments : ----... instead of -- ---...
[ https://issues.apache.org/jira/browse/HIVE-6583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Nerzic updated HIVE-6583: Description: In file metastore/scripts/upgrade/mysql/hive-schema-0.13.0.mysql.sql, lines 799 and 801, a comment is written as -- (uninterrupted line of dashes) and should be -- -- (a space after 2 dashes) (source https://dev.mysql.com/doc/refman/5.7/en/comments.html) was:In file metastore/scripts/upgrade/mysql/hive-schema-0.13.0.mysql.sql, lines 799 and 801, a comment is written as -- (uninterrupted line of dashes) and should be -- -- (a space after 2 dashes) wrong sql comments : ... instead of -- ---... - Key: HIVE-6583 URL: https://issues.apache.org/jira/browse/HIVE-6583 Project: Hive Issue Type: Bug Components: Database/Schema Affects Versions: 0.14.0 Reporter: Pierre Nerzic Priority: Minor Fix For: 0.14.0 Attachments: HIVE-6583.1.patch.txt In file metastore/scripts/upgrade/mysql/hive-schema-0.13.0.mysql.sql, lines 799 and 801, a comment is written as -- (uninterrupted line of dashes) and should be -- -- (a space after 2 dashes) (source https://dev.mysql.com/doc/refman/5.7/en/comments.html) -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5901) Query cancel should stop running MR tasks
[ https://issues.apache.org/jira/browse/HIVE-5901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13923959#comment-13923959 ] Thejas M Nair commented on HIVE-5901: - [~navis] Can you please add this to the pre commit queue, if its not already there ? Query cancel should stop running MR tasks - Key: HIVE-5901 URL: https://issues.apache.org/jira/browse/HIVE-5901 Project: Hive Issue Type: Improvement Components: Query Processor Reporter: Navis Assignee: Navis Priority: Minor Attachments: HIVE-5901.1.patch.txt, HIVE-5901.2.patch.txt, HIVE-5901.3.patch.txt, HIVE-5901.4.patch.txt, HIVE-5901.5.patch.txt, HIVE-5901.6.patch.txt, HIVE-5901.7.patch.txt Currently, query canceling does not stop running MR job immediately. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6137) Hive should report that the file/path doesn’t exist when it doesn’t
[
https://issues.apache.org/jira/browse/HIVE-6137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13923977#comment-13923977
]
Hive QA commented on HIVE-6137:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633222/HIVE-6137.6.patch
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 5358 tests executed
*Failed tests:*
{noformat}
org.apache.hive.beeline.TestSchemaTool.testSchemaInit
org.apache.hive.beeline.TestSchemaTool.testSchemaUpgrade
org.apache.hive.service.cli.TestEmbeddedThriftBinaryCLIService.testExecuteStatementAsync
org.apache.hive.service.cli.thrift.TestThriftBinaryCLIService.testExecuteStatementAsync
org.apache.hive.service.cli.thrift.TestThriftHttpCLIService.testExecuteStatementAsync
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1645/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1645/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633222
Hive should report that the file/path doesn’t exist when it doesn’t
---
Key: HIVE-6137
URL: https://issues.apache.org/jira/browse/HIVE-6137
Project: Hive
Issue Type: Bug
Reporter: Hari Sankar Sivarama Subramaniyan
Assignee: Hari Sankar Sivarama Subramaniyan
Attachments: HIVE-6137.1.patch, HIVE-6137.2.patch, HIVE-6137.3.patch,
HIVE-6137.4.patch, HIVE-6137.5.patch, HIVE-6137.6.patch
Hive should report that the file/path doesn’t exist when it doesn’t (it now
reports SocketTimeoutException):
Execute a Hive DDL query with a reference to a non-existent blob (such as
CREATE EXTERNAL TABLE...) and check Hive logs (stderr):
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception:
java.io.IOException)
This error message is not detailed enough. If a file doesn't exist, Hive
should report that it received an error while trying to locate the file.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6417) sql std auth - new users in admin role config should get added
[ https://issues.apache.org/jira/browse/HIVE-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6417: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. [~rhbutani] Since its a bug on newly introduced feature in 0.13, it will be good to have this on 0.13 as well. sql std auth - new users in admin role config should get added -- Key: HIVE-6417 URL: https://issues.apache.org/jira/browse/HIVE-6417 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Ashutosh Chauhan Fix For: 0.14.0 Attachments: HIVE-6417.1.patch, HIVE-6417.patch if metastore is started with hive.users.in.admin.role=user1, then user1 is added admin role to metastore. If the value is changed to hive.users.in.admin.role=user2, then user2 should get added to the role in metastore. Right now, if the admin role exists, new users don't get added. A work-around is - user1 adding user2 to the admin role using grant role statement. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-3635) allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type
[ https://issues.apache.org/jira/browse/HIVE-3635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xuefu Zhang updated HIVE-3635: -- Resolution: Fixed Fix Version/s: 0.14.0 Release Note: New configuration, hive.lazysimple.extended_boolean_literal, needs to be documented. Status: Resolved (was: Patch Available) Patch committed to trunk. Thanks Brock for the review. allow 't', 'T', '1', 'f', 'F', and '0' to be allowable true/false values for the boolean hive type --- Key: HIVE-3635 URL: https://issues.apache.org/jira/browse/HIVE-3635 Project: Hive Issue Type: Improvement Components: CLI Affects Versions: 0.9.0 Reporter: Alexander Alten-Lorenz Assignee: Xuefu Zhang Fix For: 0.14.0 Attachments: HIVE-3635.1.patch, HIVE-3635.2.patch, HIVE-3635.patch interpret t as true and f as false for boolean types. PostgreSQL exports represent it that way. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6571) query id should be available for logging during query compilation
[
https://issues.apache.org/jira/browse/HIVE-6571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924087#comment-13924087
]
Hive QA commented on HIVE-6571:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633236/HIVE-6571.1.patch
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 5358 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_auto_sortmerge_join_16
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver_bucketmapjoin6
org.apache.hive.beeline.TestSchemaTool.testSchemaInit
org.apache.hive.beeline.TestSchemaTool.testSchemaUpgrade
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1647/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1647/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633236
query id should be available for logging during query compilation
-
Key: HIVE-6571
URL: https://issues.apache.org/jira/browse/HIVE-6571
Project: Hive
Issue Type: Bug
Reporter: Gunther Hagleitner
Assignee: Gunther Hagleitner
Priority: Minor
Attachments: HIVE-6571.1.patch
Would be nice to have the query id set during compilation to tie logs
together etc.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: Review Request 13845: HIVE-5155: Support secure proxy user access to HiveServer2
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13845/#review36524 --- common/src/java/org/apache/hadoop/hive/conf/HiveConf.java https://reviews.apache.org/r/13845/#comment67518 I don't see this conf being used anywhere in this patch. jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java https://reviews.apache.org/r/13845/#comment67536 thanks for adding javadoc/comments! service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java https://reviews.apache.org/r/13845/#comment67538 should we not call this from .close() as well ? We can do that in a follow up jira if necessary. - Thejas Nair On March 4, 2014, 10:47 p.m., Prasad Mujumdar wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13845/ --- (Updated March 4, 2014, 10:47 p.m.) Review request for hive, Brock Noland, Carl Steinbach, and Thejas Nair. Bugs: HIVE-5155 https://issues.apache.org/jira/browse/HIVE-5155 Repository: hive-git Description --- Delegation token support - Enable delegation token connection for HiveServer2 Enhance the TCLIService interface to support delegation token requests Support passing the delegation token connection type via JDBC URL and Beeline option Direct proxy access - Define new proxy user property Shim interfaces to validate proxy access for a given user Note that the diff doesn't include thrift generated code. Diffs - beeline/pom.xml 7449430 beeline/src/java/org/apache/hive/beeline/BeeLine.java 563d242 beeline/src/java/org/apache/hive/beeline/BeeLineOpts.java 91e20ec beeline/src/java/org/apache/hive/beeline/Commands.java d2d7fd3 beeline/src/java/org/apache/hive/beeline/DatabaseConnection.java 94178ef beeline/src/test/org/apache/hive/beeline/ProxyAuthTest.java PRE-CREATION common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 388a604 conf/hive-default.xml.template 3f01e0b data/files/ProxyAuth.res PRE-CREATION itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java 8210e75 jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveConnection.java d08e05b jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java 4102d7a jdbc/src/java/org/apache/hive/jdbc/Utils.java 608837e service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java d8ba3aa service/src/java/org/apache/hive/service/auth/KerberosSaslHelper.java 519556c service/src/java/org/apache/hive/service/auth/PlainSaslHelper.java 15b1675 service/src/java/org/apache/hive/service/cli/CLIService.java 2b1e712 service/src/java/org/apache/hive/service/cli/CLIServiceClient.java b9d1489 service/src/java/org/apache/hive/service/cli/EmbeddedCLIServiceClient.java a31ea94 service/src/java/org/apache/hive/service/cli/ICLIService.java 621d689 service/src/java/org/apache/hive/service/cli/session/HiveSession.java c8fb8ec service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java d6d0d27 service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java b934ebe service/src/java/org/apache/hive/service/cli/session/SessionManager.java cec3b04 service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 26bda5a service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIServiceClient.java 3675e86 service/src/test/org/apache/hive/service/auth/TestPlainSaslHelper.java 8fa4afd service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java 2fac800 shims/0.20/src/main/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 51c8051 shims/common-secure/src/main/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java e205caa shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java 29114f0 shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java dc89de1 shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java e15ab4e shims/common/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 03f4e51 Diff: https://reviews.apache.org/r/13845/diff/ Testing --- Since this requires kerberos setup, its tested by a standalone test program that runs various existing and new secure connection scenarios. The test code is attached to the ticket at https://issues.apache.org/jira/secure/attachment/12600119/ProxyAuth.java Thanks, Prasad Mujumdar
Re: Review Request 13845: HIVE-5155: Support secure proxy user access to HiveServer2
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13845/#review36539 --- common/src/java/org/apache/hadoop/hive/conf/HiveConf.java https://reviews.apache.org/r/13845/#comment67542 Never mind. please ignore. - Thejas Nair On March 4, 2014, 10:47 p.m., Prasad Mujumdar wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/13845/ --- (Updated March 4, 2014, 10:47 p.m.) Review request for hive, Brock Noland, Carl Steinbach, and Thejas Nair. Bugs: HIVE-5155 https://issues.apache.org/jira/browse/HIVE-5155 Repository: hive-git Description --- Delegation token support - Enable delegation token connection for HiveServer2 Enhance the TCLIService interface to support delegation token requests Support passing the delegation token connection type via JDBC URL and Beeline option Direct proxy access - Define new proxy user property Shim interfaces to validate proxy access for a given user Note that the diff doesn't include thrift generated code. Diffs - beeline/pom.xml 7449430 beeline/src/java/org/apache/hive/beeline/BeeLine.java 563d242 beeline/src/java/org/apache/hive/beeline/BeeLineOpts.java 91e20ec beeline/src/java/org/apache/hive/beeline/Commands.java d2d7fd3 beeline/src/java/org/apache/hive/beeline/DatabaseConnection.java 94178ef beeline/src/test/org/apache/hive/beeline/ProxyAuthTest.java PRE-CREATION common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 388a604 conf/hive-default.xml.template 3f01e0b data/files/ProxyAuth.res PRE-CREATION itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java 8210e75 jdbc/src/java/org/apache/hadoop/hive/jdbc/HiveConnection.java d08e05b jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java 4102d7a jdbc/src/java/org/apache/hive/jdbc/Utils.java 608837e service/src/java/org/apache/hive/service/auth/HiveAuthFactory.java d8ba3aa service/src/java/org/apache/hive/service/auth/KerberosSaslHelper.java 519556c service/src/java/org/apache/hive/service/auth/PlainSaslHelper.java 15b1675 service/src/java/org/apache/hive/service/cli/CLIService.java 2b1e712 service/src/java/org/apache/hive/service/cli/CLIServiceClient.java b9d1489 service/src/java/org/apache/hive/service/cli/EmbeddedCLIServiceClient.java a31ea94 service/src/java/org/apache/hive/service/cli/ICLIService.java 621d689 service/src/java/org/apache/hive/service/cli/session/HiveSession.java c8fb8ec service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java d6d0d27 service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java b934ebe service/src/java/org/apache/hive/service/cli/session/SessionManager.java cec3b04 service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 26bda5a service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIServiceClient.java 3675e86 service/src/test/org/apache/hive/service/auth/TestPlainSaslHelper.java 8fa4afd service/src/test/org/apache/hive/service/cli/session/TestSessionHooks.java 2fac800 shims/0.20/src/main/java/org/apache/hadoop/hive/shims/Hadoop20Shims.java 51c8051 shims/common-secure/src/main/java/org/apache/hadoop/hive/shims/HadoopShimsSecure.java e205caa shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/DelegationTokenSecretManager.java 29114f0 shims/common-secure/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java dc89de1 shims/common/src/main/java/org/apache/hadoop/hive/shims/HadoopShims.java e15ab4e shims/common/src/main/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge.java 03f4e51 Diff: https://reviews.apache.org/r/13845/diff/ Testing --- Since this requires kerberos setup, its tested by a standalone test program that runs various existing and new secure connection scenarios. The test code is attached to the ticket at https://issues.apache.org/jira/secure/attachment/12600119/ProxyAuth.java Thanks, Prasad Mujumdar
[jira] [Commented] (HIVE-6414) ParquetInputFormat provides data values that do not match the object inspectors
[
https://issues.apache.org/jira/browse/HIVE-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924095#comment-13924095
]
Justin Coffey commented on HIVE-6414:
-
hello, I don't think these are related to the patch, so resubmitting for
retesting.
ParquetInputFormat provides data values that do not match the object
inspectors
---
Key: HIVE-6414
URL: https://issues.apache.org/jira/browse/HIVE-6414
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Remus Rusanu
Assignee: Justin Coffey
Labels: Parquet
Fix For: 0.13.0
Attachments: HIVE-6414.2.patch, HIVE-6414.3.patch, HIVE-6414.3.patch,
HIVE-6414.patch
While working on HIVE-5998 I noticed that the ParquetRecordReader returns
IntWritable for all 'int like' types, in disaccord with the row object
inspectors. I though fine, and I worked my way around it. But I see now that
the issue trigger failuers in other places, eg. in aggregates:
{noformat}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row
{cint:528534767,ctinyint:31,csmallint:4963,cfloat:31.0,cdouble:4963.0,cstring1:cvLH6Eat2yFsyy7p}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:534)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177)
... 8 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast
to java.lang.Short
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:808)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:87)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:92)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:524)
... 9 more
Caused by: java.lang.ClassCastException: org.apache.hadoop.io.IntWritable
cannot be cast to java.lang.Short
at
org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaShortObjectInspector.get(JavaShortObjectInspector.java:41)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(ObjectInspectorUtils.java:671)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(ObjectInspectorUtils.java:631)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMin$GenericUDAFMinEvaluator.merge(GenericUDAFMin.java:109)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMin$GenericUDAFMinEvaluator.iterate(GenericUDAFMin.java:96)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:183)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:641)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:838)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:735)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:803)
... 15 more
{noformat}
My test is (I'm writing a test .q from HIVE-5998, but the repro does not
involve vectorization):
{noformat}
create table if not exists alltypes_parquet (
cint int,
ctinyint tinyint,
csmallint smallint,
cfloat float,
cdouble double,
cstring1 string) stored as parquet;
insert overwrite table alltypes_parquet
select cint,
ctinyint,
csmallint,
cfloat,
cdouble,
cstring1
from alltypesorc;
explain select * from alltypes_parquet limit 10; select * from
alltypes_parquet limit 10;
explain select ctinyint,
max(cint),
min(csmallint),
count(cstring1),
avg(cfloat),
stddev_pop(cdouble)
from alltypes_parquet
group by ctinyint;
select ctinyint,
max(cint),
min(csmallint),
count(cstring1),
avg(cfloat),
stddev_pop(cdouble)
from alltypes_parquet
group by ctinyint;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6414) ParquetInputFormat provides data values that do not match the object inspectors
[
https://issues.apache.org/jira/browse/HIVE-6414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Justin Coffey updated HIVE-6414:
Attachment: HIVE-6414.3.patch
ParquetInputFormat provides data values that do not match the object
inspectors
---
Key: HIVE-6414
URL: https://issues.apache.org/jira/browse/HIVE-6414
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Remus Rusanu
Assignee: Justin Coffey
Labels: Parquet
Fix For: 0.13.0
Attachments: HIVE-6414.2.patch, HIVE-6414.3.patch, HIVE-6414.3.patch,
HIVE-6414.3.patch, HIVE-6414.patch
While working on HIVE-5998 I noticed that the ParquetRecordReader returns
IntWritable for all 'int like' types, in disaccord with the row object
inspectors. I though fine, and I worked my way around it. But I see now that
the issue trigger failuers in other places, eg. in aggregates:
{noformat}
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row
{cint:528534767,ctinyint:31,csmallint:4963,cfloat:31.0,cdouble:4963.0,cstring1:cvLH6Eat2yFsyy7p}
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:534)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177)
... 8 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast
to java.lang.Short
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:808)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:87)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:92)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:790)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:524)
... 9 more
Caused by: java.lang.ClassCastException: org.apache.hadoop.io.IntWritable
cannot be cast to java.lang.Short
at
org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaShortObjectInspector.get(JavaShortObjectInspector.java:41)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(ObjectInspectorUtils.java:671)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(ObjectInspectorUtils.java:631)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMin$GenericUDAFMinEvaluator.merge(GenericUDAFMin.java:109)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMin$GenericUDAFMinEvaluator.iterate(GenericUDAFMin.java:96)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:183)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:641)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:838)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:735)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:803)
... 15 more
{noformat}
My test is (I'm writing a test .q from HIVE-5998, but the repro does not
involve vectorization):
{noformat}
create table if not exists alltypes_parquet (
cint int,
ctinyint tinyint,
csmallint smallint,
cfloat float,
cdouble double,
cstring1 string) stored as parquet;
insert overwrite table alltypes_parquet
select cint,
ctinyint,
csmallint,
cfloat,
cdouble,
cstring1
from alltypesorc;
explain select * from alltypes_parquet limit 10; select * from
alltypes_parquet limit 10;
explain select ctinyint,
max(cint),
min(csmallint),
count(cstring1),
avg(cfloat),
stddev_pop(cdouble)
from alltypes_parquet
group by ctinyint;
select ctinyint,
max(cint),
min(csmallint),
count(cstring1),
avg(cfloat),
stddev_pop(cdouble)
from alltypes_parquet
group by ctinyint;
{noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: Proposal to switch to pull requests
while (true) {
+1
}
+1 // another, just in case ;)
thanks
Prasad
On Fri, Mar 7, 2014 at 6:47 AM, kulkarni.swar...@gmail.com
kulkarni.swar...@gmail.com wrote:
+1
On Fri, Mar 7, 2014 at 1:05 AM, Thejas Nair the...@hortonworks.com
wrote:
Should we start with moving our primary source code repository from
svn to git ? I feel git is more powerful and easy to use (once you go
past the learning curve!).
On Wed, Mar 5, 2014 at 7:39 AM, Brock Noland br...@cloudera.com wrote:
Personally I prefer the Github workflow, but I believe there have been
some challenges with that since the source for apache projects must be
stored in apache source control (git or svn).
Relevent:
https://blogs.apache.org/infra/entry/improved_integration_between_apache_and
On Wed, Mar 5, 2014 at 9:19 AM, kulkarni.swar...@gmail.com
kulkarni.swar...@gmail.com wrote:
Hello,
Since we have a nice mirrored git repository for hive[1], any specific
reason why we can't switch to doing pull requests instead of patches?
IMHO
pull requests are awesome for peer review plus it is also very easy to
keep
track of JIRAs with open pull requests instead of looking for JIRAs
in a
Patch Available state. Also since they get updated automatically, it
is
also very easy to see if a review comment made by a reviewer was
addressed
properly or not.
Thoughts?
Thanks,
[1] https://github.com/apache/hive
--
Swarnim
--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity
to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified
that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender
immediately
and delete it from your system. Thank You.
--
Swarnim
[jira] [Commented] (HIVE-5155) Support secure proxy user access to HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924104#comment-13924104 ] Thejas M Nair commented on HIVE-5155: - +1 I have a question - Should we cancel the token from HiveSession.close() ? (Even if it is a good thing, we can do it in a followup jira). Support secure proxy user access to HiveServer2 --- Key: HIVE-5155 URL: https://issues.apache.org/jira/browse/HIVE-5155 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.12.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-5155-1-nothrift.patch, HIVE-5155-noThrift.2.patch, HIVE-5155-noThrift.4.patch, HIVE-5155-noThrift.5.patch, HIVE-5155-noThrift.6.patch, HIVE-5155-noThrift.7.patch, HIVE-5155-noThrift.8.patch, HIVE-5155.1.patch, HIVE-5155.2.patch, HIVE-5155.3.patch, ProxyAuth.java, ProxyAuth.out, TestKERBEROS_Hive_JDBC.java The HiveServer2 can authenticate a client using via Kerberos and impersonate the connecting user with underlying secure hadoop. This becomes a gateway for a remote client to access secure hadoop cluster. Now this works fine for when the client obtains Kerberos ticket and directly connects to HiveServer2. There's another big use case for middleware tools where the end user wants to access Hive via another server. For example Oozie action or Hue submitting queries or a BI tool server accessing to HiveServer2. In these cases, the third party server doesn't have end user's Kerberos credentials and hence it can't submit queries to HiveServer2 on behalf of the end user. This ticket is for enabling proxy access to HiveServer2 for third party tools on behalf of end users. There are two parts of the solution proposed in this ticket: 1) Delegation token based connection for Oozie (OOZIE-1457) This is the common mechanism for Hadoop ecosystem components. Hive Remote Metastore and HCatalog already support this. This is suitable for tool like Oozie that submits the MR jobs as actions on behalf of its client. Oozie already uses similar mechanism for Metastore/HCatalog access. 2) Direct proxy access for privileged hadoop users The delegation token implementation can be a challenge for non-hadoop (especially non-java) components. This second part enables a privileged user to directly specify an alternate session user during the connection. If the connecting user has hadoop level privilege to impersonate the requested userid, then HiveServer2 will run the session as that requested user. For example, user Hue is allowed to impersonate user Bob (via core-site.xml proxy user configuration). Then user Hue can connect to HiveServer2 and specify Bob as session user via a session property. HiveServer2 will verify Hue's proxy user privilege and then impersonate user Bob instead of Hue. This will enable any third party tool to impersonate alternate userid without having to implement delegation token connection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6486) Support secure Subject.doAs() in HiveServer2 JDBC client.
[
https://issues.apache.org/jira/browse/HIVE-6486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shivaraju Gowda updated HIVE-6486:
--
Release Note:
Using Kerberos with Pre-Authenticated Subject:
In the current approach of using Kerberos you will need to have valid Kerberos
ticket in the ticket cache before connecting. This will entail static
login(using kinit, key tab or ticketcache) and restriction of one Kerberos user
per client. These restrictions will limit the usage in multi-user scenarios and
in scenarios where in the client wants to login programmatically to Kerberos
KDC. Using proxy users (see https://issues.apache.org/jira/browse/HIVE-5155 )
is one way to mitigate the problem of multi-user scenarios. The other way is to
use pre-authenticated subject(see
https://issues.apache.org/jira/browse/HIVE-6486 ). In this method, Hive JDBC
client uses a pre-authenticated Kerberos Subject to authenticate to
HiveServer2. To use pre-authenticated Subject you will need the following
changes.
+ Add hive-exec*.jar to the classpath in addition to the regular Hive JDBC jars
(commons-configuration-1.6.jar and hadoop-core*.jar are not required).
+ Add auth=kerberos and kerberosAuthType=fromSubject JDBC URL properties in
addition to having the “principal url property.
+ Open the connection in Subject.doAs()
The following code snippet illustrates the usage (refer to
https://issues.apache.org/jira/browse/HIVE-6486 for complete test case):
static Connection getConnection( Subject signedOnUserSubject ) throws
Exception{
Connection conn = (Connection)
Subject.doAs(signedOnUserSubject, new PrivilegedExceptionActionObject()
{
public Object run()
{
Connection con = null;
String JDBC_DB_URL =
jdbc:hive2://HiveHost:1/default;principal=hive/localhost.localdom...@example.com;auth=kerberos;kerberosAuthType=fromSubject;
try {
Class.forName(JDBC_DRIVER);
con =
DriverManager.getConnection(JDBC_DB_URL);
} catch (SQLException e) {
e.printStackTrace();
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
return con;
}
});
return conn;
}
Support secure Subject.doAs() in HiveServer2 JDBC client.
-
Key: HIVE-6486
URL: https://issues.apache.org/jira/browse/HIVE-6486
Project: Hive
Issue Type: Improvement
Components: Authentication, HiveServer2, JDBC
Affects Versions: 0.11.0, 0.12.0
Reporter: Shivaraju Gowda
Assignee: Shivaraju Gowda
Fix For: 0.13.0
Attachments: HIVE-6486.1.patch, HIVE-6486.2.patch, HIVE-6486.3.patch,
Hive_011_Support-Subject_doAS.patch, TestHive_SujectDoAs.java
HIVE-5155 addresses the problem of kerberos authentication in multi-user
middleware server using proxy user. In this mode the principal used by the
middle ware server has privileges to impersonate selected users in
Hive/Hadoop.
This enhancement is to support Subject.doAs() authentication in Hive JDBC
layer so that the end users Kerberos Subject is passed through in the middle
ware server. With this improvement there won't be any additional setup in the
server to grant proxy privileges to some users and there won't be need to
specify a proxy user in the JDBC client. This version should also be more
secure since it won't require principals with the privileges to impersonate
other users in Hive/Hadoop setup.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6486) Support secure Subject.doAs() in HiveServer2 JDBC client.
[ https://issues.apache.org/jira/browse/HIVE-6486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924133#comment-13924133 ] Shivaraju Gowda commented on HIVE-6486: --- Thejas, Vaibhav, Prasad, Lefty Leverenz I have added the release notes, please review them at your convenience. I do NOT have privileges to edit the HiveServer2 Clients: JDBC Wiki which Lefty Leverenz pointed above. The words in the release notes are the ones I had planned to add to wiki, under the JDBC Client Setup for a Secure Cluster section, if someone wants to take it from here I would be grateful. Or else if I get the privileges I can add it myself. Support secure Subject.doAs() in HiveServer2 JDBC client. - Key: HIVE-6486 URL: https://issues.apache.org/jira/browse/HIVE-6486 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.11.0, 0.12.0 Reporter: Shivaraju Gowda Assignee: Shivaraju Gowda Fix For: 0.13.0 Attachments: HIVE-6486.1.patch, HIVE-6486.2.patch, HIVE-6486.3.patch, Hive_011_Support-Subject_doAS.patch, TestHive_SujectDoAs.java HIVE-5155 addresses the problem of kerberos authentication in multi-user middleware server using proxy user. In this mode the principal used by the middle ware server has privileges to impersonate selected users in Hive/Hadoop. This enhancement is to support Subject.doAs() authentication in Hive JDBC layer so that the end users Kerberos Subject is passed through in the middle ware server. With this improvement there won't be any additional setup in the server to grant proxy privileges to some users and there won't be need to specify a proxy user in the JDBC client. This version should also be more secure since it won't require principals with the privileges to impersonate other users in Hive/Hadoop setup. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5931) SQL std auth - add metastore get_role_participants api - to support DESCRIBE ROLE
[ https://issues.apache.org/jira/browse/HIVE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-5931: Attachment: HIVE-5931.thriftapi.3.patch SQL std auth - add metastore get_role_participants api - to support DESCRIBE ROLE - Key: HIVE-5931 URL: https://issues.apache.org/jira/browse/HIVE-5931 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Attachments: HIVE-5931.thriftapi.2.patch, HIVE-5931.thriftapi.3.patch, HIVE-5931.thriftapi.followup.patch, HIVE-5931.thriftapi.patch Original Estimate: 24h Remaining Estimate: 24h This is necessary for DESCRIBE ROLE role statement. This will list all users and roles that participate in a role. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5155) Support secure proxy user access to HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924140#comment-13924140 ] Thejas M Nair commented on HIVE-5155: - [~prasadm] Can you upload the patch with thrift generated files so that the tests can run on it ? Support secure proxy user access to HiveServer2 --- Key: HIVE-5155 URL: https://issues.apache.org/jira/browse/HIVE-5155 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.12.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-5155-1-nothrift.patch, HIVE-5155-noThrift.2.patch, HIVE-5155-noThrift.4.patch, HIVE-5155-noThrift.5.patch, HIVE-5155-noThrift.6.patch, HIVE-5155-noThrift.7.patch, HIVE-5155-noThrift.8.patch, HIVE-5155.1.patch, HIVE-5155.2.patch, HIVE-5155.3.patch, ProxyAuth.java, ProxyAuth.out, TestKERBEROS_Hive_JDBC.java The HiveServer2 can authenticate a client using via Kerberos and impersonate the connecting user with underlying secure hadoop. This becomes a gateway for a remote client to access secure hadoop cluster. Now this works fine for when the client obtains Kerberos ticket and directly connects to HiveServer2. There's another big use case for middleware tools where the end user wants to access Hive via another server. For example Oozie action or Hue submitting queries or a BI tool server accessing to HiveServer2. In these cases, the third party server doesn't have end user's Kerberos credentials and hence it can't submit queries to HiveServer2 on behalf of the end user. This ticket is for enabling proxy access to HiveServer2 for third party tools on behalf of end users. There are two parts of the solution proposed in this ticket: 1) Delegation token based connection for Oozie (OOZIE-1457) This is the common mechanism for Hadoop ecosystem components. Hive Remote Metastore and HCatalog already support this. This is suitable for tool like Oozie that submits the MR jobs as actions on behalf of its client. Oozie already uses similar mechanism for Metastore/HCatalog access. 2) Direct proxy access for privileged hadoop users The delegation token implementation can be a challenge for non-hadoop (especially non-java) components. This second part enables a privileged user to directly specify an alternate session user during the connection. If the connecting user has hadoop level privilege to impersonate the requested userid, then HiveServer2 will run the session as that requested user. For example, user Hue is allowed to impersonate user Bob (via core-site.xml proxy user configuration). Then user Hue can connect to HiveServer2 and specify Bob as session user via a session property. HiveServer2 will verify Hue's proxy user privilege and then impersonate user Bob instead of Hue. This will enable any third party tool to impersonate alternate userid without having to implement delegation token connection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6584) Add HiveHBaseTableSnapshotInputFormat
Nick Dimiduk created HIVE-6584:
--
Summary: Add HiveHBaseTableSnapshotInputFormat
Key: HIVE-6584
URL: https://issues.apache.org/jira/browse/HIVE-6584
Project: Hive
Issue Type: Improvement
Components: HBase Handler
Reporter: Nick Dimiduk
HBASE-8369 provided mapreduce support for reading from HBase table snapsopts.
This allows a MR job to consume a stable, read-only view of an HBase table
directly off of HDFS. Bypassing the online region server API provides a nice
performance boost for the full scan. HBASE-10642 is backporting that feature to
0.94/0.96 and also adding a {{mapred}} implementation. Once that's available,
we should add an input format. A follow-on patch could work out how to
integrate this functionality into the StorageHandler, similar to how HIVE-6473
integrates the HFileOutputFormat into existing table definitions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HIVE-6457) Ensure Parquet integration has good error messages for data types not supported
[ https://issues.apache.org/jira/browse/HIVE-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-6457: --- Attachment: HIVE-6457.patch Ensure Parquet integration has good error messages for data types not supported --- Key: HIVE-6457 URL: https://issues.apache.org/jira/browse/HIVE-6457 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-6457.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6457) Ensure Parquet integration has good error messages for data types not supported
[ https://issues.apache.org/jira/browse/HIVE-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-6457: --- Status: Patch Available (was: Open) Ensure Parquet integration has good error messages for data types not supported --- Key: HIVE-6457 URL: https://issues.apache.org/jira/browse/HIVE-6457 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-6457.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
Review Request 18914: HIVE-6457 - Ensure Parquet integration has good error messages for data types not supported
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18914/ --- Review request for hive, Szehon Ho and Xuefu Zhang. Bugs: HIVE-6457 https://issues.apache.org/jira/browse/HIVE-6457 Repository: hive-git Description --- Improved error messages for unsupported types. Diffs - ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ArrayWritableObjectInspector.java a2c7fe0 ql/src/test/queries/clientnegative/parquet_char.q PRE-CREATION ql/src/test/queries/clientnegative/parquet_date.q PRE-CREATION ql/src/test/queries/clientnegative/parquet_decimal.q PRE-CREATION ql/src/test/queries/clientnegative/parquet_timestamp.q PRE-CREATION ql/src/test/queries/clientnegative/parquet_varchar.q PRE-CREATION ql/src/test/results/clientnegative/parquet_char.q.out PRE-CREATION ql/src/test/results/clientnegative/parquet_date.q.out PRE-CREATION ql/src/test/results/clientnegative/parquet_decimal.q.out PRE-CREATION ql/src/test/results/clientnegative/parquet_timestamp.q.out PRE-CREATION ql/src/test/results/clientnegative/parquet_varchar.q.out PRE-CREATION Diff: https://reviews.apache.org/r/18914/diff/ Testing --- New tests pass. Thanks, Brock Noland
[jira] [Commented] (HIVE-6457) Ensure Parquet integration has good error messages for data types not supported
[ https://issues.apache.org/jira/browse/HIVE-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924163#comment-13924163 ] Brock Noland commented on HIVE-6457: https://reviews.apache.org/r/18914/ FYI [~szehon] [~xuefuz] Ensure Parquet integration has good error messages for data types not supported --- Key: HIVE-6457 URL: https://issues.apache.org/jira/browse/HIVE-6457 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-6457.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6460) Need new show functionality for transactions
[ https://issues.apache.org/jira/browse/HIVE-6460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924166#comment-13924166 ] Harish Butani commented on HIVE-6460: - +1, looks good. Couple of minor points: - TestQBCompact.java:68 is a Sys.out - As I understand this: you need Owner privilege to execute compaction; but need no privileges to view compactions. This is what you intended? Wondering if Operation2Privilege.java:173 is a cut and paste oversight. Need new show functionality for transactions -- Key: HIVE-6460 URL: https://issues.apache.org/jira/browse/HIVE-6460 Project: Hive Issue Type: Sub-task Components: SQL Reporter: Alan Gates Assignee: Alan Gates Fix For: 0.13.0 Attachments: 6460.wip.patch, HIVE-6460.1.patch, HIVE-6460.patch With the addition of transactions and compactions for delta files some new show commands are required. * show transactions to show currently open or aborted transactions * show compactions to show currently waiting or running compactions * show locks needs to work with the new db style of locks as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Timeline for the Hive 0.13 release?
HIVE-6414 seems bad enough to included. On Thu, Mar 6, 2014 at 3:28 PM, Sushanth Sowmyan khorg...@gmail.com wrote: One more, I'm afraid. I'll add it to the wiki. I just uploaded a patch for https://issues.apache.org/jira/browse/HIVE-6572 . This fixes one glaring bug on usage of mapred.min.split.size.per.rack and mapred.min.split.size.per.node, and makes sure some of the other conf parameters we use are correctly through shims. I want this to go into 0.13 because I don't want a release of ours to go out with the dual/old configs (and erroneous config in the per.rack/node case) and have that be expected behaviour to support in the future. On Thu, Mar 6, 2014 at 11:04 AM, Harish Butani hbut...@hortonworks.com wrote: ok sure. Tracking these with the JQL below. I don’t have permission to setup a Shared Filter; can someone help with this. Of the 35 issues: 11 are still open, 22 are patch available, 2 are resolved. regards, Harish. JQL: id in (HIVE-5317, HIVE-5843, HIVE-6060, HIVE-6319, HIVE-6460, HIVE-5687, HIVE-5943, HIVE-5942, HIVE-6547, HIVE-5155, HIVE-6486, HIVE-6455, HIVE-4177, HIVE-4764, HIVE-6306, HIVE-6350, HIVE-6485, HIVE-6507, HIVE-6499, HIVE-6325, HIVE-6558, HIVE-6403, HIVE-4790, HIVE-4293, HIVE-6551, HIVE-6359, HIVE-6314, HIVE-6241, HIVE-5768, HIVE-2752, HIVE-6312, HIVE-6129, HIVE-6012, HIVE-6434, HIVE-6562) ORDER BY status ASC, assignee On Mar 5, 2014, at 6:50 PM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Can you consider HIVE-6562 as well? HIVE-6562 - Protection from exceptions in ORC predicate evaluation Thanks Prasanth Jayachandran On Mar 5, 2014, at 5:56 PM, Jason Dere jd...@hortonworks.com wrote: Would like to get these in, if possible: HIVE-6012 restore backward compatibility of arithmetic operations HIVE-6434 Restrict function create/drop to admin roles On Mar 5, 2014, at 5:41 PM, Navis류승우 navis@nexr.com wrote: I have really big wish list(65 pending) but it would be time to focus on finalization. - Small bugs HIVE-6403 uncorrelated subquery is failing with auto.convert.join=true HIVE-4790 MapredLocalTask task does not make virtual columns HIVE-4293 Predicates following UDTF operator are removed by PPD - Trivials HIVE-6551 group by after join with skew join optimization references invalid task sometimes HIVE-6359 beeline -f fails on scripts with tabs in them. HIVE-6314 The logging (progress reporting) is too verbose HIVE-6241 Remove direct reference of Hadoop23Shims inQTestUtil HIVE-5768 Beeline connection cannot be closed with !close command HIVE-2752 Index names are case sensitive - Memory leakage HIVE-6312 doAs with plain sasl auth should be session aware - Implementation is not accord with document HIVE-6129 alter exchange is implemented in inverted manner I'll update the wiki, too. 2014-03-05 12:18 GMT+09:00 Harish Butani hbut...@hortonworks.com: Tracking jiras to be applied to branch 0.13 here: https://cwiki.apache.org/confluence/display/Hive/Hive+0.13+release+status On Mar 4, 2014, at 5:45 PM, Harish Butani hbut...@hortonworks.com wrote: the branch is created. have changed the poms in both branches. Planning to setup a wikipage to track jiras that will get ported to 0.13 regards, Harish. On Mar 4, 2014, at 5:05 PM, Harish Butani hbut...@hortonworks.com wrote: branching now. Will be changing the pom files on trunk. Will send another email when the branch and trunk changes are in. On Mar 4, 2014, at 4:03 PM, Sushanth Sowmyan khorg...@gmail.com wrote: I have two patches still as patch-available, that have had +1s as well, but are waiting on pre-commit tests picking them up go in to 0.13: https://issues.apache.org/jira/browse/HIVE-6507 (refactor of table property names from string constants to an enum in OrcFile) https://issues.apache.org/jira/browse/HIVE-6499 (fixes bug where calls like create table and drop table can fail if metastore-side authorization is used in conjunction with custom inputformat/outputformat/serdes that are not loadable from the metastore-side) -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential,
[jira] [Updated] (HIVE-6537) NullPointerException when loading hashtable for MapJoin directly
[
https://issues.apache.org/jira/browse/HIVE-6537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergey Shelukhin updated HIVE-6537:
---
Resolution: Fixed
Fix Version/s: 0.13.0
Status: Resolved (was: Patch Available)
committed to trunk and 13
NullPointerException when loading hashtable for MapJoin directly
Key: HIVE-6537
URL: https://issues.apache.org/jira/browse/HIVE-6537
Project: Hive
Issue Type: Bug
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Fix For: 0.13.0
Attachments: HIVE-6537.01.patch, HIVE-6537.2.patch.txt,
HIVE-6537.patch
We see the following error:
{noformat}
2014-02-20 23:33:15,743 FATAL [main]
org.apache.hadoop.hive.ql.exec.mr.ExecMapper:
org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.NullPointerException
at
org.apache.hadoop.hive.ql.exec.mr.HashTableLoader.load(HashTableLoader.java:103)
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:149)
at
org.apache.hadoop.hive.ql.exec.MapJoinOperator.cleanUpInputFileChangedOp(MapJoinOperator.java:164)
at
org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1026)
at
org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1030)
at
org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1030)
at
org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:489)
at
org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:177)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:430)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:342)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.NullPointerException
at java.util.Arrays.fill(Arrays.java:2685)
at
org.apache.hadoop.hive.ql.exec.mr.HashTableLoader.loadDirectly(HashTableLoader.java:155)
at
org.apache.hadoop.hive.ql.exec.mr.HashTableLoader.load(HashTableLoader.java:81)
... 15 more
{noformat}
It appears that the tables in Arrays.fill call is nulls. I don't really have
full understanding of this path, but what I gleaned so far is this...
From what I see, tables would be set unconditionally in initializeOp of the
sink, and in no other place, so I assume for this code to ever work that
startForward calls it at least some time.
Here, it doesn't call it, so it's null.
Previous loop also uses tables, and should have NPE-d before fill was ever
called; it didn't, so I'd assume it never executed.
There's a little bit of inconsistency in the above code where directWorks are
added to parents unconditionally but sink is only added as child
conditionally. I think it may be that some of the direct works are not table
scans; in fact given that loop never executes they may be null (which is
rather strange).
Regardless, it seems that the logic should be fixed, it may be the root cause
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6417) sql std auth - new users in admin role config should get added
[ https://issues.apache.org/jira/browse/HIVE-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924171#comment-13924171 ] Harish Butani commented on HIVE-6417: - yes; do you want me to commit this to 0.13 branch? sql std auth - new users in admin role config should get added -- Key: HIVE-6417 URL: https://issues.apache.org/jira/browse/HIVE-6417 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Ashutosh Chauhan Fix For: 0.14.0 Attachments: HIVE-6417.1.patch, HIVE-6417.patch if metastore is started with hive.users.in.admin.role=user1, then user1 is added admin role to metastore. If the value is changed to hive.users.in.admin.role=user2, then user2 should get added to the role in metastore. Right now, if the admin role exists, new users don't get added. A work-around is - user1 adding user2 to the admin role using grant role statement. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6460) Need new show functionality for transactions
[ https://issues.apache.org/jira/browse/HIVE-6460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924177#comment-13924177 ] Alan Gates commented on HIVE-6460: -- Thanks for the review. I'll remove the overlooked debugging System.out. bq. you need Owner privilege to execute compaction; but need no privileges to view compactions. This is what you intended? It is what I intended, though I'm open to feedback that this isn't right. My thinking was that only owners (and admins) should be able to kick off compactions, but there isn't harm in seeing what compactions are going on in the system. I assume in general it will be admins that will use show compactions to determine what the system is auto compacting at any given time. Also, the other show commands didn't require any authorization so I assumed this one should not either. If people think this doesn't make sense we can file a follow on bug to change the permissions on show compactions. Need new show functionality for transactions -- Key: HIVE-6460 URL: https://issues.apache.org/jira/browse/HIVE-6460 Project: Hive Issue Type: Sub-task Components: SQL Reporter: Alan Gates Assignee: Alan Gates Fix For: 0.13.0 Attachments: 6460.wip.patch, HIVE-6460.1.patch, HIVE-6460.patch With the addition of transactions and compactions for delta files some new show commands are required. * show transactions to show currently open or aborted transactions * show compactions to show currently waiting or running compactions * show locks needs to work with the new db style of locks as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6417) sql std auth - new users in admin role config should get added
[ https://issues.apache.org/jira/browse/HIVE-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924180#comment-13924180 ] Ashutosh Chauhan commented on HIVE-6417: I can do it. No worries! sql std auth - new users in admin role config should get added -- Key: HIVE-6417 URL: https://issues.apache.org/jira/browse/HIVE-6417 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Ashutosh Chauhan Fix For: 0.14.0 Attachments: HIVE-6417.1.patch, HIVE-6417.patch if metastore is started with hive.users.in.admin.role=user1, then user1 is added admin role to metastore. If the value is changed to hive.users.in.admin.role=user2, then user2 should get added to the role in metastore. Right now, if the admin role exists, new users don't get added. A work-around is - user1 adding user2 to the admin role using grant role statement. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Timeline for the Hive 0.13 release?
ok adding 6414, 6572, 6574 to the list On Mar 7, 2014, at 10:27 AM, Xuefu Zhang xzh...@cloudera.com wrote: HIVE-6414 seems bad enough to included. On Thu, Mar 6, 2014 at 3:28 PM, Sushanth Sowmyan khorg...@gmail.com wrote: One more, I'm afraid. I'll add it to the wiki. I just uploaded a patch for https://issues.apache.org/jira/browse/HIVE-6572 . This fixes one glaring bug on usage of mapred.min.split.size.per.rack and mapred.min.split.size.per.node, and makes sure some of the other conf parameters we use are correctly through shims. I want this to go into 0.13 because I don't want a release of ours to go out with the dual/old configs (and erroneous config in the per.rack/node case) and have that be expected behaviour to support in the future. On Thu, Mar 6, 2014 at 11:04 AM, Harish Butani hbut...@hortonworks.com wrote: ok sure. Tracking these with the JQL below. I don’t have permission to setup a Shared Filter; can someone help with this. Of the 35 issues: 11 are still open, 22 are patch available, 2 are resolved. regards, Harish. JQL: id in (HIVE-5317, HIVE-5843, HIVE-6060, HIVE-6319, HIVE-6460, HIVE-5687, HIVE-5943, HIVE-5942, HIVE-6547, HIVE-5155, HIVE-6486, HIVE-6455, HIVE-4177, HIVE-4764, HIVE-6306, HIVE-6350, HIVE-6485, HIVE-6507, HIVE-6499, HIVE-6325, HIVE-6558, HIVE-6403, HIVE-4790, HIVE-4293, HIVE-6551, HIVE-6359, HIVE-6314, HIVE-6241, HIVE-5768, HIVE-2752, HIVE-6312, HIVE-6129, HIVE-6012, HIVE-6434, HIVE-6562) ORDER BY status ASC, assignee On Mar 5, 2014, at 6:50 PM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Can you consider HIVE-6562 as well? HIVE-6562 - Protection from exceptions in ORC predicate evaluation Thanks Prasanth Jayachandran On Mar 5, 2014, at 5:56 PM, Jason Dere jd...@hortonworks.com wrote: Would like to get these in, if possible: HIVE-6012 restore backward compatibility of arithmetic operations HIVE-6434 Restrict function create/drop to admin roles On Mar 5, 2014, at 5:41 PM, Navis류승우 navis@nexr.com wrote: I have really big wish list(65 pending) but it would be time to focus on finalization. - Small bugs HIVE-6403 uncorrelated subquery is failing with auto.convert.join=true HIVE-4790 MapredLocalTask task does not make virtual columns HIVE-4293 Predicates following UDTF operator are removed by PPD - Trivials HIVE-6551 group by after join with skew join optimization references invalid task sometimes HIVE-6359 beeline -f fails on scripts with tabs in them. HIVE-6314 The logging (progress reporting) is too verbose HIVE-6241 Remove direct reference of Hadoop23Shims inQTestUtil HIVE-5768 Beeline connection cannot be closed with !close command HIVE-2752 Index names are case sensitive - Memory leakage HIVE-6312 doAs with plain sasl auth should be session aware - Implementation is not accord with document HIVE-6129 alter exchange is implemented in inverted manner I'll update the wiki, too. 2014-03-05 12:18 GMT+09:00 Harish Butani hbut...@hortonworks.com: Tracking jiras to be applied to branch 0.13 here: https://cwiki.apache.org/confluence/display/Hive/Hive+0.13+release+status On Mar 4, 2014, at 5:45 PM, Harish Butani hbut...@hortonworks.com wrote: the branch is created. have changed the poms in both branches. Planning to setup a wikipage to track jiras that will get ported to 0.13 regards, Harish. On Mar 4, 2014, at 5:05 PM, Harish Butani hbut...@hortonworks.com wrote: branching now. Will be changing the pom files on trunk. Will send another email when the branch and trunk changes are in. On Mar 4, 2014, at 4:03 PM, Sushanth Sowmyan khorg...@gmail.com wrote: I have two patches still as patch-available, that have had +1s as well, but are waiting on pre-commit tests picking them up go in to 0.13: https://issues.apache.org/jira/browse/HIVE-6507 (refactor of table property names from string constants to an enum in OrcFile) https://issues.apache.org/jira/browse/HIVE-6499 (fixes bug where calls like create table and drop table can fail if metastore-side authorization is used in conjunction with custom inputformat/outputformat/serdes that are not loadable from the metastore-side) -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may
Re: Timeline for the Hive 0.13 release?
It would be great to have this in - HIVE-5901 (driver cancel to kill the MR job) has been +1, waiting for tests to finish. HIVE-5099 (datanucleues version upgrade) +1'd, waiting for tests to finish. On Thu, Mar 6, 2014 at 3:28 PM, Sushanth Sowmyan khorg...@gmail.com wrote: One more, I'm afraid. I'll add it to the wiki. I just uploaded a patch for https://issues.apache.org/jira/browse/HIVE-6572 . This fixes one glaring bug on usage of mapred.min.split.size.per.rack and mapred.min.split.size.per.node, and makes sure some of the other conf parameters we use are correctly through shims. I want this to go into 0.13 because I don't want a release of ours to go out with the dual/old configs (and erroneous config in the per.rack/node case) and have that be expected behaviour to support in the future. On Thu, Mar 6, 2014 at 11:04 AM, Harish Butani hbut...@hortonworks.com wrote: ok sure. Tracking these with the JQL below. I don’t have permission to setup a Shared Filter; can someone help with this. Of the 35 issues: 11 are still open, 22 are patch available, 2 are resolved. regards, Harish. JQL: id in (HIVE-5317, HIVE-5843, HIVE-6060, HIVE-6319, HIVE-6460, HIVE-5687, HIVE-5943, HIVE-5942, HIVE-6547, HIVE-5155, HIVE-6486, HIVE-6455, HIVE-4177, HIVE-4764, HIVE-6306, HIVE-6350, HIVE-6485, HIVE-6507, HIVE-6499, HIVE-6325, HIVE-6558, HIVE-6403, HIVE-4790, HIVE-4293, HIVE-6551, HIVE-6359, HIVE-6314, HIVE-6241, HIVE-5768, HIVE-2752, HIVE-6312, HIVE-6129, HIVE-6012, HIVE-6434, HIVE-6562) ORDER BY status ASC, assignee On Mar 5, 2014, at 6:50 PM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Can you consider HIVE-6562 as well? HIVE-6562 - Protection from exceptions in ORC predicate evaluation Thanks Prasanth Jayachandran On Mar 5, 2014, at 5:56 PM, Jason Dere jd...@hortonworks.com wrote: Would like to get these in, if possible: HIVE-6012 restore backward compatibility of arithmetic operations HIVE-6434 Restrict function create/drop to admin roles On Mar 5, 2014, at 5:41 PM, Navis류승우 navis@nexr.com wrote: I have really big wish list(65 pending) but it would be time to focus on finalization. - Small bugs HIVE-6403 uncorrelated subquery is failing with auto.convert.join=true HIVE-4790 MapredLocalTask task does not make virtual columns HIVE-4293 Predicates following UDTF operator are removed by PPD - Trivials HIVE-6551 group by after join with skew join optimization references invalid task sometimes HIVE-6359 beeline -f fails on scripts with tabs in them. HIVE-6314 The logging (progress reporting) is too verbose HIVE-6241 Remove direct reference of Hadoop23Shims inQTestUtil HIVE-5768 Beeline connection cannot be closed with !close command HIVE-2752 Index names are case sensitive - Memory leakage HIVE-6312 doAs with plain sasl auth should be session aware - Implementation is not accord with document HIVE-6129 alter exchange is implemented in inverted manner I'll update the wiki, too. 2014-03-05 12:18 GMT+09:00 Harish Butani hbut...@hortonworks.com: Tracking jiras to be applied to branch 0.13 here: https://cwiki.apache.org/confluence/display/Hive/Hive+0.13+release+status On Mar 4, 2014, at 5:45 PM, Harish Butani hbut...@hortonworks.com wrote: the branch is created. have changed the poms in both branches. Planning to setup a wikipage to track jiras that will get ported to 0.13 regards, Harish. On Mar 4, 2014, at 5:05 PM, Harish Butani hbut...@hortonworks.com wrote: branching now. Will be changing the pom files on trunk. Will send another email when the branch and trunk changes are in. On Mar 4, 2014, at 4:03 PM, Sushanth Sowmyan khorg...@gmail.com wrote: I have two patches still as patch-available, that have had +1s as well, but are waiting on pre-commit tests picking them up go in to 0.13: https://issues.apache.org/jira/browse/HIVE-6507 (refactor of table property names from string constants to an enum in OrcFile) https://issues.apache.org/jira/browse/HIVE-6499 (fixes bug where calls like create table and drop table can fail if metastore-side authorization is used in conjunction with custom inputformat/outputformat/serdes that are not loadable from the metastore-side) -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may
[jira] [Updated] (HIVE-6147) Support avro data stored in HBase columns
[ https://issues.apache.org/jira/browse/HIVE-6147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Swarnim Kulkarni updated HIVE-6147: --- Attachment: HIVE-6147.4.patch.txt So I cleaned up my local m2 repository, rebased the master and re-applied the patch and saw the test failure. May be it was a bad dep sitting in my repo that was causing the problem. Attaching a new patch. If the tests pass with this, I will update the RB with this latest patch. Support avro data stored in HBase columns - Key: HIVE-6147 URL: https://issues.apache.org/jira/browse/HIVE-6147 Project: Hive Issue Type: Bug Components: HBase Handler Affects Versions: 0.12.0 Reporter: Swarnim Kulkarni Assignee: Swarnim Kulkarni Attachments: HIVE-6147.1.patch.txt, HIVE-6147.2.patch.txt, HIVE-6147.3.patch.txt, HIVE-6147.3.patch.txt, HIVE-6147.4.patch.txt Presently, the HBase Hive integration supports querying only primitive data types in columns. It would be nice to be able to store and query Avro objects in HBase columns by making them visible as structs to Hive. This will allow Hive to perform ad hoc analysis of HBase data which can be deeply structured. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Timeline for the Hive 0.13 release?
We should get 5567 and 5568 applied too. They are a bit stale, but I'll refresh them next week. On Fri, Mar 7, 2014 at 10:41 AM, Harish Butani hbut...@hortonworks.comwrote: ok adding 6414, 6572, 6574 to the list On Mar 7, 2014, at 10:27 AM, Xuefu Zhang xzh...@cloudera.com wrote: HIVE-6414 seems bad enough to included. On Thu, Mar 6, 2014 at 3:28 PM, Sushanth Sowmyan khorg...@gmail.com wrote: One more, I'm afraid. I'll add it to the wiki. I just uploaded a patch for https://issues.apache.org/jira/browse/HIVE-6572 . This fixes one glaring bug on usage of mapred.min.split.size.per.rack and mapred.min.split.size.per.node, and makes sure some of the other conf parameters we use are correctly through shims. I want this to go into 0.13 because I don't want a release of ours to go out with the dual/old configs (and erroneous config in the per.rack/node case) and have that be expected behaviour to support in the future. On Thu, Mar 6, 2014 at 11:04 AM, Harish Butani hbut...@hortonworks.com wrote: ok sure. Tracking these with the JQL below. I don’t have permission to setup a Shared Filter; can someone help with this. Of the 35 issues: 11 are still open, 22 are patch available, 2 are resolved. regards, Harish. JQL: id in (HIVE-5317, HIVE-5843, HIVE-6060, HIVE-6319, HIVE-6460, HIVE-5687, HIVE-5943, HIVE-5942, HIVE-6547, HIVE-5155, HIVE-6486, HIVE-6455, HIVE-4177, HIVE-4764, HIVE-6306, HIVE-6350, HIVE-6485, HIVE-6507, HIVE-6499, HIVE-6325, HIVE-6558, HIVE-6403, HIVE-4790, HIVE-4293, HIVE-6551, HIVE-6359, HIVE-6314, HIVE-6241, HIVE-5768, HIVE-2752, HIVE-6312, HIVE-6129, HIVE-6012, HIVE-6434, HIVE-6562) ORDER BY status ASC, assignee On Mar 5, 2014, at 6:50 PM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Can you consider HIVE-6562 as well? HIVE-6562 - Protection from exceptions in ORC predicate evaluation Thanks Prasanth Jayachandran On Mar 5, 2014, at 5:56 PM, Jason Dere jd...@hortonworks.com wrote: Would like to get these in, if possible: HIVE-6012 restore backward compatibility of arithmetic operations HIVE-6434 Restrict function create/drop to admin roles On Mar 5, 2014, at 5:41 PM, Navis류승우 navis@nexr.com wrote: I have really big wish list(65 pending) but it would be time to focus on finalization. - Small bugs HIVE-6403 uncorrelated subquery is failing with auto.convert.join=true HIVE-4790 MapredLocalTask task does not make virtual columns HIVE-4293 Predicates following UDTF operator are removed by PPD - Trivials HIVE-6551 group by after join with skew join optimization references invalid task sometimes HIVE-6359 beeline -f fails on scripts with tabs in them. HIVE-6314 The logging (progress reporting) is too verbose HIVE-6241 Remove direct reference of Hadoop23Shims inQTestUtil HIVE-5768 Beeline connection cannot be closed with !close command HIVE-2752 Index names are case sensitive - Memory leakage HIVE-6312 doAs with plain sasl auth should be session aware - Implementation is not accord with document HIVE-6129 alter exchange is implemented in inverted manner I'll update the wiki, too. 2014-03-05 12:18 GMT+09:00 Harish Butani hbut...@hortonworks.com: Tracking jiras to be applied to branch 0.13 here: https://cwiki.apache.org/confluence/display/Hive/Hive+0.13+release+status On Mar 4, 2014, at 5:45 PM, Harish Butani hbut...@hortonworks.com wrote: the branch is created. have changed the poms in both branches. Planning to setup a wikipage to track jiras that will get ported to 0.13 regards, Harish. On Mar 4, 2014, at 5:05 PM, Harish Butani hbut...@hortonworks.com wrote: branching now. Will be changing the pom files on trunk. Will send another email when the branch and trunk changes are in. On Mar 4, 2014, at 4:03 PM, Sushanth Sowmyan khorg...@gmail.com wrote: I have two patches still as patch-available, that have had +1s as well, but are waiting on pre-commit tests picking them up go in to 0.13: https://issues.apache.org/jira/browse/HIVE-6507 (refactor of table property names from string constants to an enum in OrcFile) https://issues.apache.org/jira/browse/HIVE-6499 (fixes bug where calls like create table and drop table can fail if metastore-side authorization is used in conjunction with custom inputformat/outputformat/serdes that are not loadable from the metastore-side) -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly
[jira] [Commented] (HIVE-6573) Oracle metastore doesnt come up when hive.cluster.delegation.token.store.class is set to DBTokenStore
[
https://issues.apache.org/jira/browse/HIVE-6573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924189#comment-13924189
]
Thejas M Nair commented on HIVE-6573:
-
+1
Oracle metastore doesnt come up when
hive.cluster.delegation.token.store.class is set to DBTokenStore
-
Key: HIVE-6573
URL: https://issues.apache.org/jira/browse/HIVE-6573
Project: Hive
Issue Type: Bug
Components: Metastore, Security
Affects Versions: 0.12.0
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
Priority: Blocker
Fix For: 0.13.0
Attachments: HIVE-6573.patch
This config {{hive.cluster.delegation.token.store.class}} was introduced in
HIVE-3255 and is useful only if oracle metastore is used in secure setup with
HA config.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6460) Need new show functionality for transactions
[ https://issues.apache.org/jira/browse/HIVE-6460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924210#comment-13924210 ] Harish Butani commented on HIVE-6460: - no don't have a strong opinion on this. If other show commands have this behavior, good enough reason to do the same here. I pointed it out in case it was an oversight; happens to me all the time. Need new show functionality for transactions -- Key: HIVE-6460 URL: https://issues.apache.org/jira/browse/HIVE-6460 Project: Hive Issue Type: Sub-task Components: SQL Reporter: Alan Gates Assignee: Alan Gates Fix For: 0.13.0 Attachments: 6460.wip.patch, HIVE-6460.1.patch, HIVE-6460.patch With the addition of transactions and compactions for delta files some new show commands are required. * show transactions to show currently open or aborted transactions * show compactions to show currently waiting or running compactions * show locks needs to work with the new db style of locks as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Timeline for the Hive 0.13 release?
HIVE-6576 please. w/o there is a backwards incompatible change in webhcat REST api On Fri, Mar 7, 2014 at 10:45 AM, Owen O'Malley omal...@apache.org wrote: We should get 5567 and 5568 applied too. They are a bit stale, but I'll refresh them next week. On Fri, Mar 7, 2014 at 10:41 AM, Harish Butani hbut...@hortonworks.com wrote: ok adding 6414, 6572, 6574 to the list On Mar 7, 2014, at 10:27 AM, Xuefu Zhang xzh...@cloudera.com wrote: HIVE-6414 seems bad enough to included. On Thu, Mar 6, 2014 at 3:28 PM, Sushanth Sowmyan khorg...@gmail.com wrote: One more, I'm afraid. I'll add it to the wiki. I just uploaded a patch for https://issues.apache.org/jira/browse/HIVE-6572 . This fixes one glaring bug on usage of mapred.min.split.size.per.rack and mapred.min.split.size.per.node, and makes sure some of the other conf parameters we use are correctly through shims. I want this to go into 0.13 because I don't want a release of ours to go out with the dual/old configs (and erroneous config in the per.rack/node case) and have that be expected behaviour to support in the future. On Thu, Mar 6, 2014 at 11:04 AM, Harish Butani hbut...@hortonworks.com wrote: ok sure. Tracking these with the JQL below. I don’t have permission to setup a Shared Filter; can someone help with this. Of the 35 issues: 11 are still open, 22 are patch available, 2 are resolved. regards, Harish. JQL: id in (HIVE-5317, HIVE-5843, HIVE-6060, HIVE-6319, HIVE-6460, HIVE-5687, HIVE-5943, HIVE-5942, HIVE-6547, HIVE-5155, HIVE-6486, HIVE-6455, HIVE-4177, HIVE-4764, HIVE-6306, HIVE-6350, HIVE-6485, HIVE-6507, HIVE-6499, HIVE-6325, HIVE-6558, HIVE-6403, HIVE-4790, HIVE-4293, HIVE-6551, HIVE-6359, HIVE-6314, HIVE-6241, HIVE-5768, HIVE-2752, HIVE-6312, HIVE-6129, HIVE-6012, HIVE-6434, HIVE-6562) ORDER BY status ASC, assignee On Mar 5, 2014, at 6:50 PM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Can you consider HIVE-6562 as well? HIVE-6562 - Protection from exceptions in ORC predicate evaluation Thanks Prasanth Jayachandran On Mar 5, 2014, at 5:56 PM, Jason Dere jd...@hortonworks.com wrote: Would like to get these in, if possible: HIVE-6012 restore backward compatibility of arithmetic operations HIVE-6434 Restrict function create/drop to admin roles On Mar 5, 2014, at 5:41 PM, Navis류승우 navis@nexr.com wrote: I have really big wish list(65 pending) but it would be time to focus on finalization. - Small bugs HIVE-6403 uncorrelated subquery is failing with auto.convert.join=true HIVE-4790 MapredLocalTask task does not make virtual columns HIVE-4293 Predicates following UDTF operator are removed by PPD - Trivials HIVE-6551 group by after join with skew join optimization references invalid task sometimes HIVE-6359 beeline -f fails on scripts with tabs in them. HIVE-6314 The logging (progress reporting) is too verbose HIVE-6241 Remove direct reference of Hadoop23Shims inQTestUtil HIVE-5768 Beeline connection cannot be closed with !close command HIVE-2752 Index names are case sensitive - Memory leakage HIVE-6312 doAs with plain sasl auth should be session aware - Implementation is not accord with document HIVE-6129 alter exchange is implemented in inverted manner I'll update the wiki, too. 2014-03-05 12:18 GMT+09:00 Harish Butani hbut...@hortonworks.com : Tracking jiras to be applied to branch 0.13 here: https://cwiki.apache.org/confluence/display/Hive/Hive+0.13+release+status On Mar 4, 2014, at 5:45 PM, Harish Butani hbut...@hortonworks.com wrote: the branch is created. have changed the poms in both branches. Planning to setup a wikipage to track jiras that will get ported to 0.13 regards, Harish. On Mar 4, 2014, at 5:05 PM, Harish Butani hbut...@hortonworks.com wrote: branching now. Will be changing the pom files on trunk. Will send another email when the branch and trunk changes are in. On Mar 4, 2014, at 4:03 PM, Sushanth Sowmyan khorg...@gmail.com wrote: I have two patches still as patch-available, that have had +1s as well, but are waiting on pre-commit tests picking them up go in to 0.13: https://issues.apache.org/jira/browse/HIVE-6507 (refactor of table property names from string constants to an enum in OrcFile) https://issues.apache.org/jira/browse/HIVE-6499 (fixes bug where calls like create table and drop table can fail if metastore-side authorization is used in conjunction with custom inputformat/outputformat/serdes that are not loadable from the metastore-side) -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is
[jira] [Commented] (HIVE-5155) Support secure proxy user access to HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924220#comment-13924220 ] Prasad Mujumdar commented on HIVE-5155: --- [~thejas] and [~vaibhavgumashta] Thanks for the review I agree that it's good to cancel the token at the end of the session. There's a GC thread that removes the expired tokens, but the default interval is 1day. I will log a followup ticket and submit a patch. Support secure proxy user access to HiveServer2 --- Key: HIVE-5155 URL: https://issues.apache.org/jira/browse/HIVE-5155 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.12.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-5155-1-nothrift.patch, HIVE-5155-noThrift.2.patch, HIVE-5155-noThrift.4.patch, HIVE-5155-noThrift.5.patch, HIVE-5155-noThrift.6.patch, HIVE-5155-noThrift.7.patch, HIVE-5155-noThrift.8.patch, HIVE-5155.1.patch, HIVE-5155.2.patch, HIVE-5155.3.patch, ProxyAuth.java, ProxyAuth.out, TestKERBEROS_Hive_JDBC.java The HiveServer2 can authenticate a client using via Kerberos and impersonate the connecting user with underlying secure hadoop. This becomes a gateway for a remote client to access secure hadoop cluster. Now this works fine for when the client obtains Kerberos ticket and directly connects to HiveServer2. There's another big use case for middleware tools where the end user wants to access Hive via another server. For example Oozie action or Hue submitting queries or a BI tool server accessing to HiveServer2. In these cases, the third party server doesn't have end user's Kerberos credentials and hence it can't submit queries to HiveServer2 on behalf of the end user. This ticket is for enabling proxy access to HiveServer2 for third party tools on behalf of end users. There are two parts of the solution proposed in this ticket: 1) Delegation token based connection for Oozie (OOZIE-1457) This is the common mechanism for Hadoop ecosystem components. Hive Remote Metastore and HCatalog already support this. This is suitable for tool like Oozie that submits the MR jobs as actions on behalf of its client. Oozie already uses similar mechanism for Metastore/HCatalog access. 2) Direct proxy access for privileged hadoop users The delegation token implementation can be a challenge for non-hadoop (especially non-java) components. This second part enables a privileged user to directly specify an alternate session user during the connection. If the connecting user has hadoop level privilege to impersonate the requested userid, then HiveServer2 will run the session as that requested user. For example, user Hue is allowed to impersonate user Bob (via core-site.xml proxy user configuration). Then user Hue can connect to HiveServer2 and specify Bob as session user via a session property. HiveServer2 will verify Hue's proxy user privilege and then impersonate user Bob instead of Hue. This will enable any third party tool to impersonate alternate userid without having to implement delegation token connection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-5155) Support secure proxy user access to HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasad Mujumdar updated HIVE-5155: -- Attachment: HIVE-5155.4.patch Full patch with thrift generated code. Rebased with trunk, had a conflict in hive-default.xml.template that's is resolved. Support secure proxy user access to HiveServer2 --- Key: HIVE-5155 URL: https://issues.apache.org/jira/browse/HIVE-5155 Project: Hive Issue Type: Improvement Components: Authentication, HiveServer2, JDBC Affects Versions: 0.12.0 Reporter: Prasad Mujumdar Assignee: Prasad Mujumdar Attachments: HIVE-5155-1-nothrift.patch, HIVE-5155-noThrift.2.patch, HIVE-5155-noThrift.4.patch, HIVE-5155-noThrift.5.patch, HIVE-5155-noThrift.6.patch, HIVE-5155-noThrift.7.patch, HIVE-5155-noThrift.8.patch, HIVE-5155.1.patch, HIVE-5155.2.patch, HIVE-5155.3.patch, HIVE-5155.4.patch, ProxyAuth.java, ProxyAuth.out, TestKERBEROS_Hive_JDBC.java The HiveServer2 can authenticate a client using via Kerberos and impersonate the connecting user with underlying secure hadoop. This becomes a gateway for a remote client to access secure hadoop cluster. Now this works fine for when the client obtains Kerberos ticket and directly connects to HiveServer2. There's another big use case for middleware tools where the end user wants to access Hive via another server. For example Oozie action or Hue submitting queries or a BI tool server accessing to HiveServer2. In these cases, the third party server doesn't have end user's Kerberos credentials and hence it can't submit queries to HiveServer2 on behalf of the end user. This ticket is for enabling proxy access to HiveServer2 for third party tools on behalf of end users. There are two parts of the solution proposed in this ticket: 1) Delegation token based connection for Oozie (OOZIE-1457) This is the common mechanism for Hadoop ecosystem components. Hive Remote Metastore and HCatalog already support this. This is suitable for tool like Oozie that submits the MR jobs as actions on behalf of its client. Oozie already uses similar mechanism for Metastore/HCatalog access. 2) Direct proxy access for privileged hadoop users The delegation token implementation can be a challenge for non-hadoop (especially non-java) components. This second part enables a privileged user to directly specify an alternate session user during the connection. If the connecting user has hadoop level privilege to impersonate the requested userid, then HiveServer2 will run the session as that requested user. For example, user Hue is allowed to impersonate user Bob (via core-site.xml proxy user configuration). Then user Hue can connect to HiveServer2 and specify Bob as session user via a session property. HiveServer2 will verify Hue's proxy user privilege and then impersonate user Bob instead of Hue. This will enable any third party tool to impersonate alternate userid without having to implement delegation token connection. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HIVE-6585) bucket map join fails in presence of _SUCCESS file
Ashutosh Chauhan created HIVE-6585: -- Summary: bucket map join fails in presence of _SUCCESS file Key: HIVE-6585 URL: https://issues.apache.org/jira/browse/HIVE-6585 Project: Hive Issue Type: Bug Components: File Formats Affects Versions: 0.12.0, 0.13.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Reason is missing path filters. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6585) bucket map join fails in presence of _SUCCESS file
[ https://issues.apache.org/jira/browse/HIVE-6585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6585: --- Attachment: HIVE-6585.patch Simple fix. bucket map join fails in presence of _SUCCESS file -- Key: HIVE-6585 URL: https://issues.apache.org/jira/browse/HIVE-6585 Project: Hive Issue Type: Bug Components: File Formats Affects Versions: 0.12.0, 0.13.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-6585.patch Reason is missing path filters. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6585) bucket map join fails in presence of _SUCCESS file
[ https://issues.apache.org/jira/browse/HIVE-6585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6585: --- Status: Patch Available (was: Open) bucket map join fails in presence of _SUCCESS file -- Key: HIVE-6585 URL: https://issues.apache.org/jira/browse/HIVE-6585 Project: Hive Issue Type: Bug Components: File Formats Affects Versions: 0.12.0, 0.13.0 Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Attachments: HIVE-6585.patch Reason is missing path filters. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6555) TestSchemaTool is failing on trunk after branching
[
https://issues.apache.org/jira/browse/HIVE-6555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924229#comment-13924229
]
Hive QA commented on HIVE-6555:
---
{color:red}Overall{color}: -1 at least one tests failed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633238/HIVE-6555.1.patch
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 5360 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testNegativeCliDriver_mapreduce_stack_trace_hadoop20
{noformat}
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1648/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1648/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633238
TestSchemaTool is failing on trunk after branching
--
Key: HIVE-6555
URL: https://issues.apache.org/jira/browse/HIVE-6555
Project: Hive
Issue Type: Bug
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan
Attachments: HIVE-6555-branch13.patch, HIVE-6555.1.patch,
HIVE-6555.patch
This is because version was bumped to 0.14 in pom file and there are no
metastore scripts for 0.14 yet.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: Review Request 18808: Casting from decimal to tinyint, smallint, int and bigint generates different result when vectorization is on
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18808/#review36558 --- Ship it! looks good to me! common/src/java/org/apache/hadoop/hive/common/type/Decimal128.java https://reviews.apache.org/r/18808/#comment67561 Can you open a bug for scaleDownTenDestructive, based on what you found? You can make it low priority since it is not getting called in the current code paths. But it will be good to have a record of it. - Eric Hanson On March 7, 2014, 4:39 a.m., Jitendra Pandey wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18808/ --- (Updated March 7, 2014, 4:39 a.m.) Review request for hive and Eric Hanson. Bugs: HIVE-6511 https://issues.apache.org/jira/browse/HIVE-6511 Repository: hive-git Description --- Casting from decimal to tinyint,smallint, int and bigint generates different result when vectorization is on. Diffs - common/src/java/org/apache/hadoop/hive/common/type/Decimal128.java a5d7399 common/src/test/org/apache/hadoop/hive/common/type/TestDecimal128.java 426c03d Diff: https://reviews.apache.org/r/18808/diff/ Testing --- Thanks, Jitendra Pandey
Review Request 18916: BMJ fails if tbl dir contains _SUCCESS file
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18916/ --- Review request for hive. Bugs: HIVE-6585 https://issues.apache.org/jira/browse/HIVE-6585 Repository: hive-git Description --- Need to path filters. Diffs - common/src/java/org/apache/hadoop/hive/common/FileUtils.java c3a2cd4 ql/src/java/org/apache/hadoop/hive/ql/io/BucketizedHiveInputFormat.java 520590f ql/src/test/queries/clientpositive/bucket_if_with_path_filter.q PRE-CREATION ql/src/test/results/clientpositive/bucket_if_with_path_filter.q.out PRE-CREATION Diff: https://reviews.apache.org/r/18916/diff/ Testing --- Added test case. Thanks, Ashutosh Chauhan
Re: Review Request 18914: HIVE-6457 - Ensure Parquet integration has good error messages for data types not supported
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18914/#review36559 --- Thanks Brock for the change! Just a minor comment. ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ArrayWritableObjectInspector.java https://reviews.apache.org/r/18914/#comment67562 Looks like there is a class called 'serdeConstants' that has typeName finals, that we can use for these hard-coded strings? - Szehon Ho On March 7, 2014, 6:25 p.m., Brock Noland wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18914/ --- (Updated March 7, 2014, 6:25 p.m.) Review request for hive, Szehon Ho and Xuefu Zhang. Bugs: HIVE-6457 https://issues.apache.org/jira/browse/HIVE-6457 Repository: hive-git Description --- Improved error messages for unsupported types. Diffs - ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ArrayWritableObjectInspector.java a2c7fe0 ql/src/test/queries/clientnegative/parquet_char.q PRE-CREATION ql/src/test/queries/clientnegative/parquet_date.q PRE-CREATION ql/src/test/queries/clientnegative/parquet_decimal.q PRE-CREATION ql/src/test/queries/clientnegative/parquet_timestamp.q PRE-CREATION ql/src/test/queries/clientnegative/parquet_varchar.q PRE-CREATION ql/src/test/results/clientnegative/parquet_char.q.out PRE-CREATION ql/src/test/results/clientnegative/parquet_date.q.out PRE-CREATION ql/src/test/results/clientnegative/parquet_decimal.q.out PRE-CREATION ql/src/test/results/clientnegative/parquet_timestamp.q.out PRE-CREATION ql/src/test/results/clientnegative/parquet_varchar.q.out PRE-CREATION Diff: https://reviews.apache.org/r/18914/diff/ Testing --- New tests pass. Thanks, Brock Noland
[jira] [Commented] (HIVE-6430) MapJoin hash table has large memory overhead
[
https://issues.apache.org/jira/browse/HIVE-6430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924233#comment-13924233
]
Hive QA commented on HIVE-6430:
---
{color:red}Overall{color}: -1 no tests executed
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12633240/HIVE-6430.patch
Test results:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1649/testReport
Console output:
http://bigtop01.cloudera.org:8080/job/PreCommit-HIVE-Build/1649/console
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ [[ -n '' ]]
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-Build-1649/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 'metastore/scripts/upgrade/derby/upgrade.order.derby'
Reverted 'metastore/scripts/upgrade/mysql/upgrade.order.mysql'
Reverted 'metastore/scripts/upgrade/mysql/hive-schema-0.13.0.mysql.sql'
Reverted 'metastore/scripts/upgrade/oracle/upgrade.order.oracle'
Reverted 'metastore/scripts/upgrade/postgres/upgrade.order.postgres'
++ awk '{print $2}'
++ egrep -v '^X|^Performing status on external'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target
shims/0.20S/target shims/0.23/target shims/aggregator/target
shims/common/target shims/common-secure/target packaging/target
hbase-handler/target testutils/target jdbc/target metastore/target
metastore/scripts/upgrade/derby/upgrade-0.13.0-to-0.14.0.derby.sql
metastore/scripts/upgrade/derby/hive-schema-0.14.0.derby.sql
metastore/scripts/upgrade/mysql/upgrade-0.13.0-to-0.14.0.mysql.sql
metastore/scripts/upgrade/mysql/hive-schema-0.14.0.mysql.sql
metastore/scripts/upgrade/oracle/upgrade-0.13.0-to-0.14.0.oracle.sql
metastore/scripts/upgrade/oracle/hive-schema-0.14.0.oracle.sql
metastore/scripts/upgrade/postgres/upgrade-0.13.0-to-0.14.0.postgres.sql
metastore/scripts/upgrade/postgres/hive-schema-0.14.0.postgres.sql
itests/target itests/hcatalog-unit/target itests/test-serde/target
itests/qtest/target itests/hive-unit/target itests/custom-serde/target
itests/util/target hcatalog/target hcatalog/storage-handlers/hbase/target
hcatalog/server-extensions/target hcatalog/core/target
hcatalog/webhcat/svr/target hcatalog/webhcat/java-client/target
hcatalog/hcatalog-pig-adapter/target hwi/target common/target common/src/gen
service/target contrib/target serde/target beeline/target odbc/target
cli/target ql/dependency-reduced-pom.xml ql/target
+ svn update
Uql/src/test/queries/clientpositive/mapjoin_mapjoin.q
Uql/src/test/results/clientpositive/mapjoin_mapjoin.q.out
U
ql/src/java/org/apache/hadoop/hive/ql/optimizer/physical/LocalMapJoinProcFactory.java
Uql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java
Uql/src/java/org/apache/hadoop/hive/ql/exec/mr/HashTableLoader.java
Uql/src/java/org/apache/hadoop/hive/ql/exec/HashTableSinkOperator.java
Fetching external item into 'hcatalog/src/test/e2e/harness'
Updated external to revision 1575376.
Updated to revision 1575376.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12633240
MapJoin hash table has large memory overhead
Key: HIVE-6430
URL: https://issues.apache.org/jira/browse/HIVE-6430
Project: Hive
Issue Type: Improvement
Reporter: Sergey Shelukhin
Assignee: Sergey Shelukhin
Attachments: HIVE-6430.patch, HIVE-6430.patch
Right now, in some queries, I see that storing e.g. 4 ints (2 for key and 2
for row) can take several hundred bytes, which is ridiculous. I am reducing
the size of MJKey and MJRowContainer in other jiras, but in general we don't
need to have java hash table there. We can either use primitive-friendly
[jira] [Updated] (HIVE-6511) casting from decimal to tinyint,smallint, int and bigint generates different result when vectorization is on
[
https://issues.apache.org/jira/browse/HIVE-6511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jitendra Nath Pandey updated HIVE-6511:
---
Status: Patch Available (was: Open)
casting from decimal to tinyint,smallint, int and bigint generates different
result when vectorization is on
Key: HIVE-6511
URL: https://issues.apache.org/jira/browse/HIVE-6511
Project: Hive
Issue Type: Bug
Reporter: Jitendra Nath Pandey
Assignee: Jitendra Nath Pandey
Attachments: HIVE-6511.1.patch, HIVE-6511.2.patch, HIVE-6511.3.patch,
HIVE-6511.4.patch
select dc,cast(dc as int), cast(dc as smallint),cast(dc as tinyint) from
vectortab10korc limit 20 generates following result when vectorization is
enabled:
{code}
4619756289662.078125 -1628520834 -16770 126
1553532646710.316406 -1245514442 -2762 54
3367942487288.360352 688127224 -776-8
4386447830839.337891 1286221623 12087 55
-3234165331139.458008 -54957251 27453 61
-488378613475.326172 1247658269 -16099 29
-493942492598.691406 -21253559 -19895 73
3101852523586.039062 886135874 23618 66
2544105595941.381836 1484956709 -23515 37
-3997512403067.0625 1102149509 30597 -123
-1183754978977.589355 1655994718 31070 94
1408783849655.676758 34576568-26440 -72
-2993175106993.426758 417098319 27215 79
3004723551798.100586 -1753555402 -8650 54
1103792083527.786133 -14511544 -28088 72
469767055288.485352 1615620024 26552 -72
-1263700791098.294434 -980406074 12486 -58
-4244889766496.484375 -1462078048 30112 -96
-3962729491139.782715 1525323068 -27332 60
NULL NULLNULLNULL
{code}
When vectorization is disabled, result looks like this:
{code}
4619756289662.078125 -1628520834 -16770 126
1553532646710.316406 -1245514442 -2762 54
3367942487288.360352 688127224 -776-8
4386447830839.337891 1286221623 12087 55
-3234165331139.458008 -54957251 27453 61
-488378613475.326172 1247658269 -16099 29
-493942492598.691406 -21253558 -19894 74
3101852523586.039062 886135874 23618 66
2544105595941.381836 1484956709 -23515 37
-3997512403067.0625 1102149509 30597 -123
-1183754978977.589355 1655994719 31071 95
1408783849655.676758 34576567-26441 -73
-2993175106993.426758 417098319 27215 79
3004723551798.100586 -1753555402 -8650 54
1103792083527.786133 -14511545 -28089 71
469767055288.485352 1615620024 26552 -72
-1263700791098.294434 -980406074 12486 -58
-4244889766496.484375 -1462078048 30112 -96
-3962729491139.782715 1525323069 -27331 61
NULL NULLNULLNULL
{code}
This issue is visible only for certain decimal values. In above example, row
7,11,12, and 15 generates different results.
vectortab10korc table schema:
{code}
t tinyint from deserializer
sismallintfrom deserializer
i int from deserializer
b bigint from deserializer
f float from deserializer
d double from deserializer
dcdecimal(38,18) from deserializer
boboolean from deserializer
s string from deserializer
s2string from deserializer
tstimestamp from deserializer
# Detailed Table Information
Database: default
Owner:xyz
CreateTime: Tue Feb 25 21:54:28 UTC 2014
LastAccessTime: UNKNOWN
Protect Mode: None
Retention:0
Location:
hdfs://host1.domain.com:8020/apps/hive/warehouse/vectortab10korc
Table Type: MANAGED_TABLE
Table Parameters:
COLUMN_STATS_ACCURATE true
numFiles1
numRows 1
rawDataSize 0
totalSize 344748
transient_lastDdlTime 1393365281
# Storage Information
SerDe Library:org.apache.hadoop.hive.ql.io.orc.OrcSerde
InputFormat: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
[jira] [Commented] (HIVE-6511) casting from decimal to tinyint,smallint, int and bigint generates different result when vectorization is on
[
https://issues.apache.org/jira/browse/HIVE-6511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924234#comment-13924234
]
Eric Hanson commented on HIVE-6511:
---
+1
casting from decimal to tinyint,smallint, int and bigint generates different
result when vectorization is on
Key: HIVE-6511
URL: https://issues.apache.org/jira/browse/HIVE-6511
Project: Hive
Issue Type: Bug
Reporter: Jitendra Nath Pandey
Assignee: Jitendra Nath Pandey
Attachments: HIVE-6511.1.patch, HIVE-6511.2.patch, HIVE-6511.3.patch,
HIVE-6511.4.patch
select dc,cast(dc as int), cast(dc as smallint),cast(dc as tinyint) from
vectortab10korc limit 20 generates following result when vectorization is
enabled:
{code}
4619756289662.078125 -1628520834 -16770 126
1553532646710.316406 -1245514442 -2762 54
3367942487288.360352 688127224 -776-8
4386447830839.337891 1286221623 12087 55
-3234165331139.458008 -54957251 27453 61
-488378613475.326172 1247658269 -16099 29
-493942492598.691406 -21253559 -19895 73
3101852523586.039062 886135874 23618 66
2544105595941.381836 1484956709 -23515 37
-3997512403067.0625 1102149509 30597 -123
-1183754978977.589355 1655994718 31070 94
1408783849655.676758 34576568-26440 -72
-2993175106993.426758 417098319 27215 79
3004723551798.100586 -1753555402 -8650 54
1103792083527.786133 -14511544 -28088 72
469767055288.485352 1615620024 26552 -72
-1263700791098.294434 -980406074 12486 -58
-4244889766496.484375 -1462078048 30112 -96
-3962729491139.782715 1525323068 -27332 60
NULL NULLNULLNULL
{code}
When vectorization is disabled, result looks like this:
{code}
4619756289662.078125 -1628520834 -16770 126
1553532646710.316406 -1245514442 -2762 54
3367942487288.360352 688127224 -776-8
4386447830839.337891 1286221623 12087 55
-3234165331139.458008 -54957251 27453 61
-488378613475.326172 1247658269 -16099 29
-493942492598.691406 -21253558 -19894 74
3101852523586.039062 886135874 23618 66
2544105595941.381836 1484956709 -23515 37
-3997512403067.0625 1102149509 30597 -123
-1183754978977.589355 1655994719 31071 95
1408783849655.676758 34576567-26441 -73
-2993175106993.426758 417098319 27215 79
3004723551798.100586 -1753555402 -8650 54
1103792083527.786133 -14511545 -28089 71
469767055288.485352 1615620024 26552 -72
-1263700791098.294434 -980406074 12486 -58
-4244889766496.484375 -1462078048 30112 -96
-3962729491139.782715 1525323069 -27331 61
NULL NULLNULLNULL
{code}
This issue is visible only for certain decimal values. In above example, row
7,11,12, and 15 generates different results.
vectortab10korc table schema:
{code}
t tinyint from deserializer
sismallintfrom deserializer
i int from deserializer
b bigint from deserializer
f float from deserializer
d double from deserializer
dcdecimal(38,18) from deserializer
boboolean from deserializer
s string from deserializer
s2string from deserializer
tstimestamp from deserializer
# Detailed Table Information
Database: default
Owner:xyz
CreateTime: Tue Feb 25 21:54:28 UTC 2014
LastAccessTime: UNKNOWN
Protect Mode: None
Retention:0
Location:
hdfs://host1.domain.com:8020/apps/hive/warehouse/vectortab10korc
Table Type: MANAGED_TABLE
Table Parameters:
COLUMN_STATS_ACCURATE true
numFiles1
numRows 1
rawDataSize 0
totalSize 344748
transient_lastDdlTime 1393365281
# Storage Information
SerDe Library:org.apache.hadoop.hive.ql.io.orc.OrcSerde
InputFormat: org.apache.hadoop.hive.ql.io.orc.OrcInputFormat
[jira] [Commented] (HIVE-6457) Ensure Parquet integration has good error messages for data types not supported
[ https://issues.apache.org/jira/browse/HIVE-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924235#comment-13924235 ] Szehon Ho commented on HIVE-6457: - Thanks Brock for improving the message! I left a comment on the rb. Ensure Parquet integration has good error messages for data types not supported --- Key: HIVE-6457 URL: https://issues.apache.org/jira/browse/HIVE-6457 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-6457.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18914: HIVE-6457 - Ensure Parquet integration has good error messages for data types not supported
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18914/ --- (Updated March 7, 2014, 7:26 p.m.) Review request for hive, Szehon Ho and Xuefu Zhang. Changes --- Thank you for the review! Good call on the constants. I have updated. Bugs: HIVE-6457 https://issues.apache.org/jira/browse/HIVE-6457 Repository: hive-git Description --- Improved error messages for unsupported types. Diffs (updated) - ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ArrayWritableObjectInspector.java a2c7fe0 ql/src/test/queries/clientnegative/parquet_char.q PRE-CREATION ql/src/test/queries/clientnegative/parquet_date.q PRE-CREATION ql/src/test/queries/clientnegative/parquet_decimal.q PRE-CREATION ql/src/test/queries/clientnegative/parquet_timestamp.q PRE-CREATION ql/src/test/queries/clientnegative/parquet_varchar.q PRE-CREATION ql/src/test/results/clientnegative/parquet_char.q.out PRE-CREATION ql/src/test/results/clientnegative/parquet_date.q.out PRE-CREATION ql/src/test/results/clientnegative/parquet_decimal.q.out PRE-CREATION ql/src/test/results/clientnegative/parquet_timestamp.q.out PRE-CREATION ql/src/test/results/clientnegative/parquet_varchar.q.out PRE-CREATION Diff: https://reviews.apache.org/r/18914/diff/ Testing --- New tests pass. Thanks, Brock Noland
[jira] [Updated] (HIVE-6457) Ensure Parquet integration has good error messages for data types not supported
[ https://issues.apache.org/jira/browse/HIVE-6457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brock Noland updated HIVE-6457: --- Attachment: HIVE-6457.patch Ensure Parquet integration has good error messages for data types not supported --- Key: HIVE-6457 URL: https://issues.apache.org/jira/browse/HIVE-6457 Project: Hive Issue Type: Improvement Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-6457.patch, HIVE-6457.patch -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Review Request 18914: HIVE-6457 - Ensure Parquet integration has good error messages for data types not supported
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18914/#review36561 --- Ship it! lgtm, +1 (non-binding) - Szehon Ho On March 7, 2014, 7:26 p.m., Brock Noland wrote: --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/18914/ --- (Updated March 7, 2014, 7:26 p.m.) Review request for hive, Szehon Ho and Xuefu Zhang. Bugs: HIVE-6457 https://issues.apache.org/jira/browse/HIVE-6457 Repository: hive-git Description --- Improved error messages for unsupported types. Diffs - ql/src/java/org/apache/hadoop/hive/ql/io/parquet/serde/ArrayWritableObjectInspector.java a2c7fe0 ql/src/test/queries/clientnegative/parquet_char.q PRE-CREATION ql/src/test/queries/clientnegative/parquet_date.q PRE-CREATION ql/src/test/queries/clientnegative/parquet_decimal.q PRE-CREATION ql/src/test/queries/clientnegative/parquet_timestamp.q PRE-CREATION ql/src/test/queries/clientnegative/parquet_varchar.q PRE-CREATION ql/src/test/results/clientnegative/parquet_char.q.out PRE-CREATION ql/src/test/results/clientnegative/parquet_date.q.out PRE-CREATION ql/src/test/results/clientnegative/parquet_decimal.q.out PRE-CREATION ql/src/test/results/clientnegative/parquet_timestamp.q.out PRE-CREATION ql/src/test/results/clientnegative/parquet_varchar.q.out PRE-CREATION Diff: https://reviews.apache.org/r/18914/diff/ Testing --- New tests pass. Thanks, Brock Noland
Re: Timeline for the Hive 0.13 release?
HIVE-6037 isn't on the list yet (synchronize HiveConf with hive-default.xml.template). What's the correct procedure for updating the wikihttps://cwiki.apache.org/confluence/display/Hive/Hive+0.13+release+status, now that there's a JQL list as well as a table? Just add a row to the table and append the jira in the JQL list? -- Lefty On Fri, Mar 7, 2014 at 2:06 PM, Eugene Koifman ekoif...@hortonworks.comwrote: HIVE-6576 please. w/o there is a backwards incompatible change in webhcat REST api On Fri, Mar 7, 2014 at 10:45 AM, Owen O'Malley omal...@apache.org wrote: We should get 5567 and 5568 applied too. They are a bit stale, but I'll refresh them next week. On Fri, Mar 7, 2014 at 10:41 AM, Harish Butani hbut...@hortonworks.com wrote: ok adding 6414, 6572, 6574 to the list On Mar 7, 2014, at 10:27 AM, Xuefu Zhang xzh...@cloudera.com wrote: HIVE-6414 seems bad enough to included. On Thu, Mar 6, 2014 at 3:28 PM, Sushanth Sowmyan khorg...@gmail.com wrote: One more, I'm afraid. I'll add it to the wiki. I just uploaded a patch for https://issues.apache.org/jira/browse/HIVE-6572 . This fixes one glaring bug on usage of mapred.min.split.size.per.rack and mapred.min.split.size.per.node, and makes sure some of the other conf parameters we use are correctly through shims. I want this to go into 0.13 because I don't want a release of ours to go out with the dual/old configs (and erroneous config in the per.rack/node case) and have that be expected behaviour to support in the future. On Thu, Mar 6, 2014 at 11:04 AM, Harish Butani hbut...@hortonworks.com wrote: ok sure. Tracking these with the JQL below. I don’t have permission to setup a Shared Filter; can someone help with this. Of the 35 issues: 11 are still open, 22 are patch available, 2 are resolved. regards, Harish. JQL: id in (HIVE-5317, HIVE-5843, HIVE-6060, HIVE-6319, HIVE-6460, HIVE-5687, HIVE-5943, HIVE-5942, HIVE-6547, HIVE-5155, HIVE-6486, HIVE-6455, HIVE-4177, HIVE-4764, HIVE-6306, HIVE-6350, HIVE-6485, HIVE-6507, HIVE-6499, HIVE-6325, HIVE-6558, HIVE-6403, HIVE-4790, HIVE-4293, HIVE-6551, HIVE-6359, HIVE-6314, HIVE-6241, HIVE-5768, HIVE-2752, HIVE-6312, HIVE-6129, HIVE-6012, HIVE-6434, HIVE-6562) ORDER BY status ASC, assignee On Mar 5, 2014, at 6:50 PM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Can you consider HIVE-6562 as well? HIVE-6562 - Protection from exceptions in ORC predicate evaluation Thanks Prasanth Jayachandran On Mar 5, 2014, at 5:56 PM, Jason Dere jd...@hortonworks.com wrote: Would like to get these in, if possible: HIVE-6012 restore backward compatibility of arithmetic operations HIVE-6434 Restrict function create/drop to admin roles On Mar 5, 2014, at 5:41 PM, Navis류승우 navis@nexr.com wrote: I have really big wish list(65 pending) but it would be time to focus on finalization. - Small bugs HIVE-6403 uncorrelated subquery is failing with auto.convert.join=true HIVE-4790 MapredLocalTask task does not make virtual columns HIVE-4293 Predicates following UDTF operator are removed by PPD - Trivials HIVE-6551 group by after join with skew join optimization references invalid task sometimes HIVE-6359 beeline -f fails on scripts with tabs in them. HIVE-6314 The logging (progress reporting) is too verbose HIVE-6241 Remove direct reference of Hadoop23Shims inQTestUtil HIVE-5768 Beeline connection cannot be closed with !close command HIVE-2752 Index names are case sensitive - Memory leakage HIVE-6312 doAs with plain sasl auth should be session aware - Implementation is not accord with document HIVE-6129 alter exchange is implemented in inverted manner I'll update the wiki, too. 2014-03-05 12:18 GMT+09:00 Harish Butani hbut...@hortonworks.com : Tracking jiras to be applied to branch 0.13 here: https://cwiki.apache.org/confluence/display/Hive/Hive+0.13+release+status On Mar 4, 2014, at 5:45 PM, Harish Butani hbut...@hortonworks.com wrote: the branch is created. have changed the poms in both branches. Planning to setup a wikipage to track jiras that will get ported to 0.13 regards, Harish. On Mar 4, 2014, at 5:05 PM, Harish Butani hbut...@hortonworks.com wrote: branching now. Will be changing the pom files on trunk. Will send another email when the branch and trunk changes are in. On Mar 4, 2014, at 4:03 PM, Sushanth Sowmyan khorg...@gmail.com wrote: I have two patches still as patch-available, that have had +1s as well, but are waiting on pre-commit tests picking them up go
[jira] [Updated] (HIVE-6555) TestSchemaTool is failing on trunk after branching
[ https://issues.apache.org/jira/browse/HIVE-6555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6555: --- Resolution: Fixed Fix Version/s: 0.14.0 Status: Resolved (was: Patch Available) Committed to trunk. TestSchemaTool is failing on trunk after branching -- Key: HIVE-6555 URL: https://issues.apache.org/jira/browse/HIVE-6555 Project: Hive Issue Type: Bug Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.14.0 Attachments: HIVE-6555-branch13.patch, HIVE-6555.1.patch, HIVE-6555.patch This is because version was bumped to 0.14 in pom file and there are no metastore scripts for 0.14 yet. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6555) TestSchemaTool is failing on trunk after branching
[ https://issues.apache.org/jira/browse/HIVE-6555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6555: --- Fix Version/s: 0.13.0 TestSchemaTool is failing on trunk after branching -- Key: HIVE-6555 URL: https://issues.apache.org/jira/browse/HIVE-6555 Project: Hive Issue Type: Bug Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.13.0, 0.14.0 Attachments: HIVE-6555-branch13.patch, HIVE-6555.1.patch, HIVE-6555.patch This is because version was bumped to 0.14 in pom file and there are no metastore scripts for 0.14 yet. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6555) TestSchemaTool is failing on trunk after branching
[ https://issues.apache.org/jira/browse/HIVE-6555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924253#comment-13924253 ] Ashutosh Chauhan commented on HIVE-6555: Committed 0.13 version on 0.13 branch as well. TestSchemaTool is failing on trunk after branching -- Key: HIVE-6555 URL: https://issues.apache.org/jira/browse/HIVE-6555 Project: Hive Issue Type: Bug Reporter: Ashutosh Chauhan Assignee: Ashutosh Chauhan Fix For: 0.13.0, 0.14.0 Attachments: HIVE-6555-branch13.patch, HIVE-6555.1.patch, HIVE-6555.patch This is because version was bumped to 0.14 in pom file and there are no metastore scripts for 0.14 yet. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6568) Vectorized cast of decimal to string produces strings with trailing zeros.
[ https://issues.apache.org/jira/browse/HIVE-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HIVE-6568: --- Status: Patch Available (was: Open) Vectorized cast of decimal to string produces strings with trailing zeros. -- Key: HIVE-6568 URL: https://issues.apache.org/jira/browse/HIVE-6568 Project: Hive Issue Type: Bug Components: Vectorization Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Attachments: HIVE-6568.1.patch A decimal value 1.23 with scale 5 is represented in string as 1.23000. This behavior is different from HiveDecimal behavior. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6568) Vectorized cast of decimal to string produces strings with trailing zeros.
[ https://issues.apache.org/jira/browse/HIVE-6568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jitendra Nath Pandey updated HIVE-6568: --- Attachment: HIVE-6568.1.patch Vectorized cast of decimal to string produces strings with trailing zeros. -- Key: HIVE-6568 URL: https://issues.apache.org/jira/browse/HIVE-6568 Project: Hive Issue Type: Bug Components: Vectorization Reporter: Jitendra Nath Pandey Assignee: Jitendra Nath Pandey Attachments: HIVE-6568.1.patch A decimal value 1.23 with scale 5 is represented in string as 1.23000. This behavior is different from HiveDecimal behavior. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6417) sql std auth - new users in admin role config should get added
[ https://issues.apache.org/jira/browse/HIVE-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6417: --- Fix Version/s: (was: 0.14.0) sql std auth - new users in admin role config should get added -- Key: HIVE-6417 URL: https://issues.apache.org/jira/browse/HIVE-6417 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Ashutosh Chauhan Fix For: 0.13.0 Attachments: HIVE-6417.1.patch, HIVE-6417.patch if metastore is started with hive.users.in.admin.role=user1, then user1 is added admin role to metastore. If the value is changed to hive.users.in.admin.role=user2, then user2 should get added to the role in metastore. Right now, if the admin role exists, new users don't get added. A work-around is - user1 adding user2 to the admin role using grant role statement. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6417) sql std auth - new users in admin role config should get added
[ https://issues.apache.org/jira/browse/HIVE-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-6417: --- Fix Version/s: 0.13.0 sql std auth - new users in admin role config should get added -- Key: HIVE-6417 URL: https://issues.apache.org/jira/browse/HIVE-6417 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Ashutosh Chauhan Fix For: 0.13.0 Attachments: HIVE-6417.1.patch, HIVE-6417.patch if metastore is started with hive.users.in.admin.role=user1, then user1 is added admin role to metastore. If the value is changed to hive.users.in.admin.role=user2, then user2 should get added to the role in metastore. Right now, if the admin role exists, new users don't get added. A work-around is - user1 adding user2 to the admin role using grant role statement. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-6417) sql std auth - new users in admin role config should get added
[ https://issues.apache.org/jira/browse/HIVE-6417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924255#comment-13924255 ] Ashutosh Chauhan commented on HIVE-6417: Committed to 0.13 as well. sql std auth - new users in admin role config should get added -- Key: HIVE-6417 URL: https://issues.apache.org/jira/browse/HIVE-6417 Project: Hive Issue Type: Sub-task Components: Authorization Reporter: Thejas M Nair Assignee: Ashutosh Chauhan Fix For: 0.13.0 Attachments: HIVE-6417.1.patch, HIVE-6417.patch if metastore is started with hive.users.in.admin.role=user1, then user1 is added admin role to metastore. If the value is changed to hive.users.in.admin.role=user2, then user2 should get added to the role in metastore. Right now, if the admin role exists, new users don't get added. A work-around is - user1 adding user2 to the admin role using grant role statement. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HIVE-6477) Aggregation functions for tiny/smallint broken with parquet
[
https://issues.apache.org/jira/browse/HIVE-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Szehon Ho updated HIVE-6477:
Resolution: Duplicate
Status: Resolved (was: Patch Available)
Aggregation functions for tiny/smallint broken with parquet
---
Key: HIVE-6477
URL: https://issues.apache.org/jira/browse/HIVE-6477
Project: Hive
Issue Type: Bug
Components: Serializers/Deserializers
Affects Versions: 0.13.0
Reporter: Szehon Ho
Assignee: Szehon Ho
Labels: parquet
Attachments: HIVE-6477.2.patch, HIVE-6477.patch
Given the following table:
{noformat}
CREATE TABLE IF NOT EXISTS commontypesagg (
id int,
bool_col boolean,
tinyint_col tinyint,
smallint_col smallint,
int_col int,
bigint_col bigint,
float_col float,
double_col double,
date_string_col string,
string_col string)
PARTITIONED BY (year int, month int, day int)
STORED AS PARQUET;
{noformat}
The following queries throws ClassCastException:
{noformat}
select count(tinyint_col), min(tinyint_col), max(tinyint_col),
sum(tinyint_col) from commontypesagg;
select count(smallint_col), min(smallint_col), max(smallint_col),
sum(smallint_col) from commontypesagg;
{noformat}
Exception is the following:
{noformat}
2014-01-29 14:02:11,381 INFO org.apache.hadoop.mapred.TaskStatus:
task-diagnostic-info for task attempt_201401290934_0006_m_01_1 :
java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:
Hive Runtime Error while processing row
{id:null,bool_col:null,tinyint_col:1,smallint_col:null,int_col:null,bigint_col:null,float_col:null,double_col:null,date_string_col:null,string_col:null,year:2009,month:1}
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:175)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:417)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
Error while processing row
{id:null,bool_col:null,tinyint_col:1,smallint_col:null,int_col:null,bigint_col:null,float_col:null,double_col:null,date_string_col:null,string_col:null,year:2009,month:1}
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.map(ExecMapper.java:157)
... 8 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
java.lang.ClassCastException: org.apache.hadoop.io.IntWritable cannot be cast
to java.lang.Byte
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processOp(GroupByOperator.java:796)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:844)
at
org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:87)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:844)
at
org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:91)
at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:504)
at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:844)
at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:519)
... 9 more
Caused by: java.lang.ClassCastException: org.apache.hadoop.io.IntWritable
cannot be cast to java.lang.Byte
at
org.apache.hadoop.hive.serde2.objectinspector.primitive.JavaByteObjectInspector.get(JavaByteObjectInspector.java:40)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(ObjectInspectorUtils.java:666)
at
org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.compare(ObjectInspectorUtils.java:631)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMin$GenericUDAFMinEvaluator.merge(GenericUDAFMin.java:109)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFMin$GenericUDAFMinEvaluator.iterate(GenericUDAFMin.java:96)
at
org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator.aggregate(GenericUDAFEvaluator.java:183)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.updateAggregations(GroupByOperator.java:629)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processHashAggr(GroupByOperator.java:826)
at
org.apache.hadoop.hive.ql.exec.GroupByOperator.processKey(GroupByOperator.java:723)
at
Re: Proposal to switch to pull requests
+1 for moving to git!
Thanks,
--Vaibhav
On Fri, Mar 7, 2014 at 9:46 AM, Prasad Mujumdar pras...@cloudera.comwrote:
while (true) {
+1
}
+1 // another, just in case ;)
thanks
Prasad
On Fri, Mar 7, 2014 at 6:47 AM, kulkarni.swar...@gmail.com
kulkarni.swar...@gmail.com wrote:
+1
On Fri, Mar 7, 2014 at 1:05 AM, Thejas Nair the...@hortonworks.com
wrote:
Should we start with moving our primary source code repository from
svn to git ? I feel git is more powerful and easy to use (once you go
past the learning curve!).
On Wed, Mar 5, 2014 at 7:39 AM, Brock Noland br...@cloudera.com
wrote:
Personally I prefer the Github workflow, but I believe there have
been
some challenges with that since the source for apache projects must
be
stored in apache source control (git or svn).
Relevent:
https://blogs.apache.org/infra/entry/improved_integration_between_apache_and
On Wed, Mar 5, 2014 at 9:19 AM, kulkarni.swar...@gmail.com
kulkarni.swar...@gmail.com wrote:
Hello,
Since we have a nice mirrored git repository for hive[1], any
specific
reason why we can't switch to doing pull requests instead of
patches?
IMHO
pull requests are awesome for peer review plus it is also very easy
to
keep
track of JIRAs with open pull requests instead of looking for JIRAs
in a
Patch Available state. Also since they get updated automatically,
it
is
also very easy to see if a review comment made by a reviewer was
addressed
properly or not.
Thoughts?
Thanks,
[1] https://github.com/apache/hive
--
Swarnim
--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or
entity
to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the
reader
of this message is not the intended recipient, you are hereby notified
that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender
immediately
and delete it from your system. Thank You.
--
Swarnim
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Timeline for the Hive 0.13 release?
Lefty, Do we need to include HIVE-6037 in the release? I.e.: Is the current hive-default.xml.template very out of sync without it? It's a pretty large patch mostly aimed at making our lives easier, but doesn't impact the end user directly. Can we do that one trunk only? Thanks, Gunther. On Fri, Mar 7, 2014 at 11:30 AM, Lefty Leverenz leftylever...@gmail.comwrote: HIVE-6037 isn't on the list yet (synchronize HiveConf with hive-default.xml.template). What's the correct procedure for updating the wiki https://cwiki.apache.org/confluence/display/Hive/Hive+0.13+release+status , now that there's a JQL list as well as a table? Just add a row to the table and append the jira in the JQL list? -- Lefty On Fri, Mar 7, 2014 at 2:06 PM, Eugene Koifman ekoif...@hortonworks.com wrote: HIVE-6576 please. w/o there is a backwards incompatible change in webhcat REST api On Fri, Mar 7, 2014 at 10:45 AM, Owen O'Malley omal...@apache.org wrote: We should get 5567 and 5568 applied too. They are a bit stale, but I'll refresh them next week. On Fri, Mar 7, 2014 at 10:41 AM, Harish Butani hbut...@hortonworks.com wrote: ok adding 6414, 6572, 6574 to the list On Mar 7, 2014, at 10:27 AM, Xuefu Zhang xzh...@cloudera.com wrote: HIVE-6414 seems bad enough to included. On Thu, Mar 6, 2014 at 3:28 PM, Sushanth Sowmyan khorg...@gmail.com wrote: One more, I'm afraid. I'll add it to the wiki. I just uploaded a patch for https://issues.apache.org/jira/browse/HIVE-6572 . This fixes one glaring bug on usage of mapred.min.split.size.per.rack and mapred.min.split.size.per.node, and makes sure some of the other conf parameters we use are correctly through shims. I want this to go into 0.13 because I don't want a release of ours to go out with the dual/old configs (and erroneous config in the per.rack/node case) and have that be expected behaviour to support in the future. On Thu, Mar 6, 2014 at 11:04 AM, Harish Butani hbut...@hortonworks.com wrote: ok sure. Tracking these with the JQL below. I don’t have permission to setup a Shared Filter; can someone help with this. Of the 35 issues: 11 are still open, 22 are patch available, 2 are resolved. regards, Harish. JQL: id in (HIVE-5317, HIVE-5843, HIVE-6060, HIVE-6319, HIVE-6460, HIVE-5687, HIVE-5943, HIVE-5942, HIVE-6547, HIVE-5155, HIVE-6486, HIVE-6455, HIVE-4177, HIVE-4764, HIVE-6306, HIVE-6350, HIVE-6485, HIVE-6507, HIVE-6499, HIVE-6325, HIVE-6558, HIVE-6403, HIVE-4790, HIVE-4293, HIVE-6551, HIVE-6359, HIVE-6314, HIVE-6241, HIVE-5768, HIVE-2752, HIVE-6312, HIVE-6129, HIVE-6012, HIVE-6434, HIVE-6562) ORDER BY status ASC, assignee On Mar 5, 2014, at 6:50 PM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Can you consider HIVE-6562 as well? HIVE-6562 - Protection from exceptions in ORC predicate evaluation Thanks Prasanth Jayachandran On Mar 5, 2014, at 5:56 PM, Jason Dere jd...@hortonworks.com wrote: Would like to get these in, if possible: HIVE-6012 restore backward compatibility of arithmetic operations HIVE-6434 Restrict function create/drop to admin roles On Mar 5, 2014, at 5:41 PM, Navis류승우 navis@nexr.com wrote: I have really big wish list(65 pending) but it would be time to focus on finalization. - Small bugs HIVE-6403 uncorrelated subquery is failing with auto.convert.join=true HIVE-4790 MapredLocalTask task does not make virtual columns HIVE-4293 Predicates following UDTF operator are removed by PPD - Trivials HIVE-6551 group by after join with skew join optimization references invalid task sometimes HIVE-6359 beeline -f fails on scripts with tabs in them. HIVE-6314 The logging (progress reporting) is too verbose HIVE-6241 Remove direct reference of Hadoop23Shims inQTestUtil HIVE-5768 Beeline connection cannot be closed with !close command HIVE-2752 Index names are case sensitive - Memory leakage HIVE-6312 doAs with plain sasl auth should be session aware - Implementation is not accord with document HIVE-6129 alter exchange is implemented in inverted manner I'll update the wiki, too. 2014-03-05 12:18 GMT+09:00 Harish Butani hbut...@hortonworks.com : Tracking jiras to be applied to branch 0.13 here: https://cwiki.apache.org/confluence/display/Hive/Hive+0.13+release+status On Mar 4, 2014, at 5:45 PM, Harish Butani hbut...@hortonworks.com wrote: the branch is created. have changed the poms in both branches. Planning to setup a wikipage to track
[jira] [Updated] (HIVE-6566) Incorrect union-all plan with map-joins on Tez
[ https://issues.apache.org/jira/browse/HIVE-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gunther Hagleitner updated HIVE-6566: - Description: The tez dag is hooked up incorrectly for some union all queries involving map joins. That's quite common and results in either NPE or invalid results. (was: The tez dag is hooked up incorrectly for some union all queries involving map joins.) Incorrect union-all plan with map-joins on Tez -- Key: HIVE-6566 URL: https://issues.apache.org/jira/browse/HIVE-6566 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-6566.1.patch, HIVE-6566.2.patch The tez dag is hooked up incorrectly for some union all queries involving map joins. That's quite common and results in either NPE or invalid results. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5607) Hive fails to parse the % (mod) sign after brackets.
[
https://issues.apache.org/jira/browse/HIVE-5607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924274#comment-13924274
]
Prasad Mujumdar commented on HIVE-5607:
---
[~xuefuz] Thanks for picking it up.
The patch itself look fine, though I am wondering if this will break the cases
like the one you changed in the updated test.
{code}
ORDER BY (key ASC, value ASC)
{code}
Given that this patch is in Hive since 0.10, do we need to worry about
maintaining compatibility ?
Hive fails to parse the % (mod) sign after brackets.
--
Key: HIVE-5607
URL: https://issues.apache.org/jira/browse/HIVE-5607
Project: Hive
Issue Type: Bug
Components: Query Processor
Affects Versions: 0.10.0
Reporter: dima machlin
Assignee: Xuefu Zhang
Priority: Minor
Attachments: HIVE-5607.1.patch, HIVE-5607.patch
the scenario :
create table t(a int);
select * from t order by (a)%7;
will fail with the following exception :
FAILED: ParseException line 1:28 mismatched input '%' expecting EOF near ')'
I must mention that this *does* work in 0.7.1 and doesn't work in 0.10
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-6566) Incorrect union-all plan with map-joins on Tez
[ https://issues.apache.org/jira/browse/HIVE-6566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924276#comment-13924276 ] Gunther Hagleitner commented on HIVE-6566: -- Failures are unrelated. Incorrect union-all plan with map-joins on Tez -- Key: HIVE-6566 URL: https://issues.apache.org/jira/browse/HIVE-6566 Project: Hive Issue Type: Bug Affects Versions: 0.13.0 Reporter: Gunther Hagleitner Assignee: Gunther Hagleitner Attachments: HIVE-6566.1.patch, HIVE-6566.2.patch The tez dag is hooked up incorrectly for some union all queries involving map joins. -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Timeline for the Hive 0.13 release?
just update the JQL. On Mar 7, 2014, at 11:30 AM, Lefty Leverenz leftylever...@gmail.com wrote: HIVE-6037 isn't on the list yet (synchronize HiveConf with hive-default.xml.template). What's the correct procedure for updating the wikihttps://cwiki.apache.org/confluence/display/Hive/Hive+0.13+release+status, now that there's a JQL list as well as a table? Just add a row to the table and append the jira in the JQL list? -- Lefty On Fri, Mar 7, 2014 at 2:06 PM, Eugene Koifman ekoif...@hortonworks.comwrote: HIVE-6576 please. w/o there is a backwards incompatible change in webhcat REST api On Fri, Mar 7, 2014 at 10:45 AM, Owen O'Malley omal...@apache.org wrote: We should get 5567 and 5568 applied too. They are a bit stale, but I'll refresh them next week. On Fri, Mar 7, 2014 at 10:41 AM, Harish Butani hbut...@hortonworks.com wrote: ok adding 6414, 6572, 6574 to the list On Mar 7, 2014, at 10:27 AM, Xuefu Zhang xzh...@cloudera.com wrote: HIVE-6414 seems bad enough to included. On Thu, Mar 6, 2014 at 3:28 PM, Sushanth Sowmyan khorg...@gmail.com wrote: One more, I'm afraid. I'll add it to the wiki. I just uploaded a patch for https://issues.apache.org/jira/browse/HIVE-6572 . This fixes one glaring bug on usage of mapred.min.split.size.per.rack and mapred.min.split.size.per.node, and makes sure some of the other conf parameters we use are correctly through shims. I want this to go into 0.13 because I don't want a release of ours to go out with the dual/old configs (and erroneous config in the per.rack/node case) and have that be expected behaviour to support in the future. On Thu, Mar 6, 2014 at 11:04 AM, Harish Butani hbut...@hortonworks.com wrote: ok sure. Tracking these with the JQL below. I don’t have permission to setup a Shared Filter; can someone help with this. Of the 35 issues: 11 are still open, 22 are patch available, 2 are resolved. regards, Harish. JQL: id in (HIVE-5317, HIVE-5843, HIVE-6060, HIVE-6319, HIVE-6460, HIVE-5687, HIVE-5943, HIVE-5942, HIVE-6547, HIVE-5155, HIVE-6486, HIVE-6455, HIVE-4177, HIVE-4764, HIVE-6306, HIVE-6350, HIVE-6485, HIVE-6507, HIVE-6499, HIVE-6325, HIVE-6558, HIVE-6403, HIVE-4790, HIVE-4293, HIVE-6551, HIVE-6359, HIVE-6314, HIVE-6241, HIVE-5768, HIVE-2752, HIVE-6312, HIVE-6129, HIVE-6012, HIVE-6434, HIVE-6562) ORDER BY status ASC, assignee On Mar 5, 2014, at 6:50 PM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Can you consider HIVE-6562 as well? HIVE-6562 - Protection from exceptions in ORC predicate evaluation Thanks Prasanth Jayachandran On Mar 5, 2014, at 5:56 PM, Jason Dere jd...@hortonworks.com wrote: Would like to get these in, if possible: HIVE-6012 restore backward compatibility of arithmetic operations HIVE-6434 Restrict function create/drop to admin roles On Mar 5, 2014, at 5:41 PM, Navis류승우 navis@nexr.com wrote: I have really big wish list(65 pending) but it would be time to focus on finalization. - Small bugs HIVE-6403 uncorrelated subquery is failing with auto.convert.join=true HIVE-4790 MapredLocalTask task does not make virtual columns HIVE-4293 Predicates following UDTF operator are removed by PPD - Trivials HIVE-6551 group by after join with skew join optimization references invalid task sometimes HIVE-6359 beeline -f fails on scripts with tabs in them. HIVE-6314 The logging (progress reporting) is too verbose HIVE-6241 Remove direct reference of Hadoop23Shims inQTestUtil HIVE-5768 Beeline connection cannot be closed with !close command HIVE-2752 Index names are case sensitive - Memory leakage HIVE-6312 doAs with plain sasl auth should be session aware - Implementation is not accord with document HIVE-6129 alter exchange is implemented in inverted manner I'll update the wiki, too. 2014-03-05 12:18 GMT+09:00 Harish Butani hbut...@hortonworks.com : Tracking jiras to be applied to branch 0.13 here: https://cwiki.apache.org/confluence/display/Hive/Hive+0.13+release+status On Mar 4, 2014, at 5:45 PM, Harish Butani hbut...@hortonworks.com wrote: the branch is created. have changed the poms in both branches. Planning to setup a wikipage to track jiras that will get ported to 0.13 regards, Harish. On Mar 4, 2014, at 5:05 PM, Harish Butani hbut...@hortonworks.com wrote: branching now. Will be changing the pom files on trunk. Will send another email when the branch and trunk changes are in. On Mar 4, 2014, at 4:03 PM, Sushanth Sowmyan khorg...@gmail.com wrote: I have two patches still as patch-available, that have had +1s as well, but are waiting on pre-commit tests picking them up go in to 0.13: https://issues.apache.org/jira/browse/HIVE-6507 (refactor of table property names from string constants to an enum in OrcFile) https://issues.apache.org/jira/browse/HIVE-6499 (fixes bug where calls like
Re: Timeline for the Hive 0.13 release?
We should run tests on the branch. Given the number of patches going into the branch I propose for the next few days: - we trigger a run once a day. I can do this. In case of failures, we shouldn’t commit any more patches until the errors are resolved. (brock can you help with instructions on how to trigger a run on a branch) regards, Harish. On Mar 7, 2014, at 11:30 AM, Lefty Leverenz leftylever...@gmail.com wrote: HIVE-6037 isn't on the list yet (synchronize HiveConf with hive-default.xml.template). What's the correct procedure for updating the wikihttps://cwiki.apache.org/confluence/display/Hive/Hive+0.13+release+status, now that there's a JQL list as well as a table? Just add a row to the table and append the jira in the JQL list? -- Lefty On Fri, Mar 7, 2014 at 2:06 PM, Eugene Koifman ekoif...@hortonworks.comwrote: HIVE-6576 please. w/o there is a backwards incompatible change in webhcat REST api On Fri, Mar 7, 2014 at 10:45 AM, Owen O'Malley omal...@apache.org wrote: We should get 5567 and 5568 applied too. They are a bit stale, but I'll refresh them next week. On Fri, Mar 7, 2014 at 10:41 AM, Harish Butani hbut...@hortonworks.com wrote: ok adding 6414, 6572, 6574 to the list On Mar 7, 2014, at 10:27 AM, Xuefu Zhang xzh...@cloudera.com wrote: HIVE-6414 seems bad enough to included. On Thu, Mar 6, 2014 at 3:28 PM, Sushanth Sowmyan khorg...@gmail.com wrote: One more, I'm afraid. I'll add it to the wiki. I just uploaded a patch for https://issues.apache.org/jira/browse/HIVE-6572 . This fixes one glaring bug on usage of mapred.min.split.size.per.rack and mapred.min.split.size.per.node, and makes sure some of the other conf parameters we use are correctly through shims. I want this to go into 0.13 because I don't want a release of ours to go out with the dual/old configs (and erroneous config in the per.rack/node case) and have that be expected behaviour to support in the future. On Thu, Mar 6, 2014 at 11:04 AM, Harish Butani hbut...@hortonworks.com wrote: ok sure. Tracking these with the JQL below. I don’t have permission to setup a Shared Filter; can someone help with this. Of the 35 issues: 11 are still open, 22 are patch available, 2 are resolved. regards, Harish. JQL: id in (HIVE-5317, HIVE-5843, HIVE-6060, HIVE-6319, HIVE-6460, HIVE-5687, HIVE-5943, HIVE-5942, HIVE-6547, HIVE-5155, HIVE-6486, HIVE-6455, HIVE-4177, HIVE-4764, HIVE-6306, HIVE-6350, HIVE-6485, HIVE-6507, HIVE-6499, HIVE-6325, HIVE-6558, HIVE-6403, HIVE-4790, HIVE-4293, HIVE-6551, HIVE-6359, HIVE-6314, HIVE-6241, HIVE-5768, HIVE-2752, HIVE-6312, HIVE-6129, HIVE-6012, HIVE-6434, HIVE-6562) ORDER BY status ASC, assignee On Mar 5, 2014, at 6:50 PM, Prasanth Jayachandran pjayachand...@hortonworks.com wrote: Can you consider HIVE-6562 as well? HIVE-6562 - Protection from exceptions in ORC predicate evaluation Thanks Prasanth Jayachandran On Mar 5, 2014, at 5:56 PM, Jason Dere jd...@hortonworks.com wrote: Would like to get these in, if possible: HIVE-6012 restore backward compatibility of arithmetic operations HIVE-6434 Restrict function create/drop to admin roles On Mar 5, 2014, at 5:41 PM, Navis류승우 navis@nexr.com wrote: I have really big wish list(65 pending) but it would be time to focus on finalization. - Small bugs HIVE-6403 uncorrelated subquery is failing with auto.convert.join=true HIVE-4790 MapredLocalTask task does not make virtual columns HIVE-4293 Predicates following UDTF operator are removed by PPD - Trivials HIVE-6551 group by after join with skew join optimization references invalid task sometimes HIVE-6359 beeline -f fails on scripts with tabs in them. HIVE-6314 The logging (progress reporting) is too verbose HIVE-6241 Remove direct reference of Hadoop23Shims inQTestUtil HIVE-5768 Beeline connection cannot be closed with !close command HIVE-2752 Index names are case sensitive - Memory leakage HIVE-6312 doAs with plain sasl auth should be session aware - Implementation is not accord with document HIVE-6129 alter exchange is implemented in inverted manner I'll update the wiki, too. 2014-03-05 12:18 GMT+09:00 Harish Butani hbut...@hortonworks.com : Tracking jiras to be applied to branch 0.13 here: https://cwiki.apache.org/confluence/display/Hive/Hive+0.13+release+status On Mar 4, 2014, at 5:45 PM, Harish Butani hbut...@hortonworks.com wrote: the branch is created. have changed the poms in both branches. Planning to setup a wikipage to track jiras that will get ported to 0.13 regards, Harish. On Mar 4, 2014, at 5:05 PM, Harish Butani hbut...@hortonworks.com wrote: branching now. Will be changing the pom files on trunk. Will send another email when the branch and trunk changes are in. On Mar 4, 2014, at 4:03 PM, Sushanth Sowmyan khorg...@gmail.com wrote: I have two patches still as
[jira] [Commented] (HIVE-4629) HS2 should support an API to retrieve query logs
[ https://issues.apache.org/jira/browse/HIVE-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924290#comment-13924290 ] Prasad Mujumdar commented on HIVE-4629: --- [~shreepadma] I guess the patch is out of sync and won't apply. There are a bunch of changes to HS2 interface, the build framework changed from ant to maven etc. Would you mind rebasing it to the latest and update the review. If you want I can take care of the rebase. HS2 should support an API to retrieve query logs Key: HIVE-4629 URL: https://issues.apache.org/jira/browse/HIVE-4629 Project: Hive Issue Type: Sub-task Components: HiveServer2 Reporter: Shreepadma Venugopalan Assignee: Shreepadma Venugopalan Attachments: HIVE-4629-no_thrift.1.patch, HIVE-4629.1.patch, HIVE-4629.2.patch HiveServer2 should support an API to retrieve query logs. This is particularly relevant because HiveServer2 supports async execution but doesn't provide a way to report progress. Providing an API to retrieve query logs will help report progress to the client. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HIVE-5607) Hive fails to parse the % (mod) sign after brackets.
[ https://issues.apache.org/jira/browse/HIVE-5607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13924294#comment-13924294 ] Xuefu Zhang commented on HIVE-5607: --- Thanks, [~prasadm]. Your concern is valid. However, I think we should be okay because: 1. HIVE-1367 broke the backward compatibility in the first place. The current JIRA is only to revert that regression. 2. Syntactically, the current JIRA restores to align with standard SQL. 3. The regression introduced by HIVE-1367 prevents any expression in the list from having parenthesis. This seems a big limitation. From this, personally I think the restoration should proceed. However, I'm open to further discussions. Hive fails to parse the % (mod) sign after brackets. -- Key: HIVE-5607 URL: https://issues.apache.org/jira/browse/HIVE-5607 Project: Hive Issue Type: Bug Components: Query Processor Affects Versions: 0.10.0 Reporter: dima machlin Assignee: Xuefu Zhang Priority: Minor Attachments: HIVE-5607.1.patch, HIVE-5607.patch the scenario : create table t(a int); select * from t order by (a)%7; will fail with the following exception : FAILED: ParseException line 1:28 mismatched input '%' expecting EOF near ')' I must mention that this *does* work in 0.7.1 and doesn't work in 0.10 -- This message was sent by Atlassian JIRA (v6.2#6252)
Re: Proposal to switch to pull requests
Once Prasad's loop finishes I'd like to add my +1 too.
On Fri, Mar 7, 2014 at 11:44 AM, Vaibhav Gumashta vgumas...@hortonworks.com
wrote:
+1 for moving to git!
Thanks,
--Vaibhav
On Fri, Mar 7, 2014 at 9:46 AM, Prasad Mujumdar pras...@cloudera.com
wrote:
while (true) {
+1
}
+1 // another, just in case ;)
thanks
Prasad
On Fri, Mar 7, 2014 at 6:47 AM, kulkarni.swar...@gmail.com
kulkarni.swar...@gmail.com wrote:
+1
On Fri, Mar 7, 2014 at 1:05 AM, Thejas Nair the...@hortonworks.com
wrote:
Should we start with moving our primary source code repository from
svn to git ? I feel git is more powerful and easy to use (once you go
past the learning curve!).
On Wed, Mar 5, 2014 at 7:39 AM, Brock Noland br...@cloudera.com
wrote:
Personally I prefer the Github workflow, but I believe there have
been
some challenges with that since the source for apache projects must
be
stored in apache source control (git or svn).
Relevent:
https://blogs.apache.org/infra/entry/improved_integration_between_apache_and
On Wed, Mar 5, 2014 at 9:19 AM, kulkarni.swar...@gmail.com
kulkarni.swar...@gmail.com wrote:
Hello,
Since we have a nice mirrored git repository for hive[1], any
specific
reason why we can't switch to doing pull requests instead of
patches?
IMHO
pull requests are awesome for peer review plus it is also very
easy
to
keep
track of JIRAs with open pull requests instead of looking for
JIRAs
in a
Patch Available state. Also since they get updated
automatically,
it
is
also very easy to see if a review comment made by a reviewer was
addressed
properly or not.
Thoughts?
Thanks,
[1] https://github.com/apache/hive
--
Swarnim
--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or
entity
to
which it is addressed and may contain information that is
confidential,
privileged and exempt from disclosure under applicable law. If the
reader
of this message is not the intended recipient, you are hereby
notified
that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender
immediately
and delete it from your system. Thank You.
--
Swarnim
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
[jira] [Updated] (HIVE-6541) Need to write documentation for ACID work
[ https://issues.apache.org/jira/browse/HIVE-6541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alan Gates updated HIVE-6541: - Attachment: hive-6541-changesAfterFirstEdit.rtf Need to write documentation for ACID work - Key: HIVE-6541 URL: https://issues.apache.org/jira/browse/HIVE-6541 Project: Hive Issue Type: Sub-task Components: Documentation Affects Versions: 0.13.0 Reporter: Alan Gates Assignee: Alan Gates Fix For: 0.13.0 Attachments: hive-6541-changesAfterFirstEdit.rtf, hive-6541-firstEdit.rtf, hive-6541.txt ACID introduces a number of new config file options, tables in the metastore, keywords in the grammar, and a new interface for use of tools like storm and flume. These need to be documented. -- This message was sent by Atlassian JIRA (v6.2#6252)
