[jira] [Commented] (HIVE-5056) MapJoinProcessor ignores order of values in removing RS
[ https://issues.apache.org/jira/browse/HIVE-5056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13736276#comment-13736276 ] Chen Chun commented on HIVE-5056: - Hi, [~navis]. I and [~wzc1989] finally find the problem after a week of seeking. Since deal.hashCode()%16=12, dim_pay_date.hashCode()%16=12, in SemanticAnalyzer#genJoinOperatorChildren, we put deal, dim_pay_date in hash map in sequence, however in MapJoinProcessor#convertMapJoin, we put dim_pay_date, deal in hash map in sequence. And because of hash collision, the traversal order is related to the order of elements putting in the hashmap and result in bug. MapJoinProcessor ignores order of values in removing RS --- Key: HIVE-5056 URL: https://issues.apache.org/jira/browse/HIVE-5056 Project: Hive Issue Type: Bug Components: Query Processor Reporter: Navis Assignee: Navis Attachments: HIVE-5056.D12147.1.patch http://www.mail-archive.com/user@hive.apache.org/msg09073.html -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-2817) Drop any table even without privilege
[
https://issues.apache.org/jira/browse/HIVE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen Chun updated HIVE-2817:
Fix Version/s: 0.11.0
Affects Version/s: 0.8.0
0.9.0
0.10.0
Release Note: Fix bug drop any table even without privilege
Hadoop Flags: Incompatible change
Status: Patch Available (was: Open)
Drop any table even without privilege
-
Key: HIVE-2817
URL: https://issues.apache.org/jira/browse/HIVE-2817
Project: Hive
Issue Type: Bug
Affects Versions: 0.10.0, 0.9.0, 0.8.0, 0.7.1
Reporter: Benyi Wang
Assignee: Chen Chun
Fix For: 0.11.0
Attachments: HIVE-2817.D10371.1.patch, HIVE-2817.D10563.1.patch
You can drop any table if you use fully qualified name 'database.table' even
you don't have any previlige.
{code}
hive set hive.security.authorization.enabled=true;
hive revoke all on default from user test_user;
hive drop table abc;
hive drop table abc;
Authorization failed:No privilege 'Drop' found for outputs {
database:default, table:abc}. Use show grant to get more details.
hive drop table default.abc;
OK
Time taken: 0.13 seconds
{code}
The table and the file in {{/usr/hive/warehouse}} or external file will be
deleted. If you don't have hadoop access permission on
{{/usr/hive/warehouse}} or external files, you will see a hadoop access error
{code}
12/02/23 15:35:35 ERROR hive.log:
org.apache.hadoop.security.AccessControlException:
org.apache.hadoop.security.AccessControlException: Permission denied:
user=test_user, access=WRITE, inode=/user/myetl:myetl:etl:drwxr-xr-x
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HIVE-2817) Drop any table even without privilege
[
https://issues.apache.org/jira/browse/HIVE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen Chun reassigned HIVE-2817:
---
Assignee: Chen Chun
Drop any table even without privilege
-
Key: HIVE-2817
URL: https://issues.apache.org/jira/browse/HIVE-2817
Project: Hive
Issue Type: Bug
Affects Versions: 0.7.1
Reporter: Benyi Wang
Assignee: Chen Chun
Attachments: HIVE-2817.D10371.1.patch
You can drop any table if you use fully qualified name 'database.table' even
you don't have any previlige.
{code}
hive set hive.security.authorization.enabled=true;
hive revoke all on default from user test_user;
hive drop table abc;
hive drop table abc;
Authorization failed:No privilege 'Drop' found for outputs {
database:default, table:abc}. Use show grant to get more details.
hive drop table default.abc;
OK
Time taken: 0.13 seconds
{code}
The table and the file in {{/usr/hive/warehouse}} or external file will be
deleted. If you don't have hadoop access permission on
{{/usr/hive/warehouse}} or external files, you will see a hadoop access error
{code}
12/02/23 15:35:35 ERROR hive.log:
org.apache.hadoop.security.AccessControlException:
org.apache.hadoop.security.AccessControlException: Permission denied:
user=test_user, access=WRITE, inode=/user/myetl:myetl:etl:drwxr-xr-x
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-2817) Drop any table even without privilege
[
https://issues.apache.org/jira/browse/HIVE-2817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624911#comment-13624911
]
Chen Chun commented on HIVE-2817:
-
{code}
// I think the following code in DDLSemanticAnalyzer.analyzeDropTable cause the
problem
// if we set hive.exec.drop.ignorenonexistent=true, then throwException=false
and so the inputs and outputs are empty
// And it will not do authorization check after semantic analyze.
try {
Table tab = db.getTable(db.getCurrentDatabase(), tableName,
throwException);
if (tab != null) {
inputs.add(new ReadEntity(tab));
outputs.add(new WriteEntity(tab));
}
} catch (HiveException e) {
throw new SemanticException(ErrorMsg.INVALID_TABLE.getMsg(tableName));
}
{code}
A easy way to fix the problem is to set hive.exec.drop.ignorenonexistent=false.
When I test the sql drop table default.src_authorization_8 with ant, the log
/PERFLOG method=doAuthorization start=1365321309304 end=1365321309304
duration=0 duration=0 leave a clue of what I said.
{code}
2013-04-07 00:55:09,279 DEBUG parse.VariableSubstitution
(VariableSubstitution.java:substitute(67)) - Substitution is on:
drop table default.src_authorization_8
2013-04-07 00:55:09,280 INFO parse.ParseDriver (ParseDriver.java:parse(179)) -
Parsing command:
drop table default.src_authorization_8
2013-04-07 00:55:09,282 INFO parse.ParseDriver (ParseDriver.java:parse(197)) -
Parse Completed
2013-04-07 00:55:09,282 INFO metastore.HiveMetaStore
(HiveMetaStore.java:logInfo(452)) - 0: get_table : db=default
tbl=default.src_authorization_8
2013-04-07 00:55:09,286 INFO HiveMetaStore.audit
(HiveMetaStore.java:logAuditEvent(238)) - ugi=chenchun
ip=unknown-ip-addr cmd=get_table : db=default tbl=default.s
rc_authorization_8
2013-04-07 00:55:09,302 ERROR metastore.RetryingHMSHandler
(RetryingHMSHandler.java:invoke(132)) -
NoSuchObjectException(message:default.default.src_authorization_8 table not f
ound)
at
org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.get_table(HiveMetaStore.java:1369)
at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:105)
at $Proxy8.get_table(Unknown Source)
at
org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getTable(HiveMetaStoreClient.java:838)
at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:74)
at $Proxy9.getTable(Unknown Source)
at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:948)
at
org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.getTable(DDLSemanticAnalyzer.java:3032)
at
org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.getTable(DDLSemanticAnalyzer.java:3026)
at
org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeDropTable(DDLSemanticAnalyzer.java:737)
at
org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:228)
at
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:259)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:433)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348)
at org.apache.hadoop.hive.ql.QTestUtil.executeClient(QTestUtil.java:797)
at
org.apache.hadoop.hive.cli.TestCliDriver.runTest(TestCliDriver.java:124)
at
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_8(TestCliDriver.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:154)
at junit.framework.TestCase.runBare(TestCase.java:127)
at junit.framework.TestResult$1.protect(TestResult.java:106)
at
[jira] [Commented] (HIVE-4174) Round UDF converts BigInts to double
[
https://issues.apache.org/jira/browse/HIVE-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13624912#comment-13624912
]
Chen Chun commented on HIVE-4174:
-
Thanks, [~mgrover], [~namit]. By the way, I think u noticed a wrong person,
[~namit] is not [~namitjain]. And can I assign the issue HIVE-2817 to me. I
recently encounter the problem and find the cause of it.
Round UDF converts BigInts to double
Key: HIVE-4174
URL: https://issues.apache.org/jira/browse/HIVE-4174
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.10.0
Reporter: Mark Grover
Assignee: Chen Chun
Fix For: 0.11.0
Attachments: hive.4174.1.patch-nohcat, HIVE-4174.1.patch.txt,
HIVE-4174.D9687.1.patch
Chen Chun pointed out on the hive-user mailing list that round() in Hive 0.10
returns
{code}
select round(cast(1234560 as BIGINT)), round(cast(12345670 as BIGINT)) from
test limit 1;
//hive 0.10
1234560.0 1.234567E7
{code}
This is not consistent with
MySQL(http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html#function_round)
which quotes
{code}
The return type is the same type as that of the first argument (assuming that
it is integer, double, or decimal). This means that for an integer argument,
the result is an integer (no decimal places)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4174) Round UDF converts BigInts to double
[
https://issues.apache.org/jira/browse/HIVE-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen Chun updated HIVE-4174:
Attachment: HIVE-4174.1.patch.txt
Round UDF converts BigInts to double
Key: HIVE-4174
URL: https://issues.apache.org/jira/browse/HIVE-4174
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.10.0
Reporter: Mark Grover
Assignee: Mark Grover
Fix For: 0.11.0
Attachments: HIVE-4174.1.patch.txt
Chen Chun pointed out on the hive-user mailing list that round() in Hive 0.10
returns
{code}
select round(cast(1234560 as BIGINT)), round(cast(12345670 as BIGINT)) from
test limit 1;
//hive 0.10
1234560.0 1.234567E7
{code}
This is not consistent with
MySQL(http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html#function_round)
which quotes
{code}
The return type is the same type as that of the first argument (assuming that
it is integer, double, or decimal). This means that for an integer argument,
the result is an integer (no decimal places)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HIVE-4174) Round UDF converts BigInts to double
[
https://issues.apache.org/jira/browse/HIVE-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chen Chun updated HIVE-4174:
Release Note: The change makes the return type of round function be the
same type as that of the first argument (assuming that it is integer, double,
or decimal). This means that for an integer argument, the result is an integer
(no decimal places).
Hadoop Flags: Incompatible change
Status: Patch Available (was: Open)
Round UDF converts BigInts to double
Key: HIVE-4174
URL: https://issues.apache.org/jira/browse/HIVE-4174
Project: Hive
Issue Type: Bug
Components: UDF
Affects Versions: 0.10.0
Reporter: Mark Grover
Assignee: Mark Grover
Fix For: 0.11.0
Attachments: HIVE-4174.1.patch.txt
Chen Chun pointed out on the hive-user mailing list that round() in Hive 0.10
returns
{code}
select round(cast(1234560 as BIGINT)), round(cast(12345670 as BIGINT)) from
test limit 1;
//hive 0.10
1234560.0 1.234567E7
{code}
This is not consistent with
MySQL(http://dev.mysql.com/doc/refman/5.1/en/mathematical-functions.html#function_round)
which quotes
{code}
The return type is the same type as that of the first argument (assuming that
it is integer, double, or decimal). This means that for an integer argument,
the result is an integer (no decimal places)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
