[
https://issues.apache.org/jira/browse/SENTRY-810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14642005#comment-14642005
]
Bruce Nelson commented on SENTRY-810:
-------------------------------------
Sentry-810 issue verified : My test case was an extension of the HIVE-11319
test case using with Sentry.
The test case I used was 2 separate users, one user (user1) who is assigned
demo_role and another (user2) who has bn_role. bn_role has very limited
permissions - only enough to read the source table (custlog).
First I verified the HIVE-11319 test creating 2 CTAS from table custlog and
proving the overwrite in the default db with user1 :
[user1@testbed07 ~]$ kinit user1
Password for [email protected]:
[user1@testbed07 ~]$ hadoop fs -mkdir ctastest
[user1@testbed07 ~]$ ./beeline.sh
scan complete in 17ms
Connecting to
jdbc:hive2://testbed03:10000/default;principal=hive/[email protected]
15/07/25 23:04:52 INFO Utils: Supplied authorities: testbed03:10000
15/07/25 23:04:52 INFO Utils: Resolved authority: testbed03:10000
15/07/25 23:04:55 INFO HiveConnection: Will try to open client transport with
JDBC Uri:
jdbc:hive2://testbed03:10000/default;principal=hive/[email protected]
Connected to: Apache Hive (version 1.1.0-cdh5.4.4)
Driver: Hive JDBC (version 1.1.0-cdh5.4.4)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.1.0-cdh5.4.4 by Apache Hive
0: jdbc:hive2://testbed03:10000/default> show current roles;
+------------+--+
| role |
+------------+--+
| demo_role |
+------------+--+
1 row selected (0.165 seconds)
0: jdbc:hive2://testbed03:10000/default> show grant role demo_role;
+------------------------------+--------+------------+---------+-----------------+-----------------+------------+---------------+-------------------+----------+--+
| database | table | partition | column | principal_name
| principal_type | privilege | grant_option | grant_time | grantor |
+------------------------------+--------+------------+---------+-----------------+-----------------+------------+---------------+-------------------+----------+--+
| hdfs:///user/user1/custlog | | | | demo_role
| ROLE | * | true | 1429391131163000 | -- |
| default | | | | demo_role
| ROLE | * | false | 1429383219431000 | -- |
| user1test | | | | demo_role
| ROLE | * | false | 1429389518007000 | -- |
| * | | | | demo_role
| ROLE | all | true | 1429382610571000 | -- |
+------------------------------+--------+------------+---------+-----------------+-----------------+------------+---------------+-------------------+----------+--+
4 rows selected (0.271 seconds)
0: jdbc:hive2://testbed03:10000/default> create table ctas1 location
'/user/user1/ctastest'as select * from custlog limit 7 ;
INFO : Number of reduce tasks determined at compile time: 1
INFO : In order to change the average load for a reducer (in bytes):
INFO : set hive.exec.reducers.bytes.per.reducer=<number>
INFO : In order to limit the maximum number of reducers:
INFO : set hive.exec.reducers.max=<number>
INFO : In order to set a constant number of reducers:
INFO : set mapreduce.job.reduces=<number>
WARN : Hadoop command-line option parsing not performed. Implement the Tool
interface and execute your application with ToolRunner to remedy this.
INFO : number of splits:1
INFO : Submitting tokens for job: job_1437876082875_0002
INFO : Kind: HDFS_DELEGATION_TOKEN, Service: 192.168.56.126:8020, Ident:
(HDFS_DELEGATION_TOKEN token 1329 for hive)
INFO : Kind: kms-dt, Service: 192.168.56.125:16000, Ident: 00 04 68 69 76 65
04 79 61 72 6e 00 8a 01 4e c8 fd ec cb 8a 01 4e ed 0a 70 cb 02 02
INFO : The url to track the job:
https://testbed06:8090/proxy/application_1437876082875_0002/
INFO : Starting Job = job_1437876082875_0002, Tracking URL =
https://testbed06:8090/proxy/application_1437876082875_0002/
INFO : Kill Command =
/opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/lib/hadoop/bin/hadoop job
-kill job_1437876082875_0002
INFO : Hadoop job information for Stage-1: number of mappers: 1; number of
reducers: 1
INFO : 2015-07-25 23:13:22,020 Stage-1 map = 0%, reduce = 0%
INFO : 2015-07-25 23:13:52,528 Stage-1 map = 100%, reduce = 0%, Cumulative
CPU 3.44 sec
INFO : 2015-07-25 23:14:23,464 Stage-1 map = 100%, reduce = 100%, Cumulative
CPU 10.79 sec
INFO : MapReduce Total cumulative CPU time: 10 seconds 790 msec
INFO : Ended Job = job_1437876082875_0002
INFO : Moving data to: /user/user1/ctastest from
hdfs://testbed06:8020/user/hive/warehouse/.hive-staging_hive_2015-07-25_23-12-40_150_5841773288455581443-3/-ext-10001
INFO : Table default.ctas1 stats: [numFiles=1, numRows=7, totalSize=718,
rawDataSize=711]
No rows affected (106.183 seconds)
0: jdbc:hive2://testbed03:10000/default> select count(*) from ctas1;
+------+--+
| _c0 |
+------+--+
| 7 |
+------+--+
1 row selected (0.808 seconds)
0: jdbc:hive2://testbed03:10000/default> create table ctas2 location
'/user/user1/ctastest'as select * from custlog limit 5 ;
INFO : Number of reduce tasks determined at compile time: 1
INFO : In order to change the average load for a reducer (in bytes):
INFO : set hive.exec.reducers.bytes.per.reducer=<number>
INFO : In order to limit the maximum number of reducers:
INFO : set hive.exec.reducers.max=<number>
INFO : In order to set a constant number of reducers:
INFO : set mapreduce.job.reduces=<number>
WARN : Hadoop command-line option parsing not performed. Implement the Tool
interface and execute your application with ToolRunner to remedy this.
INFO : number of splits:1
INFO : Submitting tokens for job: job_1437876082875_0003
INFO : Kind: HDFS_DELEGATION_TOKEN, Service: 192.168.56.126:8020, Ident:
(HDFS_DELEGATION_TOKEN token 1330 for hive)
INFO : Kind: kms-dt, Service: 192.168.56.125:16000, Ident: 00 04 68 69 76 65
04 79 61 72 6e 00 8a 01 4e c9 01 10 45 8a 01 4e ed 0d 94 45 03 02
INFO : The url to track the job:
https://testbed06:8090/proxy/application_1437876082875_0003/
INFO : Starting Job = job_1437876082875_0003, Tracking URL =
https://testbed06:8090/proxy/application_1437876082875_0003/
INFO : Kill Command =
/opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/lib/hadoop/bin/hadoop job
-kill job_1437876082875_0003
INFO : Hadoop job information for Stage-1: number of mappers: 1; number of
reducers: 1
INFO : 2015-07-25 23:16:47,274 Stage-1 map = 0%, reduce = 0%
INFO : 2015-07-25 23:17:19,876 Stage-1 map = 100%, reduce = 0%, Cumulative
CPU 3.32 sec
INFO : 2015-07-25 23:17:51,874 Stage-1 map = 100%, reduce = 100%, Cumulative
CPU 11.3 sec
INFO : MapReduce Total cumulative CPU time: 11 seconds 300 msec
INFO : Ended Job = job_1437876082875_0003
INFO : Moving data to: /user/user1/ctastest from
hdfs://testbed06:8020/user/hive/warehouse/.hive-staging_hive_2015-07-25_23-16-06_056_7354792874232933890-3/-ext-10001
INFO : Table default.ctas2 stats: [numFiles=1, numRows=5, totalSize=510,
rawDataSize=505]
No rows affected (108.637 seconds)
0: jdbc:hive2://testbed03:10000/default> select count(*) from ctas1;
+------+--+
| _c0 |
+------+--+
| 7 |
+------+--+
1 row selected (0.446 seconds)
0: jdbc:hive2://testbed03:10000/default> select count(*) from ctas2;
+------+--+
| _c0 |
+------+--+
| 5 |
+------+--+
1 row selected (0.424 seconds)
0: jdbc:hive2://testbed03:10000/default> select * from ctas1;
+----------------------------------------------------------------------------------------------------------+--+
| ctas1.click
|
+----------------------------------------------------------------------------------------------------------+--+
|
{"custid":1010220,"transid":11547,"typeid":6,"time":"2012-07-01:00:00:42","teller":"Y","activity":6}
|
|
{"custid":1234182,"transid":11547,"typeid":6,"time":"2012-07-01:00:00:32","teller":"Y","activity":7}
|
|
{"custid":1083711,"transid":null,"typeid":null,"time":"2012-07-01:00:00:26","teller":null,"activity":9}
|
|
{"custid":1354924,"transid":1948,"typeid":9,"time":"2012-07-01:00:00:22","teller":"N","activity":7}
|
|
{"custid":1185972,"transid":null,"typeid":null,"time":"2012-07-01:00:00:07","teller":null,"activity":8}
|
+----------------------------------------------------------------------------------------------------------+--+
5 rows selected (0.676 seconds)
0: jdbc:hive2://testbed03:10000/default> select * from ctas2;
+----------------------------------------------------------------------------------------------------------+--+
| ctas2.click
|
+----------------------------------------------------------------------------------------------------------+--+
|
{"custid":1010220,"transid":11547,"typeid":6,"time":"2012-07-01:00:00:42","teller":"Y","activity":6}
|
|
{"custid":1234182,"transid":11547,"typeid":6,"time":"2012-07-01:00:00:32","teller":"Y","activity":7}
|
|
{"custid":1083711,"transid":null,"typeid":null,"time":"2012-07-01:00:00:26","teller":null,"activity":9}
|
|
{"custid":1354924,"transid":1948,"typeid":9,"time":"2012-07-01:00:00:22","teller":"N","activity":7}
|
|
{"custid":1185972,"transid":null,"typeid":null,"time":"2012-07-01:00:00:07","teller":null,"activity":8}
|
+----------------------------------------------------------------------------------------------------------+--+
5 rows selected (0.469 seconds)
0: jdbc:hive2://testbed03:10000/default> !quit
Closing: 0:
jdbc:hive2://testbed03:10000/default;principal=hive/[email protected]
[user1@testbed07 ~]$
[user1@testbed07 ~]$
I then switched to user2 (via kinit) and tried a third CTAS in the "bn" db with
the expectation that I would be blocked :
[user1@testbed07 ~]$ kinit user2
Password for [email protected]:
[user1@testbed07 ~]$ hadoop fs -ls /user/user1/ctastest
Found 1 items
-rwxrwxr-x+ 3 hive user1 510 2015-07-25 23:17
/user/user1/ctastest/000000_0
[user1@testbed07 ~]$ ./beeline.sh
scan complete in 16ms
Connecting to
jdbc:hive2://testbed03:10000/default;principal=hive/[email protected]
15/07/25 23:22:11 INFO Utils: Supplied authorities: testbed03:10000
15/07/25 23:22:11 INFO Utils: Resolved authority: testbed03:10000
15/07/25 23:22:14 INFO HiveConnection: Will try to open client transport with
JDBC Uri:
jdbc:hive2://testbed03:10000/default;principal=hive/[email protected]
Connected to: Apache Hive (version 1.1.0-cdh5.4.4)
Driver: Hive JDBC (version 1.1.0-cdh5.4.4)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.1.0-cdh5.4.4 by Apache Hive
0: jdbc:hive2://testbed03:10000/default> create table bn.ctas3 location
'/user/user1/ctastest'as select * from custlog limit 2 ;
INFO : Number of reduce tasks determined at compile time: 1
INFO : In order to change the average load for a reducer (in bytes):
INFO : set hive.exec.reducers.bytes.per.reducer=<number>
INFO : In order to limit the maximum number of reducers:
INFO : set hive.exec.reducers.max=<number>
INFO : In order to set a constant number of reducers:
INFO : set mapreduce.job.reduces=<number>
WARN : Hadoop command-line option parsing not performed. Implement the Tool
interface and execute your application with ToolRunner to remedy this.
INFO : number of splits:1
INFO : Submitting tokens for job: job_1437876082875_0007
INFO : Kind: HDFS_DELEGATION_TOKEN, Service: 192.168.56.126:8020, Ident:
(HDFS_DELEGATION_TOKEN token 1349 for hive)
INFO : Kind: kms-dt, Service: 192.168.56.125:16000, Ident: 00 04 68 69 76 65
04 79 61 72 6e 00 8a 01 4e c9 07 59 72 8a 01 4e ed 13 dd 72 12 02
INFO : The url to track the job:
https://testbed06:8090/proxy/application_1437876082875_0007/
INFO : Starting Job = job_1437876082875_0007, Tracking URL =
https://testbed06:8090/proxy/application_1437876082875_0007/
INFO : Kill Command =
/opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/lib/hadoop/bin/hadoop job
-kill job_1437876082875_0007
INFO : Hadoop job information for Stage-1: number of mappers: 1; number of
reducers: 1
INFO : 2015-07-25 23:23:39,048 Stage-1 map = 0%, reduce = 0%
INFO : 2015-07-25 23:24:13,153 Stage-1 map = 100%, reduce = 0%, Cumulative
CPU 4.48 sec
INFO : 2015-07-25 23:24:32,800 Stage-1 map = 100%, reduce = 100%, Cumulative
CPU 13.55 sec
INFO : MapReduce Total cumulative CPU time: 13 seconds 550 msec
INFO : Ended Job = job_1437876082875_0007
INFO : Moving data to: /user/user1/ctastest from
hdfs://testbed06:8020/user/hive/warehouse/.hive-staging_hive_2015-07-25_23-22-57_379_6754961746539787781-6/-ext-10001
INFO : Table bn.ctas3 stats: [numFiles=1, numRows=2, totalSize=204,
rawDataSize=202]
No rows affected (97.288 seconds)
I prove that Sentry permissions are in tact for ctas1 and ctas2, user2 has no
rights to user1's tables :
0: jdbc:hive2://testbed03:10000/default> select count(*) from ctas1;
Error: Error while compiling statement: FAILED: SemanticException No valid
privileges
Required privileges for this query:
Server=server1->Db=default->Table=ctas1->action=select; (state=42000,code=40000)
0: jdbc:hive2://testbed03:10000/default> select count(*) from ctas2;
Error: Error while compiling statement: FAILED: SemanticException No valid
privileges
Required privileges for this query:
Server=server1->Db=default->Table=ctas2->action=select; (state=42000,code=40000)
But I do have a successful overwrite at the file level :
0: jdbc:hive2://testbed03:10000/default> select count(*) from bn.ctas3;
+------+--+
| _c0 |
+------+--+
| 2 |
+------+--+
1 row selected (0.498 seconds)
0: jdbc:hive2://testbed03:10000/default> show current roles;
+----------+--+
| role |
+----------+--+
| bn_role |
+----------+--+
1 row selected (0.551 seconds)
0: jdbc:hive2://testbed03:10000/default> show grant role bn_role;
+-----------+----------+------------+---------+-----------------+-----------------+------------+---------------+-------------------+----------+--+
| database | table | partition | column | principal_name |
principal_type | privilege | grant_option | grant_time | grantor |
+-----------+----------+------------+---------+-----------------+-----------------+------------+---------------+-------------------+----------+--+
| default | custlog | | | bn_role | ROLE
| select | false | 1437792683410000 | -- |
| bn | | | | bn_role | ROLE
| all | true | 1429603022918000 | -- |
+-----------+----------+------------+---------+-----------------+-----------------+------------+---------------+-------------------+----------+--+
2 rows selected (0.203 seconds)
0: jdbc:hive2://testbed03:10000/default> !quit
Closing: 0:
jdbc:hive2://testbed03:10000/default;principal=hive/[email protected]
I switch back to user1 and find my tables are corrupted by user2.:
[user1@testbed07 ~]$ kinit user1
Password for [email protected]:
[user1@testbed07 ~]$ ./beeline.sh
scan complete in 53ms
Connecting to
jdbc:hive2://testbed03:10000/default;principal=hive/[email protected]
15/07/25 23:26:04 INFO Utils: Supplied authorities: testbed03:10000
15/07/25 23:26:04 INFO Utils: Resolved authority: testbed03:10000
15/07/25 23:26:08 INFO HiveConnection: Will try to open client transport with
JDBC Uri:
jdbc:hive2://testbed03:10000/default;principal=hive/[email protected]
Connected to: Apache Hive (version 1.1.0-cdh5.4.4)
Driver: Hive JDBC (version 1.1.0-cdh5.4.4)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 1.1.0-cdh5.4.4 by Apache Hive
0: jdbc:hive2://testbed03:10000/default> select count(*) from ctas1;
+------+--+
| _c0 |
+------+--+
| 7 |
+------+--+
1 row selected (0.773 seconds)
0: jdbc:hive2://testbed03:10000/default> select count(*) from ctas2;
+------+--+
| _c0 |
+------+--+
| 5 |
+------+--+
1 row selected (0.611 seconds)
0: jdbc:hive2://testbed03:10000/default> select * from ctas1;
+----------------------------------------------------------------------------------------------------------+--+
| ctas1.click
|
+----------------------------------------------------------------------------------------------------------+--+
|
{"custid":1354924,"transid":1948,"typeid":9,"time":"2012-07-01:00:00:22","teller":"N","activity":7}
|
|
{"custid":1185972,"transid":null,"typeid":null,"time":"2012-07-01:00:00:07","teller":null,"activity":8}
|
+----------------------------------------------------------------------------------------------------------+--+
2 rows selected (0.444 seconds)
0: jdbc:hive2://testbed03:10000/default> select * from ctas2;
+----------------------------------------------------------------------------------------------------------+--+
| ctas2.click
|
+----------------------------------------------------------------------------------------------------------+--+
|
{"custid":1354924,"transid":1948,"typeid":9,"time":"2012-07-01:00:00:22","teller":"N","activity":7}
|
|
{"custid":1185972,"transid":null,"typeid":null,"time":"2012-07-01:00:00:07","teller":null,"activity":8}
|
+----------------------------------------------------------------------------------------------------------+--+
2 rows selected (0.639 seconds)
0: jdbc:hive2://testbed03:10000/default> !quit
Closing: 0:
jdbc:hive2://testbed03:10000/default;principal=hive/[email protected]
[user1@testbed07 ~]$
> CTAS vulnerability
> -------------------
>
> Key: SENTRY-810
> URL: https://issues.apache.org/jira/browse/SENTRY-810
> Project: Sentry
> Issue Type: Bug
> Components: Hive Plugin
> Reporter: Ryan P
> Assignee: Ryan P
> Attachments: SENTRY-810.patch
>
>
> HIVE-11319 puts us in an awkward situation where you can leverage your
> current permissions to overwrite existing directories. I recommend we edit
> the CTAS permissions map to reflect that of a true insert overwrite
> directory. This will be annoying for users trying to do legit CTAS operations
> within the warehouse but as it stands it leave a gaping hole in the security
> model
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
