[jira] [Created] (SENTRY-2158) Update notification handler for grant privileges to user

2018-03-06 Thread Na Li (JIRA)
Na Li created SENTRY-2158:
-

 Summary: Update notification handler for grant privileges to user
 Key: SENTRY-2158
 URL: https://issues.apache.org/jira/browse/SENTRY-2158
 Project: Sentry
  Issue Type: Sub-task
  Components: Sentry
Affects Versions: 2.1.0
Reporter: Na Li
 Fix For: 2.1.0


Update notification handler in the following files when grantting privileges to 
user

 

sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandler.java

sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandlerInvoker.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (SENTRY-2159) Add e2e tests for granting privileges to user

2018-03-06 Thread Na Li (JIRA)
Na Li created SENTRY-2159:
-

 Summary: Add e2e tests for granting privileges to user
 Key: SENTRY-2159
 URL: https://issues.apache.org/jira/browse/SENTRY-2159
 Project: Sentry
  Issue Type: Sub-task
  Components: Sentry
Affects Versions: 2.1.0
Reporter: Na Li
 Fix For: 2.1.0


Need to add test cases in e2e for granting privileges to user



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (SENTRY-2152) Only Admin can set dbproperty 'owner.privileges'

2018-03-06 Thread Na Li (JIRA)
Na Li created SENTRY-2152:
-

 Summary: Only Admin can set dbproperty 'owner.privileges'
 Key: SENTRY-2152
 URL: https://issues.apache.org/jira/browse/SENTRY-2152
 Project: Sentry
  Issue Type: Sub-task
  Components: Sentry
Affects Versions: 2.1.0
Reporter: Na Li
 Fix For: 2.1.0


Right now, anyone can set database property. We should add authorization when 
setting dbproperty 'owner.privileges' from hive, so only admin can do it.
The allowed values are: none, all, all with grant

For example, when a non-admin user issues command
{code:java}
alter database db1 set dbproperty('owner.privileges'='all with grant')
{code}
The command should fail and the error message indicating the user does not have 
right to set this value.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (SENTRY-2153) Get owner.privileges value from hive for a given DB

2018-03-06 Thread Na Li (JIRA)
Na Li created SENTRY-2153:
-

 Summary: Get owner.privileges value from hive for a given DB
 Key: SENTRY-2153
 URL: https://issues.apache.org/jira/browse/SENTRY-2153
 Project: Sentry
  Issue Type: Sub-task
  Components: Sentry
Affects Versions: 2.1.0
Reporter: Na Li
 Fix For: 2.1.0


Sentry needs to get the "owner.privileges" value from hive for a given DB, and 
cache it. So when a new table is created, the user can get implicit owner 
privileges accordingly.

When the value of "owner.privileges changes, Sentry should update the cache 
value to the latest value.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (SENTRY-2158) Update notification handler for grant privileges to user

2018-03-06 Thread JIRA

[ 
https://issues.apache.org/jira/browse/SENTRY-2158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388681#comment-16388681
 ] 

Sergio Peña commented on SENTRY-2158:
-

What is it that needs to be updated? I think we need a better description on 
this jira.

> Update notification handler for grant privileges to user
> 
>
> Key: SENTRY-2158
> URL: https://issues.apache.org/jira/browse/SENTRY-2158
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Priority: Major
> Fix For: 2.1.0
>
>
> Update notification handler in the following files when grantting privileges 
> to user
>  
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandler.java
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandlerInvoker.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (SENTRY-2159) Add e2e tests for granting implicit owner privileges

2018-03-06 Thread Na Li (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Na Li updated SENTRY-2159:
--
Description: 
Need to add e2e test cases for granting implicit owner privileges for newly 
created table

The test case can 

1) Set db owner.privileges to all

2) grant user_role create table privilege to create table on db1

3) user with user_role create a table table1 in db1

4) verify user with user_role can insert row in table1

  was:Need to add test cases in e2e for granting implicit owner privileges for 
newly created table


> Add e2e tests for granting implicit owner privileges
> 
>
> Key: SENTRY-2159
> URL: https://issues.apache.org/jira/browse/SENTRY-2159
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Priority: Major
> Fix For: 2.1.0
>
>
> Need to add e2e test cases for granting implicit owner privileges for newly 
> created table
> The test case can 
> 1) Set db owner.privileges to all
> 2) grant user_role create table privilege to create table on db1
> 3) user with user_role create a table table1 in db1
> 4) verify user with user_role can insert row in table1



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (SENTRY-2154) Update schema to grant privileges to user

2018-03-06 Thread Na Li (JIRA)
Na Li created SENTRY-2154:
-

 Summary: Update schema to grant privileges to user
 Key: SENTRY-2154
 URL: https://issues.apache.org/jira/browse/SENTRY-2154
 Project: Sentry
  Issue Type: Sub-task
  Components: Sentry
Affects Versions: 2.1.0
Reporter: Na Li
 Fix For: 2.1.0


Need to add new DB table to support grant user to privileges
Also, a flag should be added in privilege table to indicate the privilege is 
created by user, or created by sentry implicitly. User can view the implicit 
privileges, but cannot change it directly



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (SENTRY-2157) Update audit log to grant privilege to user

2018-03-06 Thread Na Li (JIRA)
Na Li created SENTRY-2157:
-

 Summary: Update audit log to grant privilege to user
 Key: SENTRY-2157
 URL: https://issues.apache.org/jira/browse/SENTRY-2157
 Project: Sentry
  Issue Type: Sub-task
  Components: Sentry
Affects Versions: 2.1.0
Reporter: Na Li
 Fix For: 2.1.0


Update audit log to grant privileges to user



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (SENTRY-2158) Update notification handler for grant privileges to user

2018-03-06 Thread Na Li (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-2158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Na Li updated SENTRY-2158:
--
Description: 
SentryPolicyStoreProcessor calls NotificationHandlerInvoker when processing 
permission related commands. We should update notification handler in the 
following files when granting privileges to user. 

 

sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandler.java

sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandlerInvoker.java

  was:
Update notification handler in the following files when grantting privileges to 
user

 

sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandler.java

sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandlerInvoker.java


> Update notification handler for grant privileges to user
> 
>
> Key: SENTRY-2158
> URL: https://issues.apache.org/jira/browse/SENTRY-2158
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Priority: Major
> Fix For: 2.1.0
>
>
> SentryPolicyStoreProcessor calls NotificationHandlerInvoker when processing 
> permission related commands. We should update notification handler in the 
> following files when granting privileges to user. 
>  
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandler.java
> sentry-provider/sentry-provider-db/src/main/java/org/apache/sentry/provider/db/service/thrift/NotificationHandlerInvoker.java



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (SENTRY-2155) Update JDO to grant privileges to user

2018-03-06 Thread Na Li (JIRA)
Na Li created SENTRY-2155:
-

 Summary: Update JDO to grant privileges to user
 Key: SENTRY-2155
 URL: https://issues.apache.org/jira/browse/SENTRY-2155
 Project: Sentry
  Issue Type: Sub-task
  Components: Sentry
Affects Versions: 2.1.0
Reporter: Na Li
 Fix For: 2.1.0


Update JDO to match the update in DB schema in order to grant user to privileges



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (SENTRY-2160) Add owner in create table notification event

2018-03-06 Thread Na Li (JIRA)
Na Li created SENTRY-2160:
-

 Summary: Add owner in create table notification event
 Key: SENTRY-2160
 URL: https://issues.apache.org/jira/browse/SENTRY-2160
 Project: Sentry
  Issue Type: Sub-task
  Components: Sentry
Affects Versions: 2.1.0
Reporter: Na Li
 Fix For: 2.1.0


When creating notification event in SentryJSONCreateTableMessage, save the 
owner of the table, so sentry knows the owner and can create implicit 
privileges for the owner.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (SENTRY-2159) Add e2e tests for granting privileges to user

2018-03-06 Thread JIRA

[ 
https://issues.apache.org/jira/browse/SENTRY-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388684#comment-16388684
 ] 

Sergio Peña commented on SENTRY-2159:
-

Are these E2E test cases meant for implicit owner privileges? The feature is to 
grant implicit owner privileges, so I think it make sense to name these tests 
the same way, doesn't it?

> Add e2e tests for granting privileges to user
> -
>
> Key: SENTRY-2159
> URL: https://issues.apache.org/jira/browse/SENTRY-2159
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Priority: Major
> Fix For: 2.1.0
>
>
> Need to add test cases in e2e for granting privileges to user



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (SENTRY-2162) Implicit privileges is retrieved and used for authorization

2018-03-06 Thread Na Li (JIRA)
Na Li created SENTRY-2162:
-

 Summary: Implicit privileges is retrieved and used for 
authorization
 Key: SENTRY-2162
 URL: https://issues.apache.org/jira/browse/SENTRY-2162
 Project: Sentry
  Issue Type: Sub-task
  Components: Sentry
Affects Versions: 2.1.0
Reporter: Na Li
 Fix For: 2.1.0


Make sure the implicit privileges assigned to user directly are retried and 
applied for authorization request. This may require the code change in 
SentryPolicyStoreProcessor and several other places.

For example,

after user_A creates table_B and get "all" privilege on table_B, user_A can 
insert rows into that table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (SENTRY-2161) Make sure partial invoke only applies to explicit privileges

2018-03-06 Thread Na Li (JIRA)
Na Li created SENTRY-2161:
-

 Summary: Make sure partial invoke only applies to explicit 
privileges
 Key: SENTRY-2161
 URL: https://issues.apache.org/jira/browse/SENTRY-2161
 Project: Sentry
  Issue Type: Sub-task
Reporter: Na Li


*Background:*
Partial revoke
For examples:
1. When a role has been granted "all" on table and the role already has 
select/insert on privileges, they are removed automatically as "all" covers the 
"select/insert".
2. When a role already has "all" privileges on a table and "select" privilege 
are revoked, "all" privileges is revoked and "insert" is added automatically as 
there are only "select", "insert", and "all".

Hierarchical privileges:
Revoking privilege on a database would effect the privileges granted to the 
tables in that database.

*Problem:*
For example: 
1) User_A has "select" on table_B
2) User_A is set to owner of table_B and gets "all" privilege on table_B as 
implicit privilege
3) User_A is not owner of table_B any more

based on partial invoke behavior, User_A will lose "select" on table_B after 
step 3). The desired behavior is for User_A still retains "select" on table_B 
after step 3)

*Solution:*

Only apply partial revoke to user configured privileges (explicit privilege), 
and not affect implicit privileges.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (SENTRY-2157) Update audit log to grant/Revoke privilege to user

2018-03-06 Thread Na Li (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Na Li updated SENTRY-2157:
--
Summary: Update audit log to grant/Revoke privilege to user  (was: Update 
audit log to grant privilege to user)

> Update audit log to grant/Revoke privilege to user
> --
>
> Key: SENTRY-2157
> URL: https://issues.apache.org/jira/browse/SENTRY-2157
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Priority: Major
> Fix For: 2.1.0
>
>
> Update audit log to grant privileges to user



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (SENTRY-2156) Update provider-db backend code to grant privileges to user

2018-03-06 Thread Na Li (JIRA)
Na Li created SENTRY-2156:
-

 Summary: Update provider-db backend code to grant privileges to 
user
 Key: SENTRY-2156
 URL: https://issues.apache.org/jira/browse/SENTRY-2156
 Project: Sentry
  Issue Type: Sub-task
  Components: Sentry
Affects Versions: 2.1.0
Reporter: Na Li
 Fix For: 2.1.0


The code in provider-db back-end should be updated to grant privileges to user, 
and retrieve the privileges assigned to a user. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (SENTRY-2154) Update schema to grant privileges to user

2018-03-06 Thread JIRA

[ 
https://issues.apache.org/jira/browse/SENTRY-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388675#comment-16388675
 ] 

Sergio Peña commented on SENTRY-2154:
-

Is this patch going to display implicit owner privileges as well? or is it just 
a patch to make schema changes? Do you have an idea of what new table will be 
created and how is going to relate to the privileges table? Btw, should it be 
better to have another Jira for the implicit privilege schema changes?

> Update schema to grant privileges to user
> -
>
> Key: SENTRY-2154
> URL: https://issues.apache.org/jira/browse/SENTRY-2154
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Priority: Major
> Fix For: 2.1.0
>
>
> Need to add new DB table to support grant user to privileges
> Also, a flag should be added in privilege table to indicate the privilege is 
> created by user, or created by sentry implicitly. User can view the implicit 
> privileges, but cannot change it directly



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (SENTRY-2157) Update audit log to grant/Revoke privilege to user

2018-03-06 Thread Na Li (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Na Li updated SENTRY-2157:
--
Description: Update audit log to grant/revoke privileges to user  (was: 
Update audit log to grant privileges to user)

> Update audit log to grant/Revoke privilege to user
> --
>
> Key: SENTRY-2157
> URL: https://issues.apache.org/jira/browse/SENTRY-2157
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Priority: Major
> Fix For: 2.1.0
>
>
> Update audit log to grant/revoke privileges to user



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (SENTRY-2151) Object Ownership

2018-03-06 Thread Na Li (JIRA)
Na Li created SENTRY-2151:
-

 Summary: Object Ownership
 Key: SENTRY-2151
 URL: https://issues.apache.org/jira/browse/SENTRY-2151
 Project: Sentry
  Issue Type: New Feature
  Components: Sentry
Affects Versions: 2.1.0
Reporter: Na Li
Assignee: Na Li
 Fix For: 2.1.0


admins want users who create tables to get implicit owner privileges during the 
table creation. These privileges cannot be revoked.

For instance, a user under role1 with CREATE privileges gets all privileges on 
newly created tables
{noformat}
# As an admin
hive> grant create on db1 to role1;

# As a user
user1> use db1;
user1> create table t1(id int);
-- An implicit 'grant all on db1.t1 to user user1' is generated in Sentry 
user1> insert into table t1 values (1);
user1> select * from t1;
user1> drop table t1;
{noformat}
For backward compatibility, the default implicit privilege to be applied must 
be determined by a configuration set by admins. This is to ensure that an 
upgrade to this new feature does not affect the behavior of old privileges set 
before the upgrade. For newly created tables, the privilege must be obtained 
from the property ‘owner.privileges’ of the database property where the table 
is created.

For instance, a user on db1 gets "all with grant privileges" but on db2 does 
not get any privilege
{noformat}
# As an admin
hive> alter database db1 set dbproperty('owner.privileges'='all with grant');
hive> grant create on db1 to role1;
hive> alter database db1 set dbproperty('owner.privileges='none');
hive> grant create on db2 to role2;

# As a user
user1> create table db1.t1(id int);
-- An implicit 'all with grant' privilege is granted to the user on db1.t1
user1> create table db2.t1(id int);
-- No privileges are granted to the user on db2.t1
{noformat}
The privilege granted implicitly cannot be revoked by explicit revoke commands 
nor if the 'owner.privileges' property changes. The only way is to remove the 
implicit privileges is by dropping the table or changing the owner of the table.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (SENTRY-2157) Update audit log to grant privilege to user

2018-03-06 Thread JIRA

[ 
https://issues.apache.org/jira/browse/SENTRY-2157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388680#comment-16388680
 ] 

Sergio Peña commented on SENTRY-2157:
-

Is the audit log going to display only grants? What about revoking those 
privileges?

> Update audit log to grant privilege to user
> ---
>
> Key: SENTRY-2157
> URL: https://issues.apache.org/jira/browse/SENTRY-2157
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Priority: Major
> Fix For: 2.1.0
>
>
> Update audit log to grant privileges to user



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (SENTRY-2155) Update JDO to grant privileges to user

2018-03-06 Thread JIRA

[ 
https://issues.apache.org/jira/browse/SENTRY-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388677#comment-16388677
 ] 

Sergio Peña commented on SENTRY-2155:
-

Can't we merge this Jira and SENTRY-2154? In my experience, doing a JDO change 
also requires doing schema changes in the same patch so that we can test it 
correctly.

> Update JDO to grant privileges to user
> --
>
> Key: SENTRY-2155
> URL: https://issues.apache.org/jira/browse/SENTRY-2155
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Priority: Major
> Fix For: 2.1.0
>
>
> Update JDO to match the update in DB schema in order to grant user to 
> privileges



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (SENTRY-2163) Many Instances of DefaultFS Logging Observed

2018-03-06 Thread BELUGA BEHR (JIRA)
BELUGA BEHR created SENTRY-2163:
---

 Summary: Many Instances of DefaultFS Logging Observed
 Key: SENTRY-2163
 URL: https://issues.apache.org/jira/browse/SENTRY-2163
 Project: Sentry
  Issue Type: Improvement
  Components: Hive Binding
Affects Versions: 1.8.0
Reporter: BELUGA BEHR


https://github.com/apache/sentry/blob/92a183f663c16fc8daf806dcb4a0b264dc811376/sentry-binding/sentry-binding-hive-conf/src/main/java/org/apache/sentry/binding/hive/conf/HiveAuthzConf.java#L180

Seeing many instances of the following in the HiveServer2 log files:

{code}
2018-02-26 07:04:57,318  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
2018-02-26 07:04:57,471  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
2018-02-26 07:04:57,488  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
2018-02-26 07:05:03,430  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
2018-02-26 07:05:03,447  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
2018-02-26 07:05:03,582  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
2018-02-26 07:05:03,599  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
2018-02-26 07:05:44,203  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
2018-02-26 07:05:44,221  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
2018-02-26 07:05:44,364  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
2018-02-26 07:05:44,381  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
2018-02-26 07:05:50,178  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
2018-02-26 07:05:50,200  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
2018-02-26 07:05:50,374  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
2018-02-26 07:05:50,391  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
2018-02-26 07:05:56,715  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
2018-02-26 07:05:56,768  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
2018-02-26 07:05:56,913  INFO  
org.apache.sentry.binding.hive.conf.HiveAuthzConf: [HiveServer2-Handler-Pool: 
Thread-65]: DefaultFS: hdfs://nameservice1
{code}

I'm not really sure why the same thread is loading so many instances of this 
class, but spamming this log message is not helpful.  We need to load fewer 
instances, lower logging to _debug_, remove the logging altogether, or cache 
the _hiveAuthzSiteURL_ and only log the message once per URL instance instead 
of once per object instantiation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (SENTRY-2154) Update schema to grant privileges to user

2018-03-06 Thread Na Li (JIRA)

[ 
https://issues.apache.org/jira/browse/SENTRY-2154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388724#comment-16388724
 ] 

Na Li commented on SENTRY-2154:
---

No. SENTRY-2162 contains retrieving implicit privileges and display them. 

I will add more details on the new table

> Update schema to grant privileges to user
> -
>
> Key: SENTRY-2154
> URL: https://issues.apache.org/jira/browse/SENTRY-2154
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Priority: Major
> Fix For: 2.1.0
>
>
> Need to add new DB table to support grant user to privileges
> Also, a flag should be added in privilege table to indicate the privilege is 
> created by user, or created by sentry implicitly. User can view the implicit 
> privileges, but cannot change it directly



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (SENTRY-2159) Add e2e tests for granting implicit owner privileges

2018-03-06 Thread Na Li (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Na Li updated SENTRY-2159:
--
Summary: Add e2e tests for granting implicit owner privileges  (was: Add 
e2e tests for granting implicit privileges to user )

> Add e2e tests for granting implicit owner privileges
> 
>
> Key: SENTRY-2159
> URL: https://issues.apache.org/jira/browse/SENTRY-2159
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Priority: Major
> Fix For: 2.1.0
>
>
> Need to add test cases in e2e for granting implicit owner privileges for 
> newly created table



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (SENTRY-2159) Add e2e tests for granting implicit privileges to user

2018-03-06 Thread Na Li (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Na Li updated SENTRY-2159:
--
Summary: Add e2e tests for granting implicit privileges to user   (was: Add 
e2e tests for granting privileges to user)

> Add e2e tests for granting implicit privileges to user 
> ---
>
> Key: SENTRY-2159
> URL: https://issues.apache.org/jira/browse/SENTRY-2159
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Priority: Major
> Fix For: 2.1.0
>
>
> Need to add test cases in e2e for granting privileges to user



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (SENTRY-2159) Add e2e tests for granting implicit privileges to user

2018-03-06 Thread Na Li (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-2159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Na Li updated SENTRY-2159:
--
Description: Need to add test cases in e2e for granting implicit owner 
privileges for newly created table  (was: Need to add test cases in e2e for 
granting privileges to user)

> Add e2e tests for granting implicit privileges to user 
> ---
>
> Key: SENTRY-2159
> URL: https://issues.apache.org/jira/browse/SENTRY-2159
> Project: Sentry
>  Issue Type: Sub-task
>  Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Na Li
>Priority: Major
> Fix For: 2.1.0
>
>
> Need to add test cases in e2e for granting implicit owner privileges for 
> newly created table



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (SENTRY-2164) Convert uses of TransactionBlock to lambdas

2018-03-06 Thread Alexander Kolbasov (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Kolbasov updated SENTRY-2164:
---
Status: Patch Available  (was: Open)

> Convert uses of TransactionBlock to lambdas
> ---
>
> Key: SENTRY-2164
> URL: https://issues.apache.org/jira/browse/SENTRY-2164
> Project: Sentry
>  Issue Type: Bug
>  Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
>Priority: Major
> Attachments: SENTRY-2164.01.patch
>
>
> Now that we agreed to use Java 8 features for Sentry it makes sense to clean 
> up SentryStore code and convert anonymous TransactionBlock instances to 
> lambdas.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (SENTRY-2164) Convert uses of TransactionBlock to lambdas

2018-03-06 Thread Alexander Kolbasov (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Kolbasov updated SENTRY-2164:
---
Attachment: SENTRY-2164.01.patch

> Convert uses of TransactionBlock to lambdas
> ---
>
> Key: SENTRY-2164
> URL: https://issues.apache.org/jira/browse/SENTRY-2164
> Project: Sentry
>  Issue Type: Bug
>  Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
>Priority: Major
> Attachments: SENTRY-2164.01.patch
>
>
> Now that we agreed to use Java 8 features for Sentry it makes sense to clean 
> up SentryStore code and convert anonymous TransactionBlock instances to 
> lambdas.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (SENTRY-2164) Convert uses of TransactionBlock to lambdas

2018-03-06 Thread Alexander Kolbasov (JIRA)
Alexander Kolbasov created SENTRY-2164:
--

 Summary: Convert uses of TransactionBlock to lambdas
 Key: SENTRY-2164
 URL: https://issues.apache.org/jira/browse/SENTRY-2164
 Project: Sentry
  Issue Type: Bug
  Components: Sentry
Affects Versions: 2.1.0
Reporter: Alexander Kolbasov
Assignee: Alexander Kolbasov


Now that we agreed to use Java 8 features for Sentry it makes sense to clean up 
SentryStore code and convert anonymous TransactionBlock instances to lambdas.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (SENTRY-2164) Convert uses of TransactionBlock to lambdas

2018-03-06 Thread Alexander Kolbasov (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-2164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Kolbasov updated SENTRY-2164:
---
Attachment: SENTRY-2164.02.patch

> Convert uses of TransactionBlock to lambdas
> ---
>
> Key: SENTRY-2164
> URL: https://issues.apache.org/jira/browse/SENTRY-2164
> Project: Sentry
>  Issue Type: Bug
>  Components: Sentry
>Affects Versions: 2.1.0
>Reporter: Alexander Kolbasov
>Assignee: Alexander Kolbasov
>Priority: Major
> Attachments: SENTRY-2164.01.patch, SENTRY-2164.02.patch
>
>
> Now that we agreed to use Java 8 features for Sentry it makes sense to clean 
> up SentryStore code and convert anonymous TransactionBlock instances to 
> lambdas.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (SENTRY-1242) Enable getting all privileges on a hive object

2018-03-06 Thread Na Li (JIRA)

[ 
https://issues.apache.org/jira/browse/SENTRY-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388112#comment-16388112
 ] 

Na Li commented on SENTRY-1242:
---

[~moist] it is likely someone committed a fix after you build the patch. You 
can fetch the latest code, rebase, and then generate another patch. That would 
fix the issue: "The patch does not appear to apply with p0, p1, or p2".

> Enable getting all privileges on a hive object
> --
>
> Key: SENTRY-1242
> URL: https://issues.apache.org/jira/browse/SENTRY-1242
> Project: Sentry
>  Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: Sravya Tirukkovalur
>Assignee: Steve Moist
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: SENTRY-1242-001.patch, SENTRY-1242-002.patch
>
>
> Enable show grant on table/db . This syntax is already supported by 
> hive.
> This would be really useful for the admin to find out all policies on a hive 
> object.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (SENTRY-2147) Fix Javadoc for SentryHiveAuthorizerFactory

2018-03-06 Thread Colm O hEigeartaigh (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-2147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Colm O hEigeartaigh updated SENTRY-2147:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Fix Javadoc for SentryHiveAuthorizerFactory
> ---
>
> Key: SENTRY-2147
> URL: https://issues.apache.org/jira/browse/SENTRY-2147
> Project: Sentry
>  Issue Type: Improvement
>Reporter: Colm O hEigeartaigh
>Assignee: Colm O hEigeartaigh
>Priority: Trivial
> Fix For: 2.1.0
>
> Attachments: SENTRY-2147.patch, SENTRY-2147.patch.1
>
>
> The Javadoc for SentryHiveAuthorizerFactory incorrectly states that it should 
> be configured as follows:
>  
>    hive.security.authorization.enable
>   
> org.apache.sentry.binding.hive.authz.SentryHiveAuthorizerFactory
> 
> Instead it should be "hive.security.authorization.manager".



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (SENTRY-1242) Enable getting all privileges on a hive object

2018-03-06 Thread Steve Moist (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Moist updated SENTRY-1242:

Attachment: (was: SENTRY-1242-002.diff)

> Enable getting all privileges on a hive object
> --
>
> Key: SENTRY-1242
> URL: https://issues.apache.org/jira/browse/SENTRY-1242
> Project: Sentry
>  Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: Sravya Tirukkovalur
>Assignee: Steve Moist
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: SENTRY-1242-001.patch, SENTRY-1242-002.patch
>
>
> Enable show grant on table/db . This syntax is already supported by 
> hive.
> This would be really useful for the admin to find out all policies on a hive 
> object.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (SENTRY-1242) Enable getting all privileges on a hive object

2018-03-06 Thread Steve Moist (JIRA)

 [ 
https://issues.apache.org/jira/browse/SENTRY-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Moist updated SENTRY-1242:

Attachment: SENTRY-1242-002.patch

> Enable getting all privileges on a hive object
> --
>
> Key: SENTRY-1242
> URL: https://issues.apache.org/jira/browse/SENTRY-1242
> Project: Sentry
>  Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: Sravya Tirukkovalur
>Assignee: Steve Moist
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: SENTRY-1242-001.patch, SENTRY-1242-002.patch
>
>
> Enable show grant on table/db . This syntax is already supported by 
> hive.
> This would be really useful for the admin to find out all policies on a hive 
> object.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (SENTRY-1242) Enable getting all privileges on a hive object

2018-03-06 Thread Steve Moist (JIRA)

[ 
https://issues.apache.org/jira/browse/SENTRY-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388001#comment-16388001
 ] 

Steve Moist commented on SENTRY-1242:
-

Sure, give me a bit to resetup my environment.  It looks like it didn't import 
my patch correctly, I've removed it and renamed the suffix to .patch.

> Enable getting all privileges on a hive object
> --
>
> Key: SENTRY-1242
> URL: https://issues.apache.org/jira/browse/SENTRY-1242
> Project: Sentry
>  Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: Sravya Tirukkovalur
>Assignee: Steve Moist
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: SENTRY-1242-001.patch, SENTRY-1242-002.patch
>
>
> Enable show grant on table/db . This syntax is already supported by 
> hive.
> This would be really useful for the admin to find out all policies on a hive 
> object.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (SENTRY-2150) Update Apache parent pom version

2018-03-06 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/SENTRY-2150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387921#comment-16387921
 ] 

Hadoop QA commented on SENTRY-2150:
---

Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12913198/SENTRY-2150.patch 
against master.

{color:green}Overall:{color} +1 all checks pass

{color:green}SUCCESS:{color} all tests passed

Console output: 
https://builds.apache.org/job/PreCommit-SENTRY-Build/3683/console

This message is automatically generated.

> Update Apache parent pom version
> 
>
> Key: SENTRY-2150
> URL: https://issues.apache.org/jira/browse/SENTRY-2150
> Project: Sentry
>  Issue Type: Improvement
>Reporter: Colm O hEigeartaigh
>Assignee: Colm O hEigeartaigh
>Priority: Trivial
> Fix For: 2.1.0
>
> Attachments: SENTRY-2150.patch
>
>
> We should update the Apache parent pom version - see in particular 
> https://issues.apache.org/jira/browse/MPOM-118



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (SENTRY-2140) Attribute based access control

2018-03-06 Thread Na Li (JIRA)

[ 
https://issues.apache.org/jira/browse/SENTRY-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388085#comment-16388085
 ] 

Na Li commented on SENTRY-2140:
---

[~moist] Thanks for the design documentation.

1) Can you add more specific details on how ABAC work with Role Based Access 
Control? In my opinion, it happens at "Enforcement point for attribute 
privileges in Sentry bindings for Hive and Impala"

2) "Means for user to specify attribute privileges for roles (and users?)" It 
seems you only use attribute on table column, Can we use attribute on user and 
session? For example, can we grant access on accessing table column with PII 
only during working hour and user country matches the value of the "Country" 
column?

3) How is the info from "Attribute Ingestion" used in "Enforcement point for 
attribute privileges"? An example that shows the whole work flow would be very 
helpful.

> Attribute based access control
> --
>
> Key: SENTRY-2140
> URL: https://issues.apache.org/jira/browse/SENTRY-2140
> Project: Sentry
>  Issue Type: New Feature
>  Components: Core
>Reporter: Steve Moist
>Priority: Major
> Attachments: Sentry ABAC Proposal.pdf
>
>
> As a user, I want to have finer grain control over which users/roles can view 
> data in Hive.  Some information such as Social Security Number is considered 
> very confidential information.  I want to be able to tag columns in Hive with 
> "attributes" that prevent users/roles from not accessing or seeing the data.  
> For users/roles that have that attribute, they should be able to see that 
> information.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (SENTRY-1242) Enable getting all privileges on a hive object

2018-03-06 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/SENTRY-1242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388056#comment-16388056
 ] 

Hadoop QA commented on SENTRY-1242:
---

Here are the results of testing the latest attachment
https://issues.apache.org/jira/secure/attachment/12913221/SENTRY-1242-002.patch 
against master.

{color:red}Overall:{color} -1 due to 2 errors

{color:red}ERROR:{color} mvn test exited 1
{color:red}ERROR:{color} Failed: 
org.apache.sentry.binding.hive.TestSentryHiveAuthorizationTaskFactory

Console output: 
https://builds.apache.org/job/PreCommit-SENTRY-Build/3684/console

This message is automatically generated.

> Enable getting all privileges on a hive object
> --
>
> Key: SENTRY-1242
> URL: https://issues.apache.org/jira/browse/SENTRY-1242
> Project: Sentry
>  Issue Type: New Feature
>Affects Versions: 2.0.0
>Reporter: Sravya Tirukkovalur
>Assignee: Steve Moist
>Priority: Major
> Fix For: 2.1.0
>
> Attachments: SENTRY-1242-001.patch, SENTRY-1242-002.patch
>
>
> Enable show grant on table/db . This syntax is already supported by 
> hive.
> This would be really useful for the admin to find out all policies on a hive 
> object.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Comment Edited] (SENTRY-2140) Attribute based access control

2018-03-06 Thread Na Li (JIRA)

[ 
https://issues.apache.org/jira/browse/SENTRY-2140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16388085#comment-16388085
 ] 

Na Li edited comment on SENTRY-2140 at 3/6/18 4:49 PM:
---

[~moist] Thanks for the design documentation.

1) Can you add more specific details on how ABAC work with Role Based Access 
Control? In my opinion, it happens at "Enforcement point for attribute 
privileges in Sentry bindings for Hive and Impala"

2) "Means for user to specify attribute privileges for roles (and users?)" It 
seems you only use attribute on table column, Can we use attribute on user and 
session? For example, can we grant access on accessing table column with PII 
only for user with clearance > 4, during working hour and user country matches 
the value of the "Country" column?

3) How is the info from "Attribute Ingestion" used in "Enforcement point for 
attribute privileges"? An example that shows the whole work flow would be very 
helpful.


was (Author: linaataustin):
[~moist] Thanks for the design documentation.

1) Can you add more specific details on how ABAC work with Role Based Access 
Control? In my opinion, it happens at "Enforcement point for attribute 
privileges in Sentry bindings for Hive and Impala"

2) "Means for user to specify attribute privileges for roles (and users?)" It 
seems you only use attribute on table column, Can we use attribute on user and 
session? For example, can we grant access on accessing table column with PII 
only during working hour and user country matches the value of the "Country" 
column?

3) How is the info from "Attribute Ingestion" used in "Enforcement point for 
attribute privileges"? An example that shows the whole work flow would be very 
helpful.

> Attribute based access control
> --
>
> Key: SENTRY-2140
> URL: https://issues.apache.org/jira/browse/SENTRY-2140
> Project: Sentry
>  Issue Type: New Feature
>  Components: Core
>Reporter: Steve Moist
>Priority: Major
> Attachments: Sentry ABAC Proposal.pdf
>
>
> As a user, I want to have finer grain control over which users/roles can view 
> data in Hive.  Some information such as Social Security Number is considered 
> very confidential information.  I want to be able to tag columns in Hive with 
> "attributes" that prevent users/roles from not accessing or seeing the data.  
> For users/roles that have that attribute, they should be able to see that 
> information.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (SENTRY-2149) Implement functionality to show groups

2018-03-06 Thread Sachin (JIRA)

[ 
https://issues.apache.org/jira/browse/SENTRY-2149?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387745#comment-16387745
 ] 

Sachin commented on SENTRY-2149:


Thanks for comments. Yes request is use to see all the groups from sentry. 
Sentry CLI mean Sentry shell?. I have checked with Sentry shell, it doesn't 
have the "SHOW GROUPS".  

Please correct me If I am wrong.

> Implement functionality to show groups
> --
>
> Key: SENTRY-2149
> URL: https://issues.apache.org/jira/browse/SENTRY-2149
> Project: Sentry
>  Issue Type: New Feature
>Reporter: Sachin
>Priority: Major
>
> Sentry allows to list the roles
>  SHOW ROLES;
>  There should be also a way to show the groups . Currently it seems that this 
> is only possible by directly querying the Sentry database. This functionality 
> should be provided out-of-the-box similar to the statement above.
> The functionality could look similar to the following statement
> {code:sql}
> SHOW GROUPS;{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)