Re: Review Request 69620: SENTRY-2483: Implement HMS PreReadEvent support in MetastoreAuthzBinding

2019-01-04 Thread Na Li via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69620/#review211680
---


Ship it!




Ship It!

- Na Li


On Dec. 21, 2018, 5:39 p.m., Sergio Pena wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69620/
> ---
> 
> (Updated Dec. 21, 2018, 5:39 p.m.)
> 
> 
> Review request for sentry and Na Li.
> 
> 
> Bugs: sentry-2483
> https://issues.apache.org/jira/browse/sentry-2483
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Add READ_DATABASE and READ_TABLE events support to provide read authorization 
> to HMS.
> 
> 
> Diffs
> -
> 
>   
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/MetastoreAuthzBindingBase.java
>  328d2b5c69451922e062cc3f04d37c5e7347d17f 
>   sentry-tests/sentry-tests-hive/pom.xml 
> 74777bbff590ea63c18492c77ae86042734d8e70 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithStaticConfiguration.java
>  8bf486e7d7d7a2e89278f1287115bf835513ef3f 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/hiveserver/HiveServerFactory.java
>  7d41348572f0c01001b6bfa03d5ffb780f5a5e75 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/metastore/TestMetastoreEndToEnd.java
>  f8f304fbb9926d98939ee0aa8c74f0abc8789fa9 
> 
> 
> Diff: https://reviews.apache.org/r/69620/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergio Pena
> 
>

Re: Review Request 69620: SENTRY-2483: Implement HMS PreReadEvent support in MetastoreAuthzBinding

2019-01-04 Thread Na Li via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69620/#review211679
---




sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithStaticConfiguration.java
Lines 284 (patched)


if enableAuthorizingObjectStore is false, should you set 
METASTORE_RAW_STORE_IMPL to be ObjectStore? The default value is 
"org.apache.hadoop.hive.metastore.ObjectStore". Setting it here would make it 
more obvious what implementation is used.


- Na Li


On Dec. 21, 2018, 5:39 p.m., Sergio Pena wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69620/
> ---
> 
> (Updated Dec. 21, 2018, 5:39 p.m.)
> 
> 
> Review request for sentry and Na Li.
> 
> 
> Bugs: sentry-2483
> https://issues.apache.org/jira/browse/sentry-2483
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Add READ_DATABASE and READ_TABLE events support to provide read authorization 
> to HMS.
> 
> 
> Diffs
> -
> 
>   
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/MetastoreAuthzBindingBase.java
>  328d2b5c69451922e062cc3f04d37c5e7347d17f 
>   sentry-tests/sentry-tests-hive/pom.xml 
> 74777bbff590ea63c18492c77ae86042734d8e70 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithStaticConfiguration.java
>  8bf486e7d7d7a2e89278f1287115bf835513ef3f 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/hiveserver/HiveServerFactory.java
>  7d41348572f0c01001b6bfa03d5ffb780f5a5e75 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/metastore/TestMetastoreEndToEnd.java
>  f8f304fbb9926d98939ee0aa8c74f0abc8789fa9 
> 
> 
> Diff: https://reviews.apache.org/r/69620/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergio Pena
> 
>

Re: Review Request 69620: SENTRY-2483: Implement HMS PreReadEvent support in MetastoreAuthzBinding

2019-01-04 Thread Arjun Mishra via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69620/#review211674
---


Ship it!




Fix failed tests

- Arjun Mishra


On Dec. 21, 2018, 5:39 p.m., Sergio Pena wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69620/
> ---
> 
> (Updated Dec. 21, 2018, 5:39 p.m.)
> 
> 
> Review request for sentry and Na Li.
> 
> 
> Bugs: sentry-2483
> https://issues.apache.org/jira/browse/sentry-2483
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Add READ_DATABASE and READ_TABLE events support to provide read authorization 
> to HMS.
> 
> 
> Diffs
> -
> 
>   
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/MetastoreAuthzBindingBase.java
>  328d2b5c69451922e062cc3f04d37c5e7347d17f 
>   sentry-tests/sentry-tests-hive/pom.xml 
> 74777bbff590ea63c18492c77ae86042734d8e70 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithStaticConfiguration.java
>  8bf486e7d7d7a2e89278f1287115bf835513ef3f 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/hiveserver/HiveServerFactory.java
>  7d41348572f0c01001b6bfa03d5ffb780f5a5e75 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/metastore/TestMetastoreEndToEnd.java
>  f8f304fbb9926d98939ee0aa8c74f0abc8789fa9 
> 
> 
> Diff: https://reviews.apache.org/r/69620/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergio Pena
> 
>

Re: Review Request 69620: SENTRY-2483: Implement HMS PreReadEvent support in MetastoreAuthzBinding

2019-01-03 Thread Arjun Mishra via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69620/#review211665
---



Looks good. Please fix the test cases. Would like to review the next patch as 
well.

- Arjun Mishra


On Dec. 21, 2018, 5:39 p.m., Sergio Pena wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69620/
> ---
> 
> (Updated Dec. 21, 2018, 5:39 p.m.)
> 
> 
> Review request for sentry and Na Li.
> 
> 
> Bugs: sentry-2483
> https://issues.apache.org/jira/browse/sentry-2483
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Add READ_DATABASE and READ_TABLE events support to provide read authorization 
> to HMS.
> 
> 
> Diffs
> -
> 
>   
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/MetastoreAuthzBindingBase.java
>  328d2b5c69451922e062cc3f04d37c5e7347d17f 
>   sentry-tests/sentry-tests-hive/pom.xml 
> 74777bbff590ea63c18492c77ae86042734d8e70 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithStaticConfiguration.java
>  8bf486e7d7d7a2e89278f1287115bf835513ef3f 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/hiveserver/HiveServerFactory.java
>  7d41348572f0c01001b6bfa03d5ffb780f5a5e75 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/metastore/TestMetastoreEndToEnd.java
>  f8f304fbb9926d98939ee0aa8c74f0abc8789fa9 
> 
> 
> Diff: https://reviews.apache.org/r/69620/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergio Pena
> 
>

Re: Review Request 69620: SENTRY-2483: Implement HMS PreReadEvent support in MetastoreAuthzBinding

2019-01-03 Thread Sergio Pena via Review Board 


> On Dec. 31, 2018, 4:48 a.m., Na Li wrote:
> > sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithStaticConfiguration.java
> > Lines 284 (patched)
> > 
> >
> > I don't think we should use 
> > "org.apache.sentry.binding.metastore.AuthorizingObjectStore" in testing 
> > because in production, the property should be set to  
> > "org.apache.hadoop.hive.metastore.ObjectStore", which is implemented by HMS.
> > 
> > SENTRY-355 "Support metadata read privilege enforcement for Metastore 
> > pluging" replaces the implementation of ObjectStore, but we are not going 
> > to use this approach.
> > 
> > The approach we decide to take is for HMS server to call Preeventlisten 
> > for authorization and filter hook to remove items that user does not have 
> > access. In HMS server, Sentry implementation of the hook (refered as 
> > , and should be SentryMetaStoreFilterHook ) is configured 
> > in following way.  
> > MetastoreConf.setClass(conf, ConfVars.FILTER_HOOK, 
> > .class,
> > MetaStoreFilterHook.class);
> > 
> > Therefore, in e2e test, we should configure HMS server to use filter 
> > hook and keep the value of HiveConf.ConfVars.METASTORE_RAW_STORE_IMPL to be 
> > default, which is "org.apache.hadoop.hive.metastore.ObjectStore". In this 
> > way, we can test the real sentry-hive integration. 
> > On the other hand, you need fix of HIVE-20776 in order to make the test 
> > work.

TestMetastoreEndToEnd.java sets the enableAuthorizingObjectStore to false to 
avoid using it on the tests. I cannot remove it unless I do more work on 
removing the AuthorizingObjectStore, but we don't know who's using it.

This patch is just another way to protect HMS.


- Sergio


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69620/#review211584
---


On Dec. 21, 2018, 5:39 p.m., Sergio Pena wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69620/
> ---
> 
> (Updated Dec. 21, 2018, 5:39 p.m.)
> 
> 
> Review request for sentry and Na Li.
> 
> 
> Bugs: sentry-2483
> https://issues.apache.org/jira/browse/sentry-2483
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Add READ_DATABASE and READ_TABLE events support to provide read authorization 
> to HMS.
> 
> 
> Diffs
> -
> 
>   
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/MetastoreAuthzBindingBase.java
>  328d2b5c69451922e062cc3f04d37c5e7347d17f 
>   sentry-tests/sentry-tests-hive/pom.xml 
> 74777bbff590ea63c18492c77ae86042734d8e70 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithStaticConfiguration.java
>  8bf486e7d7d7a2e89278f1287115bf835513ef3f 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/hiveserver/HiveServerFactory.java
>  7d41348572f0c01001b6bfa03d5ffb780f5a5e75 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/metastore/TestMetastoreEndToEnd.java
>  f8f304fbb9926d98939ee0aa8c74f0abc8789fa9 
> 
> 
> Diff: https://reviews.apache.org/r/69620/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergio Pena
> 
>

Re: Review Request 69620: SENTRY-2483: Implement HMS PreReadEvent support in MetastoreAuthzBinding

2018-12-30 Thread Na Li via Review Board 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69620/#review211585
---



- Na Li


On Dec. 21, 2018, 5:39 p.m., Sergio Pena wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69620/
> ---
> 
> (Updated Dec. 21, 2018, 5:39 p.m.)
> 
> 
> Review request for sentry and Na Li.
> 
> 
> Bugs: sentry-2483
> https://issues.apache.org/jira/browse/sentry-2483
> 
> 
> Repository: sentry
> 
> 
> Description
> ---
> 
> Add READ_DATABASE and READ_TABLE events support to provide read authorization 
> to HMS.
> 
> 
> Diffs
> -
> 
>   
> sentry-binding/sentry-binding-hive/src/main/java/org/apache/sentry/binding/metastore/MetastoreAuthzBindingBase.java
>  328d2b5c69451922e062cc3f04d37c5e7347d17f 
>   sentry-tests/sentry-tests-hive/pom.xml 
> 74777bbff590ea63c18492c77ae86042734d8e70 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/AbstractTestWithStaticConfiguration.java
>  8bf486e7d7d7a2e89278f1287115bf835513ef3f 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/hive/hiveserver/HiveServerFactory.java
>  7d41348572f0c01001b6bfa03d5ffb780f5a5e75 
>   
> sentry-tests/sentry-tests-hive/src/test/java/org/apache/sentry/tests/e2e/metastore/TestMetastoreEndToEnd.java
>  f8f304fbb9926d98939ee0aa8c74f0abc8789fa9 
> 
> 
> Diff: https://reviews.apache.org/r/69620/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sergio Pena
> 
>