Re: Review Request 68428: RANGER-1935/RANGER-1738 - Upgrade Ranger to support Apache Hadoop 3.0.0

2018-09-06 Thread Ramesh Mani 


> On Sept. 5, 2018, 11:09 p.m., Ramesh Mani wrote:
> > security-admin/pom.xml
> > Line 263 (original), 263 (patched)
> > 
> >
> > why using hadoop.2.7 version for ranger secuirty-admin?
> 
> Colm O hEigeartaigh wrote:
> As I said in the description above, if the Admin service uses the new 
> Hadoop jars, when you click on the users part of the Admin UI, no users 
> appear. I'm not expert enough with the UI to figure out why, so I left the 
> admin service using the old hadoop jars in the patch, and planned to create a 
> separate JIRA to upgrade the admin service afterwards.

Thanks Colm, I was not sure of what was the issue from the description


- Ramesh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68428/#review208381
---


On Sept. 5, 2018, 1:22 p.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68428/
> ---
> 
> (Updated Sept. 5, 2018, 1:22 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1935
> https://issues.apache.org/jira/browse/RANGER-1935
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Upgrade Ranger to use Apache Hadoop 3.0.0. Some comments:
> 
> - KMS reencryptEncryptedKey not implemented.
> - HDFS unit test - HDFSContentSummaryTest test failing. @Ignore'd for now, 
> I'll create a separate JIRA for this.
> - Hive tests commented out - waiting for Hive upgrade task.
> - Admin service users not appearing so in the patch it's using the old Hadoop 
> jars. I'll create a separate JIRA for this to upgrade it as well.
> 
> 
> Diffs
> -
> 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/utils/InMemoryJAASConfiguration.java
>  0f291389f 
>   credentialbuilder/pom.xml 461dcd0b6 
>   
> hdfs-agent/src/test/java/org/apache/ranger/services/hdfs/HDFSRangerTest.java 
> a7215ce92 
>   
> hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
>  7dc5c54b0 
>   kms/pom.xml 6fc190f35 
>   
> kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/EagerKeyGeneratorKeyProviderCryptoExtension.java
>  f853a844b 
>   kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMS.java 
> be3700ffa 
>   
> kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSAuthenticationFilter.java
>  ada9a56f4 
>   kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java 
> b2540cb42 
>   knox-agent/pom.xml 1d8fdae62 
>   
> plugin-yarn/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
>  a4de016ea 
>   pom.xml ae3f4be4c 
>   ranger-examples/plugin-sampleapp/pom.xml bef0f5c1f 
>   ranger-storm-plugin-shim/pom.xml 89efe34de 
>   
> ranger-yarn-plugin-shim/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
>  7b7f1f303 
>   security-admin/pom.xml 84bbd6650 
>   src/main/assembly/hbase-agent.xml 3ebc334d1 
>   src/main/assembly/hdfs-agent.xml 561d13737 
>   src/main/assembly/hive-agent.xml 03bd31a55 
>   src/main/assembly/kms.xml 7fbc1410c 
>   src/main/assembly/knox-agent.xml 8357d4989 
>   src/main/assembly/plugin-atlas.xml 8f4a64cb6 
>   src/main/assembly/plugin-kafka.xml 97ff8ad1b 
>   src/main/assembly/plugin-kms.xml 6d15f2a10 
>   src/main/assembly/plugin-kylin.xml f4e082007 
>   src/main/assembly/plugin-solr.xml de30bfb13 
>   src/main/assembly/plugin-sqoop.xml d2bd69ab3 
>   src/main/assembly/plugin-yarn.xml c6a48e888 
>   src/main/assembly/ranger-tools.xml 249de9a2d 
>   src/main/assembly/storm-agent.xml fdaf67863 
>   src/main/assembly/tagsync.xml d1b83df42 
>   src/main/assembly/usersync.xml d170d8c1c 
>   storm-agent/pom.xml a084e6891 
>   tagsync/pom.xml e98cf63dc 
>   ugsync/ldapconfigchecktool/ldapconfigcheck/pom.xml d49001a65 
>   
> ugsync/ldapconfigchecktool/ldapconfigcheck/src/main/java/org/apache/ranger/ldapconfigcheck/LdapConfig.java
>  9a77cedf7 
>   ugsync/pom.xml f4c24e739 
>   unixauthclient/pom.xml cd3d15604 
>   unixauthservice/pom.xml 2c0ed2eef 
> 
> 
> Diff: https://reviews.apache.org/r/68428/diff/3/
> 
> 
> Testing
> ---
> 
> Tested plugins: HDFS, Yarn, Storm (uses Hadoop 2.7.1), Knox, HBase, KMS, 
> Sqoop, Kafka, Solr, usersync. Test admin service works OK.
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

Re: Review Request 68428: RANGER-1935/RANGER-1738 - Upgrade Ranger to support Apache Hadoop 3.0.0

2018-09-06 Thread Colm O hEigeartaigh 


> On Sept. 5, 2018, 11:09 p.m., Ramesh Mani wrote:
> > security-admin/pom.xml
> > Line 263 (original), 263 (patched)
> > 
> >
> > why using hadoop.2.7 version for ranger secuirty-admin?

As I said in the description above, if the Admin service uses the new Hadoop 
jars, when you click on the users part of the Admin UI, no users appear. I'm 
not expert enough with the UI to figure out why, so I left the admin service 
using the old hadoop jars in the patch, and planned to create a separate JIRA 
to upgrade the admin service afterwards.


- Colm


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68428/#review208381
---


On Sept. 5, 2018, 1:22 p.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68428/
> ---
> 
> (Updated Sept. 5, 2018, 1:22 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1935
> https://issues.apache.org/jira/browse/RANGER-1935
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Upgrade Ranger to use Apache Hadoop 3.0.0. Some comments:
> 
> - KMS reencryptEncryptedKey not implemented.
> - HDFS unit test - HDFSContentSummaryTest test failing. @Ignore'd for now, 
> I'll create a separate JIRA for this.
> - Hive tests commented out - waiting for Hive upgrade task.
> - Admin service users not appearing so in the patch it's using the old Hadoop 
> jars. I'll create a separate JIRA for this to upgrade it as well.
> 
> 
> Diffs
> -
> 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/utils/InMemoryJAASConfiguration.java
>  0f291389f 
>   credentialbuilder/pom.xml 461dcd0b6 
>   
> hdfs-agent/src/test/java/org/apache/ranger/services/hdfs/HDFSRangerTest.java 
> a7215ce92 
>   
> hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
>  7dc5c54b0 
>   kms/pom.xml 6fc190f35 
>   
> kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/EagerKeyGeneratorKeyProviderCryptoExtension.java
>  f853a844b 
>   kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMS.java 
> be3700ffa 
>   
> kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSAuthenticationFilter.java
>  ada9a56f4 
>   kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java 
> b2540cb42 
>   knox-agent/pom.xml 1d8fdae62 
>   
> plugin-yarn/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
>  a4de016ea 
>   pom.xml ae3f4be4c 
>   ranger-examples/plugin-sampleapp/pom.xml bef0f5c1f 
>   ranger-storm-plugin-shim/pom.xml 89efe34de 
>   
> ranger-yarn-plugin-shim/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
>  7b7f1f303 
>   security-admin/pom.xml 84bbd6650 
>   src/main/assembly/hbase-agent.xml 3ebc334d1 
>   src/main/assembly/hdfs-agent.xml 561d13737 
>   src/main/assembly/hive-agent.xml 03bd31a55 
>   src/main/assembly/kms.xml 7fbc1410c 
>   src/main/assembly/knox-agent.xml 8357d4989 
>   src/main/assembly/plugin-atlas.xml 8f4a64cb6 
>   src/main/assembly/plugin-kafka.xml 97ff8ad1b 
>   src/main/assembly/plugin-kms.xml 6d15f2a10 
>   src/main/assembly/plugin-kylin.xml f4e082007 
>   src/main/assembly/plugin-solr.xml de30bfb13 
>   src/main/assembly/plugin-sqoop.xml d2bd69ab3 
>   src/main/assembly/plugin-yarn.xml c6a48e888 
>   src/main/assembly/ranger-tools.xml 249de9a2d 
>   src/main/assembly/storm-agent.xml fdaf67863 
>   src/main/assembly/tagsync.xml d1b83df42 
>   src/main/assembly/usersync.xml d170d8c1c 
>   storm-agent/pom.xml a084e6891 
>   tagsync/pom.xml e98cf63dc 
>   ugsync/ldapconfigchecktool/ldapconfigcheck/pom.xml d49001a65 
>   
> ugsync/ldapconfigchecktool/ldapconfigcheck/src/main/java/org/apache/ranger/ldapconfigcheck/LdapConfig.java
>  9a77cedf7 
>   ugsync/pom.xml f4c24e739 
>   unixauthclient/pom.xml cd3d15604 
>   unixauthservice/pom.xml 2c0ed2eef 
> 
> 
> Diff: https://reviews.apache.org/r/68428/diff/3/
> 
> 
> Testing
> ---
> 
> Tested plugins: HDFS, Yarn, Storm (uses Hadoop 2.7.1), Knox, HBase, KMS, 
> Sqoop, Kafka, Solr, usersync. Test admin service works OK.
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

Re: Review Request 68428: RANGER-1935/RANGER-1738 - Upgrade Ranger to support Apache Hadoop 3.0.0

2018-09-05 Thread Ramesh Mani 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68428/#review208381
---




security-admin/pom.xml
Line 263 (original), 263 (patched)


why using hadoop.2.7 version for ranger secuirty-admin?


- Ramesh Mani


On Sept. 5, 2018, 1:22 p.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68428/
> ---
> 
> (Updated Sept. 5, 2018, 1:22 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1935
> https://issues.apache.org/jira/browse/RANGER-1935
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Upgrade Ranger to use Apache Hadoop 3.0.0. Some comments:
> 
> - KMS reencryptEncryptedKey not implemented.
> - HDFS unit test - HDFSContentSummaryTest test failing. @Ignore'd for now, 
> I'll create a separate JIRA for this.
> - Hive tests commented out - waiting for Hive upgrade task.
> - Admin service users not appearing so in the patch it's using the old Hadoop 
> jars. I'll create a separate JIRA for this to upgrade it as well.
> 
> 
> Diffs
> -
> 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/utils/InMemoryJAASConfiguration.java
>  0f291389f 
>   credentialbuilder/pom.xml 461dcd0b6 
>   
> hdfs-agent/src/test/java/org/apache/ranger/services/hdfs/HDFSRangerTest.java 
> a7215ce92 
>   
> hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
>  7dc5c54b0 
>   kms/pom.xml 6fc190f35 
>   
> kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/EagerKeyGeneratorKeyProviderCryptoExtension.java
>  f853a844b 
>   kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMS.java 
> be3700ffa 
>   
> kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSAuthenticationFilter.java
>  ada9a56f4 
>   kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java 
> b2540cb42 
>   knox-agent/pom.xml 1d8fdae62 
>   
> plugin-yarn/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
>  a4de016ea 
>   pom.xml ae3f4be4c 
>   ranger-examples/plugin-sampleapp/pom.xml bef0f5c1f 
>   ranger-storm-plugin-shim/pom.xml 89efe34de 
>   
> ranger-yarn-plugin-shim/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
>  7b7f1f303 
>   security-admin/pom.xml 84bbd6650 
>   src/main/assembly/hbase-agent.xml 3ebc334d1 
>   src/main/assembly/hdfs-agent.xml 561d13737 
>   src/main/assembly/hive-agent.xml 03bd31a55 
>   src/main/assembly/kms.xml 7fbc1410c 
>   src/main/assembly/knox-agent.xml 8357d4989 
>   src/main/assembly/plugin-atlas.xml 8f4a64cb6 
>   src/main/assembly/plugin-kafka.xml 97ff8ad1b 
>   src/main/assembly/plugin-kms.xml 6d15f2a10 
>   src/main/assembly/plugin-kylin.xml f4e082007 
>   src/main/assembly/plugin-solr.xml de30bfb13 
>   src/main/assembly/plugin-sqoop.xml d2bd69ab3 
>   src/main/assembly/plugin-yarn.xml c6a48e888 
>   src/main/assembly/ranger-tools.xml 249de9a2d 
>   src/main/assembly/storm-agent.xml fdaf67863 
>   src/main/assembly/tagsync.xml d1b83df42 
>   src/main/assembly/usersync.xml d170d8c1c 
>   storm-agent/pom.xml a084e6891 
>   tagsync/pom.xml e98cf63dc 
>   ugsync/ldapconfigchecktool/ldapconfigcheck/pom.xml d49001a65 
>   
> ugsync/ldapconfigchecktool/ldapconfigcheck/src/main/java/org/apache/ranger/ldapconfigcheck/LdapConfig.java
>  9a77cedf7 
>   ugsync/pom.xml f4c24e739 
>   unixauthclient/pom.xml cd3d15604 
>   unixauthservice/pom.xml 2c0ed2eef 
> 
> 
> Diff: https://reviews.apache.org/r/68428/diff/3/
> 
> 
> Testing
> ---
> 
> Tested plugins: HDFS, Yarn, Storm (uses Hadoop 2.7.1), Knox, HBase, KMS, 
> Sqoop, Kafka, Solr, usersync. Test admin service works OK.
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

Re: Review Request 68428: RANGER-1935/RANGER-1738 - Upgrade Ranger to support Apache Hadoop 3.0.0

2018-09-05 Thread Colm O hEigeartaigh 


> On Sept. 5, 2018, 11:53 a.m., Mehul Parikh wrote:
> > @colm : your patch is not getting applied on latest master branch. Can you 
> > please rebase your patch?

OK please try again...


- Colm


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68428/#review208347
---


On Sept. 5, 2018, 1:22 p.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68428/
> ---
> 
> (Updated Sept. 5, 2018, 1:22 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1935
> https://issues.apache.org/jira/browse/RANGER-1935
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Upgrade Ranger to use Apache Hadoop 3.0.0. Some comments:
> 
> - KMS reencryptEncryptedKey not implemented.
> - HDFS unit test - HDFSContentSummaryTest test failing. @Ignore'd for now, 
> I'll create a separate JIRA for this.
> - Hive tests commented out - waiting for Hive upgrade task.
> - Admin service users not appearing so in the patch it's using the old Hadoop 
> jars. I'll create a separate JIRA for this to upgrade it as well.
> 
> 
> Diffs
> -
> 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/utils/InMemoryJAASConfiguration.java
>  0f291389f 
>   credentialbuilder/pom.xml 461dcd0b6 
>   
> hdfs-agent/src/test/java/org/apache/ranger/services/hdfs/HDFSRangerTest.java 
> a7215ce92 
>   
> hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
>  7dc5c54b0 
>   kms/pom.xml 6fc190f35 
>   
> kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/EagerKeyGeneratorKeyProviderCryptoExtension.java
>  f853a844b 
>   kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMS.java 
> be3700ffa 
>   
> kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSAuthenticationFilter.java
>  ada9a56f4 
>   kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java 
> b2540cb42 
>   knox-agent/pom.xml 1d8fdae62 
>   
> plugin-yarn/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
>  a4de016ea 
>   pom.xml ae3f4be4c 
>   ranger-examples/plugin-sampleapp/pom.xml bef0f5c1f 
>   ranger-storm-plugin-shim/pom.xml 89efe34de 
>   
> ranger-yarn-plugin-shim/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
>  7b7f1f303 
>   security-admin/pom.xml 84bbd6650 
>   src/main/assembly/hbase-agent.xml 3ebc334d1 
>   src/main/assembly/hdfs-agent.xml 561d13737 
>   src/main/assembly/hive-agent.xml 03bd31a55 
>   src/main/assembly/kms.xml 7fbc1410c 
>   src/main/assembly/knox-agent.xml 8357d4989 
>   src/main/assembly/plugin-atlas.xml 8f4a64cb6 
>   src/main/assembly/plugin-kafka.xml 97ff8ad1b 
>   src/main/assembly/plugin-kms.xml 6d15f2a10 
>   src/main/assembly/plugin-kylin.xml f4e082007 
>   src/main/assembly/plugin-solr.xml de30bfb13 
>   src/main/assembly/plugin-sqoop.xml d2bd69ab3 
>   src/main/assembly/plugin-yarn.xml c6a48e888 
>   src/main/assembly/ranger-tools.xml 249de9a2d 
>   src/main/assembly/storm-agent.xml fdaf67863 
>   src/main/assembly/tagsync.xml d1b83df42 
>   src/main/assembly/usersync.xml d170d8c1c 
>   storm-agent/pom.xml a084e6891 
>   tagsync/pom.xml e98cf63dc 
>   ugsync/ldapconfigchecktool/ldapconfigcheck/pom.xml d49001a65 
>   
> ugsync/ldapconfigchecktool/ldapconfigcheck/src/main/java/org/apache/ranger/ldapconfigcheck/LdapConfig.java
>  9a77cedf7 
>   ugsync/pom.xml f4c24e739 
>   unixauthclient/pom.xml cd3d15604 
>   unixauthservice/pom.xml 2c0ed2eef 
> 
> 
> Diff: https://reviews.apache.org/r/68428/diff/3/
> 
> 
> Testing
> ---
> 
> Tested plugins: HDFS, Yarn, Storm (uses Hadoop 2.7.1), Knox, HBase, KMS, 
> Sqoop, Kafka, Solr, usersync. Test admin service works OK.
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

Re: Review Request 68428: RANGER-1935/RANGER-1738 - Upgrade Ranger to support Apache Hadoop 3.0.0

2018-09-05 Thread Colm O hEigeartaigh 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68428/
---

(Updated Sept. 5, 2018, 1:22 p.m.)


Review request for ranger.


Bugs: RANGER-1935
https://issues.apache.org/jira/browse/RANGER-1935


Repository: ranger


Description
---

Upgrade Ranger to use Apache Hadoop 3.0.0. Some comments:

- KMS reencryptEncryptedKey not implemented.
- HDFS unit test - HDFSContentSummaryTest test failing. @Ignore'd for now, I'll 
create a separate JIRA for this.
- Hive tests commented out - waiting for Hive upgrade task.
- Admin service users not appearing so in the patch it's using the old Hadoop 
jars. I'll create a separate JIRA for this to upgrade it as well.


Diffs (updated)
-

  
agents-audit/src/main/java/org/apache/ranger/audit/utils/InMemoryJAASConfiguration.java
 0f291389f 
  credentialbuilder/pom.xml 461dcd0b6 
  hdfs-agent/src/test/java/org/apache/ranger/services/hdfs/HDFSRangerTest.java 
a7215ce92 
  
hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
 7dc5c54b0 
  kms/pom.xml 6fc190f35 
  
kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/EagerKeyGeneratorKeyProviderCryptoExtension.java
 f853a844b 
  kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMS.java be3700ffa 
  
kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSAuthenticationFilter.java
 ada9a56f4 
  kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java 
b2540cb42 
  knox-agent/pom.xml 1d8fdae62 
  
plugin-yarn/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
 a4de016ea 
  pom.xml ae3f4be4c 
  ranger-examples/plugin-sampleapp/pom.xml bef0f5c1f 
  ranger-storm-plugin-shim/pom.xml 89efe34de 
  
ranger-yarn-plugin-shim/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
 7b7f1f303 
  security-admin/pom.xml 84bbd6650 
  src/main/assembly/hbase-agent.xml 3ebc334d1 
  src/main/assembly/hdfs-agent.xml 561d13737 
  src/main/assembly/hive-agent.xml 03bd31a55 
  src/main/assembly/kms.xml 7fbc1410c 
  src/main/assembly/knox-agent.xml 8357d4989 
  src/main/assembly/plugin-atlas.xml 8f4a64cb6 
  src/main/assembly/plugin-kafka.xml 97ff8ad1b 
  src/main/assembly/plugin-kms.xml 6d15f2a10 
  src/main/assembly/plugin-kylin.xml f4e082007 
  src/main/assembly/plugin-solr.xml de30bfb13 
  src/main/assembly/plugin-sqoop.xml d2bd69ab3 
  src/main/assembly/plugin-yarn.xml c6a48e888 
  src/main/assembly/ranger-tools.xml 249de9a2d 
  src/main/assembly/storm-agent.xml fdaf67863 
  src/main/assembly/tagsync.xml d1b83df42 
  src/main/assembly/usersync.xml d170d8c1c 
  storm-agent/pom.xml a084e6891 
  tagsync/pom.xml e98cf63dc 
  ugsync/ldapconfigchecktool/ldapconfigcheck/pom.xml d49001a65 
  
ugsync/ldapconfigchecktool/ldapconfigcheck/src/main/java/org/apache/ranger/ldapconfigcheck/LdapConfig.java
 9a77cedf7 
  ugsync/pom.xml f4c24e739 
  unixauthclient/pom.xml cd3d15604 
  unixauthservice/pom.xml 2c0ed2eef 


Diff: https://reviews.apache.org/r/68428/diff/3/

Changes: https://reviews.apache.org/r/68428/diff/2-3/


Testing
---

Tested plugins: HDFS, Yarn, Storm (uses Hadoop 2.7.1), Knox, HBase, KMS, Sqoop, 
Kafka, Solr, usersync. Test admin service works OK.


Thanks,

Colm O hEigeartaigh

Re: Review Request 68428: RANGER-1935/RANGER-1738 - Upgrade Ranger to support Apache Hadoop 3.0.0

2018-09-05 Thread Mehul Parikh 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68428/#review208347
---



@colm : your patch is not getting applied on latest master branch. Can you 
please rebase your patch?

- Mehul Parikh


On Sept. 5, 2018, 8:56 a.m., Colm O hEigeartaigh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/68428/
> ---
> 
> (Updated Sept. 5, 2018, 8:56 a.m.)
> 
> 
> Review request for ranger.
> 
> 
> Bugs: RANGER-1935
> https://issues.apache.org/jira/browse/RANGER-1935
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Upgrade Ranger to use Apache Hadoop 3.0.0. Some comments:
> 
> - KMS reencryptEncryptedKey not implemented.
> - HDFS unit test - HDFSContentSummaryTest test failing. @Ignore'd for now, 
> I'll create a separate JIRA for this.
> - Hive tests commented out - waiting for Hive upgrade task.
> - Admin service users not appearing so in the patch it's using the old Hadoop 
> jars. I'll create a separate JIRA for this to upgrade it as well.
> 
> 
> Diffs
> -
> 
>   
> agents-audit/src/main/java/org/apache/ranger/audit/utils/InMemoryJAASConfiguration.java
>  0f291389f 
>   credentialbuilder/pom.xml 461dcd0b6 
>   
> hdfs-agent/src/test/java/org/apache/ranger/services/hdfs/HDFSRangerTest.java 
> a7215ce92 
>   
> hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
>  7dc5c54b0 
>   kms/pom.xml 6fc190f35 
>   
> kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/EagerKeyGeneratorKeyProviderCryptoExtension.java
>  f853a844b 
>   kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMS.java 
> be3700ffa 
>   
> kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSAuthenticationFilter.java
>  ada9a56f4 
>   kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java 
> b2540cb42 
>   knox-agent/pom.xml 1d8fdae62 
>   
> plugin-yarn/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
>  a4de016ea 
>   pom.xml 8d7ea131e 
>   ranger-examples/plugin-sampleapp/pom.xml bef0f5c1f 
>   ranger-storm-plugin-shim/pom.xml 89efe34de 
>   
> ranger-yarn-plugin-shim/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
>  7b7f1f303 
>   security-admin/pom.xml 84bbd6650 
>   src/main/assembly/hbase-agent.xml 3ebc334d1 
>   src/main/assembly/hdfs-agent.xml 561d13737 
>   src/main/assembly/hive-agent.xml 03bd31a55 
>   src/main/assembly/kms.xml 7fbc1410c 
>   src/main/assembly/knox-agent.xml 8357d4989 
>   src/main/assembly/plugin-atlas.xml 8f4a64cb6 
>   src/main/assembly/plugin-kafka.xml 97ff8ad1b 
>   src/main/assembly/plugin-kms.xml 6d15f2a10 
>   src/main/assembly/plugin-kylin.xml f4e082007 
>   src/main/assembly/plugin-solr.xml de30bfb13 
>   src/main/assembly/plugin-sqoop.xml d2bd69ab3 
>   src/main/assembly/plugin-yarn.xml c6a48e888 
>   src/main/assembly/ranger-tools.xml 249de9a2d 
>   src/main/assembly/storm-agent.xml fdaf67863 
>   src/main/assembly/tagsync.xml bc6e28b94 
>   src/main/assembly/usersync.xml a6bad1dec 
>   storm-agent/pom.xml a084e6891 
>   tagsync/pom.xml e98cf63dc 
>   ugsync/ldapconfigchecktool/ldapconfigcheck/pom.xml d49001a65 
>   
> ugsync/ldapconfigchecktool/ldapconfigcheck/src/main/java/org/apache/ranger/ldapconfigcheck/LdapConfig.java
>  9a77cedf7 
>   ugsync/pom.xml f4c24e739 
>   unixauthclient/pom.xml cd3d15604 
>   unixauthservice/pom.xml 2c0ed2eef 
> 
> 
> Diff: https://reviews.apache.org/r/68428/diff/2/
> 
> 
> Testing
> ---
> 
> Tested plugins: HDFS, Yarn, Storm (uses Hadoop 2.7.1), Knox, HBase, KMS, 
> Sqoop, Kafka, Solr, usersync. Test admin service works OK.
> 
> 
> Thanks,
> 
> Colm O hEigeartaigh
> 
>

Re: Review Request 68428: RANGER-1935/RANGER-1738 - Upgrade Ranger to support Apache Hadoop 3.0.0

2018-09-05 Thread Colm O hEigeartaigh 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68428/
---

(Updated Sept. 5, 2018, 8:56 a.m.)


Review request for ranger.


Bugs: RANGER-1935
https://issues.apache.org/jira/browse/RANGER-1935


Repository: ranger


Description
---

Upgrade Ranger to use Apache Hadoop 3.0.0. Some comments:

- KMS reencryptEncryptedKey not implemented.
- HDFS unit test - HDFSContentSummaryTest test failing. @Ignore'd for now, I'll 
create a separate JIRA for this.
- Hive tests commented out - waiting for Hive upgrade task.
- Admin service users not appearing so in the patch it's using the old Hadoop 
jars. I'll create a separate JIRA for this to upgrade it as well.


Diffs (updated)
-

  
agents-audit/src/main/java/org/apache/ranger/audit/utils/InMemoryJAASConfiguration.java
 0f291389f 
  credentialbuilder/pom.xml 461dcd0b6 
  hdfs-agent/src/test/java/org/apache/ranger/services/hdfs/HDFSRangerTest.java 
a7215ce92 
  
hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
 7dc5c54b0 
  kms/pom.xml 6fc190f35 
  
kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/EagerKeyGeneratorKeyProviderCryptoExtension.java
 f853a844b 
  kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMS.java be3700ffa 
  
kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSAuthenticationFilter.java
 ada9a56f4 
  kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java 
b2540cb42 
  knox-agent/pom.xml 1d8fdae62 
  
plugin-yarn/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
 a4de016ea 
  pom.xml 8d7ea131e 
  ranger-examples/plugin-sampleapp/pom.xml bef0f5c1f 
  ranger-storm-plugin-shim/pom.xml 89efe34de 
  
ranger-yarn-plugin-shim/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
 7b7f1f303 
  security-admin/pom.xml 84bbd6650 
  src/main/assembly/hbase-agent.xml 3ebc334d1 
  src/main/assembly/hdfs-agent.xml 561d13737 
  src/main/assembly/hive-agent.xml 03bd31a55 
  src/main/assembly/kms.xml 7fbc1410c 
  src/main/assembly/knox-agent.xml 8357d4989 
  src/main/assembly/plugin-atlas.xml 8f4a64cb6 
  src/main/assembly/plugin-kafka.xml 97ff8ad1b 
  src/main/assembly/plugin-kms.xml 6d15f2a10 
  src/main/assembly/plugin-kylin.xml f4e082007 
  src/main/assembly/plugin-solr.xml de30bfb13 
  src/main/assembly/plugin-sqoop.xml d2bd69ab3 
  src/main/assembly/plugin-yarn.xml c6a48e888 
  src/main/assembly/ranger-tools.xml 249de9a2d 
  src/main/assembly/storm-agent.xml fdaf67863 
  src/main/assembly/tagsync.xml bc6e28b94 
  src/main/assembly/usersync.xml a6bad1dec 
  storm-agent/pom.xml a084e6891 
  tagsync/pom.xml e98cf63dc 
  ugsync/ldapconfigchecktool/ldapconfigcheck/pom.xml d49001a65 
  
ugsync/ldapconfigchecktool/ldapconfigcheck/src/main/java/org/apache/ranger/ldapconfigcheck/LdapConfig.java
 9a77cedf7 
  ugsync/pom.xml f4c24e739 
  unixauthclient/pom.xml cd3d15604 
  unixauthservice/pom.xml 2c0ed2eef 


Diff: https://reviews.apache.org/r/68428/diff/2/

Changes: https://reviews.apache.org/r/68428/diff/1-2/


Testing
---

Tested plugins: HDFS, Yarn, Storm (uses Hadoop 2.7.1), Knox, HBase, KMS, Sqoop, 
Kafka, Solr, usersync. Test admin service works OK.


Thanks,

Colm O hEigeartaigh

Review Request 68428: RANGER-1935/RANGER-1738 - Upgrade Ranger to support Apache Hadoop 3.0.0

2018-08-20 Thread Colm O hEigeartaigh 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68428/
---

Review request for ranger.


Bugs: RANGER-1935
https://issues.apache.org/jira/browse/RANGER-1935


Repository: ranger


Description
---

Upgrade Ranger to use Apache Hadoop 3.0.0. Some comments:

- KMS reencryptEncryptedKey not implemented.
- HDFS unit test - HDFSContentSummaryTest test failing. @Ignore'd for now, I'll 
create a separate JIRA for this.
- Hive tests commented out - waiting for Hive upgrade task.
- Admin service users not appearing so in the patch it's using the old Hadoop 
jars. I'll create a separate JIRA for this to upgrade it as well.


Diffs
-

  
agents-audit/src/main/java/org/apache/ranger/audit/utils/InMemoryJAASConfiguration.java
 0f291389f 
  credentialbuilder/pom.xml 461dcd0b6 
  hdfs-agent/src/test/java/org/apache/ranger/services/hdfs/HDFSRangerTest.java 
a7215ce92 
  
hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java
 7dc5c54b0 
  kms/pom.xml 6fc190f35 
  
kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/EagerKeyGeneratorKeyProviderCryptoExtension.java
 f853a844b 
  kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMS.java be3700ffa 
  
kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSAuthenticationFilter.java
 ada9a56f4 
  kms/src/main/java/org/apache/hadoop/crypto/key/kms/server/KMSWebApp.java 
b2540cb42 
  knox-agent/pom.xml 1d8fdae62 
  
plugin-yarn/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
 a4de016ea 
  pom.xml ae3f4be4c 
  ranger-examples/plugin-sampleapp/pom.xml bef0f5c1f 
  ranger-storm-plugin-shim/pom.xml 89efe34de 
  
ranger-yarn-plugin-shim/src/main/java/org/apache/ranger/authorization/yarn/authorizer/RangerYarnAuthorizer.java
 7b7f1f303 
  security-admin/pom.xml 84bbd6650 
  src/main/assembly/hbase-agent.xml 3ebc334d1 
  src/main/assembly/hdfs-agent.xml 561d13737 
  src/main/assembly/hive-agent.xml 03bd31a55 
  src/main/assembly/kms.xml 7fbc1410c 
  src/main/assembly/knox-agent.xml 8357d4989 
  src/main/assembly/plugin-atlas.xml 8f4a64cb6 
  src/main/assembly/plugin-kafka.xml 97ff8ad1b 
  src/main/assembly/plugin-kms.xml 6d15f2a10 
  src/main/assembly/plugin-kylin.xml f4e082007 
  src/main/assembly/plugin-solr.xml de30bfb13 
  src/main/assembly/plugin-sqoop.xml d2bd69ab3 
  src/main/assembly/plugin-yarn.xml c6a48e888 
  src/main/assembly/ranger-tools.xml 249de9a2d 
  src/main/assembly/storm-agent.xml fdaf67863 
  src/main/assembly/tagsync.xml d1b83df42 
  src/main/assembly/usersync.xml d170d8c1c 
  storm-agent/pom.xml a084e6891 
  tagsync/pom.xml e98cf63dc 
  ugsync/ldapconfigchecktool/ldapconfigcheck/pom.xml d49001a65 
  
ugsync/ldapconfigchecktool/ldapconfigcheck/src/main/java/org/apache/ranger/ldapconfigcheck/LdapConfig.java
 9a77cedf7 
  ugsync/pom.xml f4c24e739 
  unixauthclient/pom.xml cd3d15604 
  unixauthservice/pom.xml 2c0ed2eef 


Diff: https://reviews.apache.org/r/68428/diff/1/


Testing
---

Tested plugins: HDFS, Yarn, Storm (uses Hadoop 2.7.1), Knox, HBase, KMS, Sqoop, 
Kafka, Solr, usersync. Test admin service works OK.


Thanks,

Colm O hEigeartaigh