[jira] [Resolved] (RANGER-2330) Ensure that policy/resource based searches are security-zone aware

[Abhay Kulkarni (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Abhay Kulkarni resolved RANGER-2330.

Resolution: Fixed

Commit details:
master:
https://github.com/apache/ranger/commit/48b594c0aedede17b6bb241155a3a7426c1c92cc

> Ensure that policy/resource based searches are security-zone aware
> --
>
> Key: RANGER-2330
> URL: https://issues.apache.org/jira/browse/RANGER-2330
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: master
>Reporter: Abhay Kulkarni
>Assignee: Abhay Kulkarni
>Priority: Major
> Fix For: master
>
>
> Ranger Admin supports many APIs to browse and filter policies based on the 
> resource value or policy. The implementation of these APIs must return 
> results for only for the relevant security zone which contains specified 
> resource or policy. If the specified resource value does not correspond to a 
> unique security zone, then APIs must return result for all security zones 
> (including default security zone).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (RANGER-2148) Update Ranger Hive dependency version to 3.0

[t oo (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16768787#comment-16768787
 ] 

t oo commented on RANGER-2148:
--

how about skip to hive 3.1.x?

> Update Ranger Hive dependency version to 3.0 
> -
>
> Key: RANGER-2148
> URL: https://issues.apache.org/jira/browse/RANGER-2148
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: master
>Reporter: Ramesh Mani
>Assignee: Colm O hEigeartaigh
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: 
> 0001-RANGER-2148-Update-Ranger-Hive-dependency-version-to.patch
>
>
> Update Ranger Hive dependency version to 3.0 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69930: RANGER-2333: Logs does not get generated for Zone Description field available on Security Zone page.

[Abhay Kulkarni]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69930/#review212851
---


Ship it!




Ship It!

- Abhay Kulkarni


On Feb. 14, 2019, 9:11 a.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69930/
> ---
> 
> (Updated Feb. 14, 2019, 9:11 a.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
> Nikhil P, Nitin Galave, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2333
> https://issues.apache.org/jira/browse/RANGER-2333
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> **Problem Stamtents:**
> 
> 1) Zone Description field does not exist in x_security_zone table hence zone 
> desciption provided in UI does not get saved.
> 2) Zone name with blank space can be created which should be restricted.
> 3) adminUsers or adminUsersGroups and auditUsers or auditUsersGroups and 
> Resource field should be restricted for blank values.
> 4) All users should have access to Security zone module.
> 5) If users are not allowed to create service zone then the response code 
> should be 403 rather 400.
> 
> **Proposed Solutions:**
> 
> 1) Added a column in x_security_zone table added code to add/update 
> description field values.
> 2) Trimmed the zone name value before the existing validation check. 
> 3) Removed empty list items from the various list before the validation.
> 4) Added access on the security zone module to all users.
> 5) Changed response code and message.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/RangerSecurityZone.java
>  9eaf10214 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerSecurityZoneValidator.java
>  40137072a 
>   
> agents-common/src/test/java/org/apache/ranger/plugin/model/validation/RangerSecurityZoneValidatorTest.java
>  b16ccd6a4 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 
> 71cfa8f8c 
>   security-admin/db/mysql/patches/037-create-security-zone-schema.sql 
> 0df549137 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
> 6b569f254 
>   security-admin/db/oracle/patches/037-create-security-zone-schema.sql 
> e71f3db38 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
> a4e93cada 
>   security-admin/db/postgres/patches/037-create-security-zone-schema.sql 
> e81da8078 
>   
> security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
>  71f0ba410 
>   security-admin/db/sqlanywhere/patches/037-create-security-zone-schema.sql 
> b96b6e5c5 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
> d69c2dd37 
>   security-admin/db/sqlserver/patches/037-create-security-zone-schema.sql 
> 1c687220b 
>   security-admin/src/main/java/org/apache/ranger/biz/RangerBizUtil.java 
> d350fd1c0 
>   security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 933e99fff 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXSecurityZoneBase.java 
> 9c7871041 
>   
> security-admin/src/main/java/org/apache/ranger/patch/PatchAssignSecurityZonePersmissionToAdmin_J10026.java
>  64e39e391 
>   security-admin/src/main/java/org/apache/ranger/rest/SecurityZoneREST.java 
> 1145122cf 
>   
> security-admin/src/main/java/org/apache/ranger/service/RangerSecurityZoneService.java
>  cc796d5aa 
>   
> security-admin/src/main/java/org/apache/ranger/service/RangerSecurityZoneServiceBase.java
>  0620441b9 
>   
> security-admin/src/test/java/org/apache/ranger/rest/TestSecurityZoneREST.java 
> 456b858a3 
> 
> 
> Diff: https://reviews.apache.org/r/69930/diff/3/
> 
> 
> Testing
> ---
> 
> Tested above mentioned usecases on my local environments.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

[jira] [Commented] (RANGER-1935) Upgrade Ranger to support Apache Hadoop 3.0.0

[t oo (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-1935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16768788#comment-16768788
 ] 

t oo commented on RANGER-1935:
--

how about skip to hadoop 3.1.x or 3.2?

> Upgrade Ranger to support Apache Hadoop 3.0.0
> -
>
> Key: RANGER-1935
> URL: https://issues.apache.org/jira/browse/RANGER-1935
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Reporter: Colm O hEigeartaigh
>Assignee: Colm O hEigeartaigh
>Priority: Major
> Fix For: 2.0.0
>
> Attachments: 
> 0001-RANGER-1935-Upgrade-Ranger-to-support-Apache-Hadoop-.patch
>
>
> This task is to upgrade Ranger to support Apache Hadoop 3.0.0. Here are some 
> notes about the upgrade:
> a) The Hive plugin needs the Hadoop 3.0.0 jars to run the tests properly, as 
> Hive only supports the older Hadoop version, so an exclusion and some 
> additional 3.0.0 dependencies need to be added.
> b) The Storm plugin bundles the hadoop-auth jars in storm-core (although they 
> really should be renamed here). Therefore, we have no option but to package 
> Storm with the Hadoop 2.7.x jars, until such time that Storm upgrades the 
> Hadoop dependency.
> This is an initial patch to get some feedback. If there is broad agreement on 
> the upgrade I will test the distributions properly.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69966: Context-Enrichers need to clean up completely when the policy-engine is destroyed

[Ramesh Mani]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69966/#review212837
---


Ship it!




Ship It!

- Ramesh Mani


On Feb. 13, 2019, 12:29 a.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69966/
> ---
> 
> (Updated Feb. 13, 2019, 12:29 a.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj, Ramesh Mani, and Sailaja 
> Polavarapu.
> 
> 
> Bugs: RANGER-2337
> https://issues.apache.org/jira/browse/RANGER-2337
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> An instance of context enricher is created when policy-engine is 
> instantiated. When policy-engine is destroyed, all references to context 
> enricher need to be cleaned up.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerAbstractContextEnricher.java
>  a74511298 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/contextenricher/RangerTagEnricher.java
>  d671b73be 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java
>  265d7a64d 
> 
> 
> Diff: https://reviews.apache.org/r/69966/diff/1/
> 
> 
> Testing
> ---
> 
> Passes all unit tests
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 69984: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration

[Gautam Borad]

> On Feb. 14, 2019, 10:28 a.m., Zsombor Gegesy wrote:
> > I see two review request, I guess, one of it is duplicate, could you please 
> > close one of it?

Its for the master branch. The other one is for ranger-0.7 branch


- Gautam


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69984/#review212825
---


On Feb. 14, 2019, 9:55 a.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69984/
> ---
> 
> (Updated Feb. 14, 2019, 9:55 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni, 
> Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja 
> Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2331
> https://issues.apache.org/jira/browse/RANGER-2331
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> User story: As a security admin, I want to manage encryption keys for 
> securing my Hadoop cluster files in Ranger KMS service with Safenet KeySecure 
> crypto platform.
> 
> For Safenet KeySecure overview refer to: 
> https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/
> 
> Acceptance Criteria:
> 
> 1) Ranger KMS has ability to configure Safenet KeySecure platform to be used 
> for key offload
> 
> 2) Ranger KMS provides ability to provide key management functions (create 
> keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform
> 
> 3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure 
> platform
> 
> 
> Diffs
> -
> 
>   kms/config/kms-webapp/dbks-site.xml ec649a7 
>   kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION 
>   kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION 
>   kms/scripts/install.properties aea0bb8 
>   kms/scripts/setup.sh c695e74 
>   kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java 
> PRE-CREATION 
>   kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java dd4408f 
>   
> kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java 
> PRE-CREATION 
>   kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 4f337bb 
>   kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 
> 88a545e 
>   kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java eb4f75a 
>   kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java 
> PRE-CREATION 
>   src/main/assembly/kms.xml 3adc55c 
> 
> 
> Diff: https://reviews.apache.org/r/69984/diff/1/
> 
> 
> Testing
> ---
> 
> Verified below scenario:
> 
> 1) Fresh Installation Of Ranger KMS with Safenet Key Secure (NAE-XML Protocol)
> 2) DB to Key Secure (NAE-XML) master key Migration utility
> 3) Key Secure (NAE-XML) to DB master key Migration utility
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Re: Review Request 69985: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration

[Zsombor Gegesy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69985/#review212827
---




kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java
Lines 82 (patched)


I would expect that if the import was failed, the process ends with a 
non-zero exit code.



kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
Lines 50 (patched)


You can mark all variable as final



kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
Lines 52 (patched)


This variable shouldn't be static.



kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
Lines 59 (patched)


Unnecessary constructor



kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
Lines 87 (patched)


Why don't you simply re-throw the exception(s)?
Having a non-usable RangerSafenetKeySecure object for the caller doesn't 
make too much sense.
So later, you don't need to check that myStore is not null



kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
Lines 115 (patched)


Why the e.printStackTrace(), could you just add that 'e' to the 
logger.error call?



kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
Lines 117 (patched)


It's not an issue with your code, but I think RangerKMSKI is a bit 
confusing, what's the reason for having a 'Throwable' in the method 
declaration, and returning a boolean=false. One of them is unnecessary.



kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
Lines 135 (patched)


If 'key' is null, then it will throw an NPE from here, get catched in the 
'catch (Exception e)' and returned null later. Maybe it's simpler to return 
null in the if: 

   if (key == null) {
   logger.warn('getMasterKey(pw) returned null!');
   return null;
   }



kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
Lines 154 (patched)


I don't get, why it throws NoSuchAlgorithmException, CertificateException, 
and IOException, but catch KeyStoreException ?


- Zsombor Gegesy


On Feb. 14, 2019, 9:59 a.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69985/
> ---
> 
> (Updated Feb. 14, 2019, 9:59 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni, 
> Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja 
> Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2331
> https://issues.apache.org/jira/browse/RANGER-2331
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> User story: As a security admin, I want to manage encryption keys for 
> securing my Hadoop cluster files in Ranger KMS service with Safenet KeySecure 
> crypto platform.
> 
> 
> For Safenet KeySecure overview refer to: 
> https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/
> 
> 
> Acceptance Criteria:
> 
> 
> 1) Ranger KMS has ability to configure Safenet KeySecure platform to be used 
> for key offload
> 
> 
> 2) Ranger KMS provides ability to provide key management functions (create 
> keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform
> 
> 
> 3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure 
> platform
> 
> 
> Diffs
> -
> 
>   kms/config/kms-webapp/dbks-site.xml 0e0f2ec 
>   kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION 
>   kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION 
>   kms/scripts/install.properties ddc779d 
>   kms/scripts/setup.sh 2db05b8 
>   kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java 
> PRE-CREATION 
>   kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 22dce0f 
>   
> kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java 
> PRE-CREATION 
>   kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 1abbf8e 
>   kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 
> 267fcf0 
>   kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 5614c16 
>   kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java 
> PRE-CREATION 
>   src/main/assembly/kms.xml 

Re: Review Request 69984: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration

[Zsombor Gegesy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69984/#review212825
---



I see two review request, I guess, one of it is duplicate, could you please 
close one of it?

- Zsombor Gegesy


On Feb. 14, 2019, 9:55 a.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69984/
> ---
> 
> (Updated Feb. 14, 2019, 9:55 a.m.)
> 
> 
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni, 
> Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja 
> Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2331
> https://issues.apache.org/jira/browse/RANGER-2331
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> User story: As a security admin, I want to manage encryption keys for 
> securing my Hadoop cluster files in Ranger KMS service with Safenet KeySecure 
> crypto platform.
> 
> For Safenet KeySecure overview refer to: 
> https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/
> 
> Acceptance Criteria:
> 
> 1) Ranger KMS has ability to configure Safenet KeySecure platform to be used 
> for key offload
> 
> 2) Ranger KMS provides ability to provide key management functions (create 
> keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform
> 
> 3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure 
> platform
> 
> 
> Diffs
> -
> 
>   kms/config/kms-webapp/dbks-site.xml ec649a7 
>   kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION 
>   kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION 
>   kms/scripts/install.properties aea0bb8 
>   kms/scripts/setup.sh c695e74 
>   kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java 
> PRE-CREATION 
>   kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java dd4408f 
>   
> kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java 
> PRE-CREATION 
>   kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 4f337bb 
>   kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 
> 88a545e 
>   kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java eb4f75a 
>   kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java 
> PRE-CREATION 
>   src/main/assembly/kms.xml 3adc55c 
> 
> 
> Diff: https://reviews.apache.org/r/69984/diff/1/
> 
> 
> Testing
> ---
> 
> Verified below scenario:
> 
> 1) Fresh Installation Of Ranger KMS with Safenet Key Secure (NAE-XML Protocol)
> 2) DB to Key Secure (NAE-XML) master key Migration utility
> 3) Key Secure (NAE-XML) to DB master key Migration utility
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

[jira] [Commented] (RANGER-2331) Ranger-KMS - KeySecure HSM Integration

[Gautam Borad (JIRA)]

[ 
https://issues.apache.org/jira/browse/RANGER-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16768046#comment-16768046
 ] 

Gautam Borad commented on RANGER-2331:
--

Review Request for master branch : [https://reviews.apache.org/r/69984/]

Review Request for ranger-0.7 branch : https://reviews.apache.org/r/69985/

> Ranger-KMS - KeySecure HSM Integration
> --
>
> Key: RANGER-2331
> URL: https://issues.apache.org/jira/browse/RANGER-2331
> Project: Ranger
>  Issue Type: New Feature
>  Components: kms, Ranger
>Reporter: Velmurugan Periasamy
>Assignee: Gautam Borad
>Priority: Major
> Fix For: 0.7.2, 2.0.0
>
> Attachments: RANGER-2331-01.patch, RANGER-2331-ranger-0.7.patch, 
> RANGER-2331.patch
>
>
> This JIRA is to support Ranger KMS integration with Gemalto KeySecure HSM to 
> manage master keys.
> Similar to Luna integration - https://issues.apache.org/jira/browse/RANGER-868
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Review Request 69985: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration

[Gautam Borad]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69985/
---

Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni, 
Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja 
Polavarapu, and Velmurugan Periasamy.


Bugs: RANGER-2331
https://issues.apache.org/jira/browse/RANGER-2331


Repository: ranger


Description
---

User story: As a security admin, I want to manage encryption keys for securing 
my Hadoop cluster files in Ranger KMS service with Safenet KeySecure crypto 
platform.


For Safenet KeySecure overview refer to: 
https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/


Acceptance Criteria:


1) Ranger KMS has ability to configure Safenet KeySecure platform to be used 
for key offload


2) Ranger KMS provides ability to provide key management functions (create 
keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform


3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure 
platform


Diffs
-

  kms/config/kms-webapp/dbks-site.xml 0e0f2ec 
  kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION 
  kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION 
  kms/scripts/install.properties ddc779d 
  kms/scripts/setup.sh 2db05b8 
  kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java 
PRE-CREATION 
  kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 22dce0f 
  kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java 
PRE-CREATION 
  kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 1abbf8e 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 
267fcf0 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 5614c16 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java 
PRE-CREATION 
  src/main/assembly/kms.xml fca6a32 


Diff: https://reviews.apache.org/r/69985/diff/1/


Testing
---

Verified below scenario:


1) Fresh Installation Of Ranger KMS with Safenet Key Secure (NAE-XML Protocol)
2) DB to Key Secure (NAE-XML) master key Migration utility
3) Key Secure (NAE-XML) to DB master key Migration utility


Thanks,

Gautam Borad

Review Request 69984: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration

[Gautam Borad]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69984/
---

Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni, 
Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja 
Polavarapu, and Velmurugan Periasamy.


Bugs: RANGER-2331
https://issues.apache.org/jira/browse/RANGER-2331


Repository: ranger


Description
---

User story: As a security admin, I want to manage encryption keys for securing 
my Hadoop cluster files in Ranger KMS service with Safenet KeySecure crypto 
platform.

For Safenet KeySecure overview refer to: 
https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/

Acceptance Criteria:

1) Ranger KMS has ability to configure Safenet KeySecure platform to be used 
for key offload

2) Ranger KMS provides ability to provide key management functions (create 
keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform

3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure 
platform


Diffs
-

  kms/config/kms-webapp/dbks-site.xml ec649a7 
  kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION 
  kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION 
  kms/scripts/install.properties aea0bb8 
  kms/scripts/setup.sh c695e74 
  kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java 
PRE-CREATION 
  kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java dd4408f 
  kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java 
PRE-CREATION 
  kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 4f337bb 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java 
88a545e 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java eb4f75a 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java 
PRE-CREATION 
  src/main/assembly/kms.xml 3adc55c 


Diff: https://reviews.apache.org/r/69984/diff/1/


Testing
---

Verified below scenario:

1) Fresh Installation Of Ranger KMS with Safenet Key Secure (NAE-XML Protocol)
2) DB to Key Secure (NAE-XML) master key Migration utility
3) Key Secure (NAE-XML) to DB master key Migration utility


Thanks,

Gautam Borad

[jira] [Updated] (RANGER-2331) Ranger-KMS - KeySecure HSM Integration

[Gautam Borad (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gautam Borad updated RANGER-2331:
-
Attachment: (was: RANGER-2331-01.path)

> Ranger-KMS - KeySecure HSM Integration
> --
>
> Key: RANGER-2331
> URL: https://issues.apache.org/jira/browse/RANGER-2331
> Project: Ranger
>  Issue Type: New Feature
>  Components: kms, Ranger
>Reporter: Velmurugan Periasamy
>Assignee: Gautam Borad
>Priority: Major
> Fix For: 0.7.2, 2.0.0
>
> Attachments: RANGER-2331-01.patch, RANGER-2331-ranger-0.7.patch, 
> RANGER-2331.patch
>
>
> This JIRA is to support Ranger KMS integration with Gemalto KeySecure HSM to 
> manage master keys.
> Similar to Luna integration - https://issues.apache.org/jira/browse/RANGER-868
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (RANGER-2331) Ranger-KMS - KeySecure HSM Integration

[Gautam Borad (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gautam Borad updated RANGER-2331:
-
Attachment: RANGER-2331-01.patch

> Ranger-KMS - KeySecure HSM Integration
> --
>
> Key: RANGER-2331
> URL: https://issues.apache.org/jira/browse/RANGER-2331
> Project: Ranger
>  Issue Type: New Feature
>  Components: kms, Ranger
>Reporter: Velmurugan Periasamy
>Assignee: Gautam Borad
>Priority: Major
> Fix For: 0.7.2, 2.0.0
>
> Attachments: RANGER-2331-01.patch, RANGER-2331-ranger-0.7.patch, 
> RANGER-2331.patch
>
>
> This JIRA is to support Ranger KMS integration with Gemalto KeySecure HSM to 
> manage master keys.
> Similar to Luna integration - https://issues.apache.org/jira/browse/RANGER-868
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (RANGER-2331) Ranger-KMS - KeySecure HSM Integration

[Gautam Borad (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gautam Borad updated RANGER-2331:
-
Attachment: RANGER-2331-01.path

> Ranger-KMS - KeySecure HSM Integration
> --
>
> Key: RANGER-2331
> URL: https://issues.apache.org/jira/browse/RANGER-2331
> Project: Ranger
>  Issue Type: New Feature
>  Components: kms, Ranger
>Reporter: Velmurugan Periasamy
>Assignee: Gautam Borad
>Priority: Major
> Fix For: 0.7.2, 2.0.0
>
> Attachments: RANGER-2331-01.path, RANGER-2331-ranger-0.7.patch, 
> RANGER-2331.patch
>
>
> This JIRA is to support Ranger KMS integration with Gemalto KeySecure HSM to 
> manage master keys.
> Similar to Luna integration - https://issues.apache.org/jira/browse/RANGER-868
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (RANGER-2331) Ranger-KMS - KeySecure HSM Integration

[Ankita Sinha (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankita Sinha updated RANGER-2331:
-
Attachment: (was: RANGER-2331-ranger-0.7.patch)

> Ranger-KMS - KeySecure HSM Integration
> --
>
> Key: RANGER-2331
> URL: https://issues.apache.org/jira/browse/RANGER-2331
> Project: Ranger
>  Issue Type: New Feature
>  Components: kms, Ranger
>Reporter: Velmurugan Periasamy
>Assignee: Ankita Sinha
>Priority: Major
> Fix For: 0.7.2, 2.0.0
>
> Attachments: RANGER-2331.patch
>
>
> This JIRA is to support Ranger KMS integration with Gemalto KeySecure HSM to 
> manage master keys.
> Similar to Luna integration - https://issues.apache.org/jira/browse/RANGER-868
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 69930: RANGER-2333: Logs does not get generated for Zone Description field available on Security Zone page.

[Pradeep Agrawal]

> On Feb. 13, 2019, 10:56 p.m., Abhay Kulkarni wrote:
> > security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql
> > Line 2690 (original), 2691 (patched)
> > 
> >
> > Was this a bug in the earlier version of this SQL script?

Yes


- Pradeep


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69930/#review212810
---


On Feb. 14, 2019, 9:11 a.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69930/
> ---
> 
> (Updated Feb. 14, 2019, 9:11 a.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
> Nikhil P, Nitin Galave, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2333
> https://issues.apache.org/jira/browse/RANGER-2333
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> **Problem Stamtents:**
> 
> 1) Zone Description field does not exist in x_security_zone table hence zone 
> desciption provided in UI does not get saved.
> 2) Zone name with blank space can be created which should be restricted.
> 3) adminUsers or adminUsersGroups and auditUsers or auditUsersGroups and 
> Resource field should be restricted for blank values.
> 4) All users should have access to Security zone module.
> 5) If users are not allowed to create service zone then the response code 
> should be 403 rather 400.
> 
> **Proposed Solutions:**
> 
> 1) Added a column in x_security_zone table added code to add/update 
> description field values.
> 2) Trimmed the zone name value before the existing validation check. 
> 3) Removed empty list items from the various list before the validation.
> 4) Added access on the security zone module to all users.
> 5) Changed response code and message.
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/RangerSecurityZone.java
>  9eaf10214 
>   
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerSecurityZoneValidator.java
>  40137072a 
>   
> agents-common/src/test/java/org/apache/ranger/plugin/model/validation/RangerSecurityZoneValidatorTest.java
>  b16ccd6a4 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 
> 71cfa8f8c 
>   security-admin/db/mysql/patches/037-create-security-zone-schema.sql 
> 0df549137 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
> 6b569f254 
>   security-admin/db/oracle/patches/037-create-security-zone-schema.sql 
> e71f3db38 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
> a4e93cada 
>   security-admin/db/postgres/patches/037-create-security-zone-schema.sql 
> e81da8078 
>   
> security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
>  71f0ba410 
>   security-admin/db/sqlanywhere/patches/037-create-security-zone-schema.sql 
> b96b6e5c5 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
> d69c2dd37 
>   security-admin/db/sqlserver/patches/037-create-security-zone-schema.sql 
> 1c687220b 
>   security-admin/src/main/java/org/apache/ranger/biz/RangerBizUtil.java 
> d350fd1c0 
>   security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 933e99fff 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXSecurityZoneBase.java 
> 9c7871041 
>   
> security-admin/src/main/java/org/apache/ranger/patch/PatchAssignSecurityZonePersmissionToAdmin_J10026.java
>  64e39e391 
>   security-admin/src/main/java/org/apache/ranger/rest/SecurityZoneREST.java 
> 1145122cf 
>   
> security-admin/src/main/java/org/apache/ranger/service/RangerSecurityZoneService.java
>  cc796d5aa 
>   
> security-admin/src/main/java/org/apache/ranger/service/RangerSecurityZoneServiceBase.java
>  0620441b9 
>   
> security-admin/src/test/java/org/apache/ranger/rest/TestSecurityZoneREST.java 
> 456b858a3 
> 
> 
> Diff: https://reviews.apache.org/r/69930/diff/3/
> 
> 
> Testing
> ---
> 
> Tested above mentioned usecases on my local environments.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

Re: Review Request 69930: RANGER-2333: Logs does not get generated for Zone Description field available on Security Zone page.

[Pradeep Agrawal]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69930/
---

(Updated Feb. 14, 2019, 9:11 a.m.)


Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
Nikhil P, Nitin Galave, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and 
Velmurugan Periasamy.


Changes
---

addressed review comments


Bugs: RANGER-2333
https://issues.apache.org/jira/browse/RANGER-2333


Repository: ranger


Description
---

**Problem Stamtents:**

1) Zone Description field does not exist in x_security_zone table hence zone 
desciption provided in UI does not get saved.
2) Zone name with blank space can be created which should be restricted.
3) adminUsers or adminUsersGroups and auditUsers or auditUsersGroups and 
Resource field should be restricted for blank values.
4) All users should have access to Security zone module.
5) If users are not allowed to create service zone then the response code 
should be 403 rather 400.

**Proposed Solutions:**

1) Added a column in x_security_zone table added code to add/update description 
field values.
2) Trimmed the zone name value before the existing validation check. 
3) Removed empty list items from the various list before the validation.
4) Added access on the security zone module to all users.
5) Changed response code and message.


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/model/RangerSecurityZone.java
 9eaf10214 
  
agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerSecurityZoneValidator.java
 40137072a 
  
agents-common/src/test/java/org/apache/ranger/plugin/model/validation/RangerSecurityZoneValidatorTest.java
 b16ccd6a4 
  security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 71cfa8f8c 
  security-admin/db/mysql/patches/037-create-security-zone-schema.sql 0df549137 
  security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
6b569f254 
  security-admin/db/oracle/patches/037-create-security-zone-schema.sql 
e71f3db38 
  security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
a4e93cada 
  security-admin/db/postgres/patches/037-create-security-zone-schema.sql 
e81da8078 
  
security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 
71f0ba410 
  security-admin/db/sqlanywhere/patches/037-create-security-zone-schema.sql 
b96b6e5c5 
  security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
d69c2dd37 
  security-admin/db/sqlserver/patches/037-create-security-zone-schema.sql 
1c687220b 
  security-admin/src/main/java/org/apache/ranger/biz/RangerBizUtil.java 
d350fd1c0 
  security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 933e99fff 
  security-admin/src/main/java/org/apache/ranger/entity/XXSecurityZoneBase.java 
9c7871041 
  
security-admin/src/main/java/org/apache/ranger/patch/PatchAssignSecurityZonePersmissionToAdmin_J10026.java
 64e39e391 
  security-admin/src/main/java/org/apache/ranger/rest/SecurityZoneREST.java 
1145122cf 
  
security-admin/src/main/java/org/apache/ranger/service/RangerSecurityZoneService.java
 cc796d5aa 
  
security-admin/src/main/java/org/apache/ranger/service/RangerSecurityZoneServiceBase.java
 0620441b9 
  security-admin/src/test/java/org/apache/ranger/rest/TestSecurityZoneREST.java 
456b858a3 


Diff: https://reviews.apache.org/r/69930/diff/3/

Changes: https://reviews.apache.org/r/69930/diff/2-3/


Testing
---

Tested above mentioned usecases on my local environments.


Thanks,

Pradeep Agrawal

[jira] [Assigned] (RANGER-2331) Ranger-KMS - KeySecure HSM Integration

[Ankita Sinha (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankita Sinha reassigned RANGER-2331:


Assignee: Gautam Borad  (was: Ankita Sinha)

> Ranger-KMS - KeySecure HSM Integration
> --
>
> Key: RANGER-2331
> URL: https://issues.apache.org/jira/browse/RANGER-2331
> Project: Ranger
>  Issue Type: New Feature
>  Components: kms, Ranger
>Reporter: Velmurugan Periasamy
>Assignee: Gautam Borad
>Priority: Major
> Fix For: 0.7.2, 2.0.0
>
> Attachments: RANGER-2331.patch
>
>
> This JIRA is to support Ranger KMS integration with Gemalto KeySecure HSM to 
> manage master keys.
> Similar to Luna integration - https://issues.apache.org/jira/browse/RANGER-868
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (RANGER-2331) Ranger-KMS - KeySecure HSM Integration

[Ankita Sinha (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankita Sinha updated RANGER-2331:
-
Attachment: (was: RANGER-2331-01.path)

> Ranger-KMS - KeySecure HSM Integration
> --
>
> Key: RANGER-2331
> URL: https://issues.apache.org/jira/browse/RANGER-2331
> Project: Ranger
>  Issue Type: New Feature
>  Components: kms, Ranger
>Reporter: Velmurugan Periasamy
>Assignee: Ankita Sinha
>Priority: Major
> Fix For: 0.7.2, 2.0.0
>
> Attachments: RANGER-2331.patch
>
>
> This JIRA is to support Ranger KMS integration with Gemalto KeySecure HSM to 
> manage master keys.
> Similar to Luna integration - https://issues.apache.org/jira/browse/RANGER-868
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (RANGER-2331) Ranger-KMS - KeySecure HSM Integration

[Ankita Sinha (JIRA)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankita Sinha updated RANGER-2331:
-
Attachment: RANGER-2331-ranger-0.7.patch

> Ranger-KMS - KeySecure HSM Integration
> --
>
> Key: RANGER-2331
> URL: https://issues.apache.org/jira/browse/RANGER-2331
> Project: Ranger
>  Issue Type: New Feature
>  Components: kms, Ranger
>Reporter: Velmurugan Periasamy
>Assignee: Ankita Sinha
>Priority: Major
> Fix For: 0.7.2, 2.0.0
>
> Attachments: RANGER-2331-01.path, RANGER-2331-ranger-0.7.patch, 
> RANGER-2331.patch
>
>
> This JIRA is to support Ranger KMS integration with Gemalto KeySecure HSM to 
> manage master keys.
> Similar to Luna integration - https://issues.apache.org/jira/browse/RANGER-868
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)