Re: Review Request 74288: RANGER-4058 : Improvement in Ranger React

[Dhaval Rajpara]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74288/#review225172
---


Ship it!




Ship It!

- Dhaval Rajpara


On Feb. 3, 2023, 2:02 p.m., Brijesh Bhalala wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74288/
> ---
> 
> (Updated Feb. 3, 2023, 2:02 p.m.)
> 
> 
> Review request for ranger, Dhaval Rajpara, Madhan Neethiraj, Mehul Parikh, 
> Mugdha Varadkar, Nikunj Pansuriya, and Nitin Galave.
> 
> 
> Bugs: RANGER-4058
> https://issues.apache.org/jira/browse/RANGER-4058
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> 1)Landing Page :-
> -Scroll for more services at  service list in landing page.
> - Import Module zone filtering services issue solved
> -Sorting the service list by service name.
> -Loader Issue in Import Module in landing page solved.
> 2)Policy view modal - 
>  -Policy Items Permission data minimum width  if 
>permissions are more
> 3)Policy Logs :-
>   - Added Policy ID in Import Delete Policy Logs at Audit 
>Admin Tab
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/webapp/react-webapp/package-lock.json 
> f37a2e2c03d5b0680672681b223fbedd6efc02cc 
>   security-admin/src/main/webapp/react-webapp/src/styles/style.css 
> 77ecb7105121cef763545c534c0b96f69d69e07f 
>   
> security-admin/src/main/webapp/react-webapp/src/views/AuditEvent/AdminLogs/PolicyLogs.jsx
>  d8b684c47afdc2b2b006f3087b3f17c808d1efbf 
>   
> security-admin/src/main/webapp/react-webapp/src/views/AuditEvent/AdminLogs/PolicyViewDetails.jsx
>  accd48b0ca663d5b882350b23e4b0157efa3526b 
>   
> security-admin/src/main/webapp/react-webapp/src/views/SecurityZone/ZoneDisplay.jsx
>  cb9415e7b58ccd39fb0b21bca714b24bdd269a50 
>   
> security-admin/src/main/webapp/react-webapp/src/views/ServiceManager/ExportPolicy.jsx
>  5ce61e73fba39b1c0a992e1c1f0b70bb5085ec7d 
>   
> security-admin/src/main/webapp/react-webapp/src/views/ServiceManager/ImportPolicy.jsx
>  e67e88e410a0607c5a7c1b4aa79c8e4737fb94fc 
>   
> security-admin/src/main/webapp/react-webapp/src/views/ServiceManager/ServiceDefinition.jsx
>  15174080bf056c1efd5884f3bb3530b2ac61eeed 
>   
> security-admin/src/main/webapp/react-webapp/src/views/ServiceManager/ServiceDefinitions.jsx
>  aa82bbf323a3f6e7aac220db81be2f28f7a62745 
>   
> security-admin/src/main/webapp/react-webapp/src/views/UserGroupRoleListing/groups_details/GroupListing.jsx
>  57eacc8e0e96850b46723070da5fc3408eccc3f0 
> 
> 
> Diff: https://reviews.apache.org/r/74288/diff/2/
> 
> 
> Testing
> ---
> 
> 1)Build and Verified Ranger Admin setup with this changes.
> 2)Verified the following things:-
>   1)CRUD Operations for Policy, Services.
>   2)Landing Page.
>   3)Loader at Import Export Module.
> 
> 
> Thanks,
> 
> Brijesh Bhalala
> 
>

[jira] [Commented] (RANGER-3828) Fine-grained Access Control over nested structures

[Madhan Neethiraj (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17686171#comment-17686171
 ] 

Madhan Neethiraj commented on RANGER-3828:
--

> Does this mean 2.4.0, but not 3.0.0?

[~barbara_eck...@comcast.com] - the plugin is present in both 3.0.0 and 2.4.0 
release branches (master and ranger-2.4). I updated the {{Fix Version/s}} field 
earlier today to reflect this.

> Fine-grained Access Control over nested structures
> --
>
> Key: RANGER-3828
> URL: https://issues.apache.org/jira/browse/RANGER-3828
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins, Ranger
>Reporter: Barbara Eckman
>Assignee: Barbara Eckman
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
> Attachments: image001.png, image002.png, image003.png
>
>
> It would be nice to be able to do fine-grained access control (FGA) over 
> nested structures, e.g., the JSON responses of API calls.  This requires the 
> individual attributes in a JSON object to be first-class metadata objects 
> which can be tagged and on which policies can be written.  We have built a 
> plugin and the corresponding Apache Atlas metadata structures and 
> tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures.   Our 
> instigating use case was FGA over the JSON responses of API calls, but this 
> plugin has potential value anywhere FGA over the individual attributes of 
> nested structures is needed, eg JSON messages read from Kafka topics.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-3899) Policy creation takes more time when there are more users,groups,roles

[Ramachandran (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ramachandran updated RANGER-3899:
-
Description: 
*{color:#ff}Policy Creation Steps in Apache Ranger:{color}*{color:#ff} 
{color}

 
{code:java}
1. Get the service by Name:
RangerService service = getServiceByName(policy.getService());→ 1DB Read call
2. Get XXServiceDef by Name
XXServiceDef xServiceDef = 
daoMgr.getXXServiceDef().findByName(service.getType()); → 1DB Read call
3. Get the existing XXPolicy by Name
XXPolicy existing = 
daoMgr.getXXPolicy().findByNameAndServiceIdAndZoneId(policy.getName(), 
service.getId(), zoneId); → 1DB Read call
4. Create a policy
policy = policyService.create(policy, true); → 1DB Write call
5. Get policy by Id
XXPolicy xCreatedPolicy = daoMgr.getXXPolicy().getById(policy.getId()); → 1DB 
Read call
6. createObjectDataHistory for the newly created policy
dataHistService.createObjectDataHistory(createdPolicy, 
RangerDataHistService.ACTION_CREATE); → 1DB Write call
7. createTrxLog for the newly created policy
bizUtil.createTrxLog(trxLogList); → 1DB Write call
8. Create Policy Label for the newly created policy
createOrMapLabels(xCreatedPolicy, uniquePolicyLabels); → 1DB Write call
9. Create Policy Mapping For Ref Table
policyRefUpdater.createNewPolMappingForRefTable(policy, xCreatedPolicy, 
xServiceDef);
1. Getting 
roleNames,groupNames,userNames,accessTypes,dataMaskTypes,conditionTypes from 
each policy items of the policy
2. Getting resourceNames from the policy
3.For each resource
  Getting the XXResourceDef by resource and policy.getId() 
  XXResourceDef xResDef = 
daoMgr.getXXResourceDef().findByNameAndPolicyId(resource, policy.getId()); → 
1DB Read call for each resource
  populate XXPolicyRefResource
4. Create the batch of XXPolicyRefResource in DB 
  daoMgr.getXXPolicyRefResource().batchCreate(xPolResources); → 1DB Batch Write 
call
5.For each roleNames 
   Getting XXRole by roleName
   XXRole xRole = daoMgr.getXXRole().findByRoleName(role); → 1DB Read call for 
each role
   populate XXPolicyRefRole
6. Create the batch of XXPolicyRefRole in DB
    daoMgr.getXXPolicyRefResource().batchCreate(xPolRoles); → 1DB Batch Write 
call
7.For each groupNames
Getting the XXGroup by groupName
XXGroup xGroup = daoMgr.getXXGroup().findByGroupName(context.group.getName());→ 
1DB Read call for each group
populate XXPolicyRefGroup
Insert into DB
daoMgr.getXXPolicyRefGroup().create(xPolGroup);→ 1DB write call for each group
8.For each userNames 
Getting XXUser by userName
XXUser xUser = daoMgr.getXXUser().findByUserName(user); → 1DB Read call for 
each user
populate XXPolicyRefUser
daoMgr.getXXPolicyRefUser().create(xPolUser); → 1DB write call for each user
9.For each accessTypes
Getting the XXAccessTypeDef by accessType and xPolicy.getService() 
XXAccessTypeDef xAccTypeDef = 
daoMgr.getXXAccessTypeDef().findByNameAndServiceId(accessType, 
xPolicy.getService()); → 1DB Read call for each accesType
populate XXAccessTypeDef
10. Create the batch of xPolAccesses in DB
daoMgr.getXXPolicyRefAccessType().batchCreate(xPolAccesses); → 1DB Batch Write 
call
11.For each conditionTypes
Getting the XXPolicyConditionDef by condition and xServiceDef.getId()
XXPolicyConditionDef xPolCondDef = 
daoMgr.getXXPolicyConditionDef().findByServiceDefIdAndName(xServiceDef.getId(), 
condition); → 1DB Read call for each conditionType
populate XXPolicyConditionDef
12. Create the batch of xPolConds in DB
daoMgr.getXXPolicyRefCondition().batchCreate(xPolConds);  → 1DB Batch Write call
{code}
 

*{color:#ff}A total number of DB calls involved for the below Policy 
creation in Apache Ranger(Before Fix):{color}*
 
The Policy contains 500 users,5 access Types (permissions),12 resources 
(1DB,1Table,10 columns)

     
||DB Write count||DB Read count||DB Batch Write count||Time is taken to create 
the policy||
|504|526|3|{color:#ff}4~8 seconds{color}|

*{color:#ff}A total number of DB calls involved for the below Policy 
creation in Apache Ranger(After Fix):{color}*

 

 The policy contains 500 users,5 access Types (permissions),12 resources 
(1DB,1Table,10 columns)

{color:#ff}Proposal :{color}

{color:#ff}After changing to batch create of XXPolicyRefUser  instead of 
calling 1 DB write XXPolicyRefUser for every user of the policy:{color}

 
||DB Write count||DB Read count||DB Batch Write count||Time is taken to create 
the policy||
|6|526|5|{color:#ff}2.6~4 seconds{color}|

A similar will be applied for roles and groups as well.

 

After  fixing (RANGER-3899, RANGER-2732,)

When Policy contains 500 users,5 access Types (permissions),12 resources 
(1DB,1Table,10 columns)

After the Batch read the calls will be reduced into 
||DB Write count||DB Read count||DB Batch Write count||Time is taken to create 
the policy||
|6|27|5|<2.6 seconds |

DB read calls will be reduced to {color:#de350b}27{color} from 
{color:#403294}526{color}

Re: Review Request 74301: RANGER-2713: Remove audit fields from XXPolicyRef objects

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74301/#review225171
---




security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql
Line 1433 (original), 1397 (patched)


Columns create_time/update_time/added_by_id/upd_by_id can be removed from 
following tables as well:
- x_security_zone_ref_resource
- x_security_zone_ref_service
- x_security_zone_ref_tag_srvc
- x_security_zone_ref_user
- x_security_zone_ref_group
- x_role_ref_user
- x_role_ref_group
- x_role_ref_role

Also, I suggest to remove column guid from all _ref tables as they are not 
referenced from Java entity classes.


- Madhan Neethiraj


On Feb. 8, 2023, 9:05 p.m., Andrew Luo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74301/
> ---
> 
> (Updated Feb. 8, 2023, 9:05 p.m.)
> 
> 
> Review request for ranger and Pradeep Agrawal.
> 
> 
> Bugs: RANGER-2713
> https://issues.apache.org/jira/browse/RANGER-2713
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> XXPolicyRef objects have fields such as create time, update time, added by 
> user ID, updated by user ID, but there fields are entirely useless since they 
> are all copied from the XXPolicy object. In addition, while improving 
> performance for creation of policies with large numbers of users, we 
> discovered that a lot of time was being spent in JPA converting these Date 
> objects especially. After removing these fields we saw a significant 
> performance improvement (a secondary benefit is less database space usage).
> 
> We previously tried this commit and it caused a few issues (some typos on SQL 
> Server and SQL Anywhere, as well as upgrade scenarios - however this has been 
> fixed).
> 
> 
> Diffs
> -
> 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 
> 9a79fe8ad 
>   
> security-admin/db/mysql/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
> fd6cec9a7 
>   
> security-admin/db/oracle/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
> 4d5a8cedf 
>   
> security-admin/db/postgres/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   
> security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
>  3ed2a5b9c 
>   
> security-admin/db/sqlanywhere/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
> ca8f7da1f 
>   
> security-admin/db/sqlserver/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 
> 6cc3509d8 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java
>  6af8f99f4 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java
>  4f4409d6a 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefDataMaskType.java
>  cb926740e 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java 
> 32a1b9f24 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefResource.java
>  115064621 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefRole.java 
> 7aee502e0 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefUser.java 
> 8dfb92833 
>   
> security-admin/src/main/java/org/apache/ranger/service/XPortalUserService.java
>  85e457efa 
>   security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java 
> 691ab52b3 
> 
> 
> Diff: https://reviews.apache.org/r/74301/diff/3/
> 
> 
> Testing
> ---
> 
> Fresh install and upgrade tested on all 5 databases (tested service and 
> policy creation)
> 
> Also tested user deletion on MySQL:
> 
> 1. Created new user
> 2. Logged in as new user, created a policy
> 3. Logged back in as original user, delete user that created policy
> 4. Verified no errors, policy is still present/correct
> 
> Also searched the source code and JPA queries xml for other references to 
> these fields
> 
> 
> Thanks,
> 
> Andrew Luo
> 
>

[jira] [Updated] (RANGER-2713) Remove audit fields from XXPolicyRef objects

[Andrew Luo (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Luo updated RANGER-2713:
---
Attachment: (was: 
0001-RANGER-2713-Remove-audit-fields-from-XXPolicyRef-obj.patch)

> Remove audit fields from XXPolicyRef objects
> 
>
> Key: RANGER-2713
> URL: https://issues.apache.org/jira/browse/RANGER-2713
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin, audit
>Affects Versions: master
>Reporter: Andrew Luo
>Priority: Minor
> Attachments: 
> 0001-RANGER-2713-Remove-audit-fields-from-XXPolicyRef-obj.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> XXPolicyRef objects have fields such as create time, update time, added by 
> user ID, updated by user ID, but there fields are entirely useless since they 
> are all copied from the XXPolicy object.  In addition, while improving 
> performance for creation of policies with large numbers of users, we 
> discovered that a lot of time was being spent in JPA converting these Date 
> objects especially.  After removing these fields we saw a significant 
> performance improvement (a secondary benefit is less database space usage).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-2713) Remove audit fields from XXPolicyRef objects

[Andrew Luo (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Luo updated RANGER-2713:
---
Attachment: 0001-RANGER-2713-Remove-audit-fields-from-XXPolicyRef-obj.patch

> Remove audit fields from XXPolicyRef objects
> 
>
> Key: RANGER-2713
> URL: https://issues.apache.org/jira/browse/RANGER-2713
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin, audit
>Affects Versions: master
>Reporter: Andrew Luo
>Priority: Minor
> Attachments: 
> 0001-RANGER-2713-Remove-audit-fields-from-XXPolicyRef-obj.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> XXPolicyRef objects have fields such as create time, update time, added by 
> user ID, updated by user ID, but there fields are entirely useless since they 
> are all copied from the XXPolicy object.  In addition, while improving 
> performance for creation of policies with large numbers of users, we 
> discovered that a lot of time was being spent in JPA converting these Date 
> objects especially.  After removing these fields we saw a significant 
> performance improvement (a secondary benefit is less database space usage).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74301: RANGER-2713: Remove audit fields from XXPolicyRef objects

[Andrew Luo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74301/
---

(Updated Feb. 8, 2023, 9:05 p.m.)


Review request for ranger and Pradeep Agrawal.


Bugs: RANGER-2713
https://issues.apache.org/jira/browse/RANGER-2713


Repository: ranger


Description
---

XXPolicyRef objects have fields such as create time, update time, added by user 
ID, updated by user ID, but there fields are entirely useless since they are 
all copied from the XXPolicy object. In addition, while improving performance 
for creation of policies with large numbers of users, we discovered that a lot 
of time was being spent in JPA converting these Date objects especially. After 
removing these fields we saw a significant performance improvement (a secondary 
benefit is less database space usage).

We previously tried this commit and it caused a few issues (some typos on SQL 
Server and SQL Anywhere, as well as upgrade scenarios - however this has been 
fixed).


Diffs (updated)
-

  security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 9a79fe8ad 
  
security-admin/db/mysql/patches/061-drop-audit-columns-from-policy-ref-tables.sql
 PRE-CREATION 
  security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
fd6cec9a7 
  
security-admin/db/oracle/patches/061-drop-audit-columns-from-policy-ref-tables.sql
 PRE-CREATION 
  security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
4d5a8cedf 
  
security-admin/db/postgres/patches/061-drop-audit-columns-from-policy-ref-tables.sql
 PRE-CREATION 
  
security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 
3ed2a5b9c 
  
security-admin/db/sqlanywhere/patches/061-drop-audit-columns-from-policy-ref-tables.sql
 PRE-CREATION 
  security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
ca8f7da1f 
  
security-admin/db/sqlserver/patches/061-drop-audit-columns-from-policy-ref-tables.sql
 PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 
6cc3509d8 
  
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java
 6af8f99f4 
  
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java 
4f4409d6a 
  
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefDataMaskType.java
 cb926740e 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java 
32a1b9f24 
  
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefResource.java 
115064621 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefRole.java 
7aee502e0 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefUser.java 
8dfb92833 
  
security-admin/src/main/java/org/apache/ranger/service/XPortalUserService.java 
85e457efa 
  security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java 
691ab52b3 


Diff: https://reviews.apache.org/r/74301/diff/3/

Changes: https://reviews.apache.org/r/74301/diff/2-3/


Testing
---

Fresh install and upgrade tested on all 5 databases (tested service and policy 
creation)

Also tested user deletion on MySQL:

1. Created new user
2. Logged in as new user, created a policy
3. Logged back in as original user, delete user that created policy
4. Verified no errors, policy is still present/correct

Also searched the source code and JPA queries xml for other references to these 
fields


Thanks,

Andrew Luo

[jira] [Updated] (RANGER-4076) Support Java 17 for build and runtime

[Andrew Luo (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Luo updated RANGER-4076:
---
Attachment: 0001-RANGER-4076-Support-Java-17-for-build-and-runtime.patch

> Support Java 17 for build and runtime
> -
>
> Key: RANGER-4076
> URL: https://issues.apache.org/jira/browse/RANGER-4076
> Project: Ranger
>  Issue Type: New Feature
>  Components: admin, build-infra
>Reporter: Andrew Luo
>Priority: Major
> Attachments: 
> 0001-RANGER-4076-Support-Java-17-for-build-and-runtime.patch
>
>
> Currently only Java 8 and 11 are supported.  Java 17 is a major LTS version 
> of Java and adding support would modernize our Java version support.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-4076) Support Java 17 for build and runtime

[Andrew Luo (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Luo updated RANGER-4076:
---
Attachment: (was: 
0001-RANGER-4076-Support-Java-17-for-build-and-runtime.patch)

> Support Java 17 for build and runtime
> -
>
> Key: RANGER-4076
> URL: https://issues.apache.org/jira/browse/RANGER-4076
> Project: Ranger
>  Issue Type: New Feature
>  Components: admin, build-infra
>Reporter: Andrew Luo
>Priority: Major
> Attachments: 
> 0001-RANGER-4076-Support-Java-17-for-build-and-runtime.patch
>
>
> Currently only Java 8 and 11 are supported.  Java 17 is a major LTS version 
> of Java and adding support would modernize our Java version support.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74302: RANGER-4076: Support Java 17 for build and runtime

[Andrew Luo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74302/
---

(Updated Feb. 8, 2023, 8:46 p.m.)


Review request for ranger.


Bugs: RANGER-4076
https://issues.apache.org/jira/browse/RANGER-4076


Repository: ranger


Description
---

This requires upgrading a few maven plugins, libraries, as well as changing 
from Nashorn to GraalJS.  This uses the latest version of GraalJS that supports 
JDK 8.

If preferred, I can split this into smaller commits.


Diffs (updated)
-

  .mvn/jvm.config PRE-CREATION 
  agents-common/pom.xml b753c1368 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/NashornScriptEngineCreator.java
 db620df92 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/ScriptEngineUtil.java 
8d76c1d81 
  hdfs-agent/pom.xml a23cdfea0 
  kms/pom.xml e97b993d7 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 7188b19b2 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java b6fc32950 
  plugin-nestedstructure/pom.xml 0e208f99c 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/RecordFilterJavaScript.java
 77767767c 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java
 9cb161b8d 
  pom.xml e402bcc5d 
  security-admin/pom.xml 2e5e21950 


Diff: https://reviews.apache.org/r/74302/diff/4/

Changes: https://reviews.apache.org/r/74302/diff/3-4/


Testing
---

Tested compile and run on both Java 8 and 17.


File Attachments


0001-RANGER-4076-Support-Java-17-for-build-and-runtime.patch
  
https://reviews.apache.org/media/uploaded/files/2023/02/08/d87b4c7c-e84c-4ddb-b059-51c98bf75772__0001-RANGER-4076-Support-Java-17-for-build-and-runtime.patch


Thanks,

Andrew Luo

Re: Review Request 74302: RANGER-4076: Support Java 17 for build and runtime

[Andrew Luo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74302/
---

(Updated Feb. 8, 2023, 8:43 p.m.)


Review request for ranger.


Bugs: RANGER-4076
https://issues.apache.org/jira/browse/RANGER-4076


Repository: ranger


Description
---

This requires upgrading a few maven plugins, libraries, as well as changing 
from Nashorn to GraalJS.  This uses the latest version of GraalJS that supports 
JDK 8.

If preferred, I can split this into smaller commits.


Diffs (updated)
-

  .mvn/jvm.config PRE-CREATION 
  agents-common/pom.xml b753c1368 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/NashornScriptEngineCreator.java
 db620df92 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/ScriptEngineUtil.java 
8d76c1d81 
  hdfs-agent/pom.xml a23cdfea0 
  kms/pom.xml e97b993d7 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 7188b19b2 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java b6fc32950 
  plugin-nestedstructure/pom.xml 0e208f99c 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/RecordFilterJavaScript.java
 77767767c 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java
 9cb161b8d 
  pom.xml e402bcc5d 
  security-admin/pom.xml 2e5e21950 


Diff: https://reviews.apache.org/r/74302/diff/3/

Changes: https://reviews.apache.org/r/74302/diff/2-3/


Testing
---

Tested compile and run on both Java 8 and 17.


File Attachments (updated)


0001-RANGER-4076-Support-Java-17-for-build-and-runtime.patch
  
https://reviews.apache.org/media/uploaded/files/2023/02/08/d87b4c7c-e84c-4ddb-b059-51c98bf75772__0001-RANGER-4076-Support-Java-17-for-build-and-runtime.patch


Thanks,

Andrew Luo

[jira] [Updated] (RANGER-4076) Support Java 17 for build and runtime

[Andrew Luo (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Luo updated RANGER-4076:
---
Attachment: 0001-RANGER-4076-Support-Java-17-for-build-and-runtime.patch

> Support Java 17 for build and runtime
> -
>
> Key: RANGER-4076
> URL: https://issues.apache.org/jira/browse/RANGER-4076
> Project: Ranger
>  Issue Type: New Feature
>  Components: admin, build-infra
>Reporter: Andrew Luo
>Priority: Major
> Attachments: 
> 0001-RANGER-4076-Support-Java-17-for-build-and-runtime.patch
>
>
> Currently only Java 8 and 11 are supported.  Java 17 is a major LTS version 
> of Java and adding support would modernize our Java version support.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-4076) Support Java 17 for build and runtime

[Andrew Luo (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Luo updated RANGER-4076:
---
Attachment: (was: 0001-JDK-17.patch)

> Support Java 17 for build and runtime
> -
>
> Key: RANGER-4076
> URL: https://issues.apache.org/jira/browse/RANGER-4076
> Project: Ranger
>  Issue Type: New Feature
>  Components: admin, build-infra
>Reporter: Andrew Luo
>Priority: Major
> Attachments: 
> 0001-RANGER-4076-Support-Java-17-for-build-and-runtime.patch
>
>
> Currently only Java 8 and 11 are supported.  Java 17 is a major LTS version 
> of Java and adding support would modernize our Java version support.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [ranger] kumaab opened a new pull request, #214: RANGER-4074: Fix x_user_module_perm entry creation with NULL module Id

[via GitHub]

kumaab opened a new pull request, #214:
URL: https://github.com/apache/ranger/pull/214

   ## What changes were proposed in this pull request?
   
   assignPermissionToUser in XUserMgr assigns permissions for a module (which 
does not exist) to users with Auditor role. The PR fixes the bug.
   
   ## How was this patch tested?
   
   Testing underway


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Updated] (RANGER-3828) Fine-grained Access Control over nested structures

[Eckman, Barbara (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eckman, Barbara updated RANGER-3828:

Attachment: image001.png
image002.png
image003.png

Does this mean 2.4.0, but not 3.0.0?

--
Barbara Eckman, Ph.D.
she/her/hers
Distinguished Architect
Enterprise Metadata, Lineage and Access Control
Comcast
 [Logo  Description automatically generated] [Green badge indicating the 
completion of the Inclusive Development Program.]
[signature_951927206]



From: Madhan Neethiraj (Jira) 
Date: Tuesday, February 7, 2023 at 8:16 PM
To: Eckman, Barbara 
Subject: [EXTERNAL] [jira] [Updated] (RANGER-3828) Fine-grained Access Control 
over nested structures

 [ 
https://urldefense.com/v3/__https://issues.apache.org/jira/browse/RANGER-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel__;!!CQl3mcHX2A!Cm-QMTcEmoAuWqZlC-ngcYLcXat5clPBOuc1jfNVSitvAu98U7HNUrRznqK_FBERM7v6zl7Ai6-ya50jcQ0$
  ]

Madhan Neethiraj updated RANGER-3828:
-
Fix Version/s: 2.4.0




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


> Fine-grained Access Control over nested structures
> --
>
> Key: RANGER-3828
> URL: https://issues.apache.org/jira/browse/RANGER-3828
> Project: Ranger
>  Issue Type: New Feature
>  Components: plugins, Ranger
>Reporter: Barbara Eckman
>Assignee: Barbara Eckman
>Priority: Major
> Fix For: 3.0.0, 2.4.0
>
> Attachments: image001.png, image002.png, image003.png
>
>
> It would be nice to be able to do fine-grained access control (FGA) over 
> nested structures, e.g., the JSON responses of API calls.  This requires the 
> individual attributes in a JSON object to be first-class metadata objects 
> which can be tagged and on which policies can be written.  We have built a 
> plugin and the corresponding Apache Atlas metadata structures and 
> tagsync-mapper to support TBAC/RBAC/ABAC FGA over JSON structures.   Our 
> instigating use case was FGA over the JSON responses of API calls, but this 
> plugin has potential value anywhere FGA over the individual attributes of 
> nested structures is needed, eg JSON messages read from Kafka topics.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74301: RANGER-2713: Remove audit fields from XXPolicyRef objects

[Andrew Luo]

> On Feb. 8, 2023, 4:54 p.m., Madhan Neethiraj wrote:
> > @Andrew - thank you for this patch to improve performance of policy 
> > changes. Changes in the patch look good.
> > 
> > I have few suggestions:
> > 
> > 1. Column named guid in ref tables is not mapped to any field in respective 
> > Java entity class. It might help to remove this unused column.
> > 2. DB patch to drop constriants look good. However, removing columns from 
> > tables would require following sequence of steps - to avoid earlier version 
> > Ranger running into errors while referring to these columns:
> > - stop Ranger server instances
> > - apply the DB patch to remove columns
> > - update Ranger admin server to newer version containing this patch
> > - start Ranger server instances
> > 
> > This will result in Ranger being not available for the duration of above 
> > steps. This will not be acceptable in deployments having Ranger in high 
> > availablity configuration.
> > 
> > I suggest not to remove columns that are used by current version of Ranger. 
> > Note that these columns can be removed from scripts that create Ranger 
> > schema in a new deployment (scripts under directory named optimized).
> 
> Andrew Luo wrote:
> I see.  What I could recommend here is that we can not remove the columns 
> during the upgrade, and revisit this a few months/years later when we're 
> comfortable.
> 
> Madhan Neethiraj wrote:
> yes, that sounds good. These columns can be removed in next major 
> release, 3.0 i.e., master branch.
> 
> Assuming this patch makes to 2.4 release, upgrade from 2.4 to 3.0 will 
> not require restart of Ranger - since Ranger 2.4 will not be referring to 
> columns deleted in 3.0.

Okay.  I'll comment out the SQL scripts - but I think I need to reserve that 
patch number so that when we eventually put it in 3.0, we have an indication 
that the "optimized" schema already has this patch.  Thanks for the suggestion!


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74301/#review225166
---


On Feb. 8, 2023, 4:09 p.m., Andrew Luo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74301/
> ---
> 
> (Updated Feb. 8, 2023, 4:09 p.m.)
> 
> 
> Review request for ranger and Pradeep Agrawal.
> 
> 
> Bugs: RANGER-2713
> https://issues.apache.org/jira/browse/RANGER-2713
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> XXPolicyRef objects have fields such as create time, update time, added by 
> user ID, updated by user ID, but there fields are entirely useless since they 
> are all copied from the XXPolicy object. In addition, while improving 
> performance for creation of policies with large numbers of users, we 
> discovered that a lot of time was being spent in JPA converting these Date 
> objects especially. After removing these fields we saw a significant 
> performance improvement (a secondary benefit is less database space usage).
> 
> We previously tried this commit and it caused a few issues (some typos on SQL 
> Server and SQL Anywhere, as well as upgrade scenarios - however this has been 
> fixed).
> 
> 
> Diffs
> -
> 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 
> 9a79fe8ad 
>   
> security-admin/db/mysql/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
> fd6cec9a7 
>   
> security-admin/db/oracle/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
> 4d5a8cedf 
>   
> security-admin/db/postgres/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   
> security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
>  3ed2a5b9c 
>   
> security-admin/db/sqlanywhere/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
> ca8f7da1f 
>   
> security-admin/db/sqlserver/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 
> 6cc3509d8 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java
>  6af8f99f4 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java
>  4f4409d6a 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefDataMaskType.java
>  cb926740e 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java 
> 32a1b9f24 
>   
> 

Re: Review Request 74301: RANGER-2713: Remove audit fields from XXPolicyRef objects

[Madhan Neethiraj]

> On Feb. 8, 2023, 4:54 p.m., Madhan Neethiraj wrote:
> > @Andrew - thank you for this patch to improve performance of policy 
> > changes. Changes in the patch look good.
> > 
> > I have few suggestions:
> > 
> > 1. Column named guid in ref tables is not mapped to any field in respective 
> > Java entity class. It might help to remove this unused column.
> > 2. DB patch to drop constriants look good. However, removing columns from 
> > tables would require following sequence of steps - to avoid earlier version 
> > Ranger running into errors while referring to these columns:
> > - stop Ranger server instances
> > - apply the DB patch to remove columns
> > - update Ranger admin server to newer version containing this patch
> > - start Ranger server instances
> > 
> > This will result in Ranger being not available for the duration of above 
> > steps. This will not be acceptable in deployments having Ranger in high 
> > availablity configuration.
> > 
> > I suggest not to remove columns that are used by current version of Ranger. 
> > Note that these columns can be removed from scripts that create Ranger 
> > schema in a new deployment (scripts under directory named optimized).
> 
> Andrew Luo wrote:
> I see.  What I could recommend here is that we can not remove the columns 
> during the upgrade, and revisit this a few months/years later when we're 
> comfortable.

yes, that sounds good. These columns can be removed in next major release, 3.0 
i.e., master branch.

Assuming this patch makes to 2.4 release, upgrade from 2.4 to 3.0 will not 
require restart of Ranger - since Ranger 2.4 will not be referring to columns 
deleted in 3.0.


- Madhan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74301/#review225166
---


On Feb. 8, 2023, 4:09 p.m., Andrew Luo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74301/
> ---
> 
> (Updated Feb. 8, 2023, 4:09 p.m.)
> 
> 
> Review request for ranger and Pradeep Agrawal.
> 
> 
> Bugs: RANGER-2713
> https://issues.apache.org/jira/browse/RANGER-2713
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> XXPolicyRef objects have fields such as create time, update time, added by 
> user ID, updated by user ID, but there fields are entirely useless since they 
> are all copied from the XXPolicy object. In addition, while improving 
> performance for creation of policies with large numbers of users, we 
> discovered that a lot of time was being spent in JPA converting these Date 
> objects especially. After removing these fields we saw a significant 
> performance improvement (a secondary benefit is less database space usage).
> 
> We previously tried this commit and it caused a few issues (some typos on SQL 
> Server and SQL Anywhere, as well as upgrade scenarios - however this has been 
> fixed).
> 
> 
> Diffs
> -
> 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 
> 9a79fe8ad 
>   
> security-admin/db/mysql/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
> fd6cec9a7 
>   
> security-admin/db/oracle/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
> 4d5a8cedf 
>   
> security-admin/db/postgres/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   
> security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
>  3ed2a5b9c 
>   
> security-admin/db/sqlanywhere/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
> ca8f7da1f 
>   
> security-admin/db/sqlserver/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 
> 6cc3509d8 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java
>  6af8f99f4 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java
>  4f4409d6a 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefDataMaskType.java
>  cb926740e 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java 
> 32a1b9f24 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefResource.java
>  115064621 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefRole.java 
> 7aee502e0 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefUser.java 
> 8dfb92833 
>   
> 

Re: Review Request 74301: RANGER-2713: Remove audit fields from XXPolicyRef objects

[Andrew Luo]

> On Feb. 8, 2023, 4:54 p.m., Madhan Neethiraj wrote:
> > @Andrew - thank you for this patch to improve performance of policy 
> > changes. Changes in the patch look good.
> > 
> > I have few suggestions:
> > 
> > 1. Column named guid in ref tables is not mapped to any field in respective 
> > Java entity class. It might help to remove this unused column.
> > 2. DB patch to drop constriants look good. However, removing columns from 
> > tables would require following sequence of steps - to avoid earlier version 
> > Ranger running into errors while referring to these columns:
> > - stop Ranger server instances
> > - apply the DB patch to remove columns
> > - update Ranger admin server to newer version containing this patch
> > - start Ranger server instances
> > 
> > This will result in Ranger being not available for the duration of above 
> > steps. This will not be acceptable in deployments having Ranger in high 
> > availablity configuration.
> > 
> > I suggest not to remove columns that are used by current version of Ranger. 
> > Note that these columns can be removed from scripts that create Ranger 
> > schema in a new deployment (scripts under directory named optimized).

I see.  What I could recommend here is that we can not remove the columns 
during the upgrade, and revisit this a few months/years later when we're 
comfortable.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74301/#review225166
---


On Feb. 8, 2023, 4:09 p.m., Andrew Luo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74301/
> ---
> 
> (Updated Feb. 8, 2023, 4:09 p.m.)
> 
> 
> Review request for ranger and Pradeep Agrawal.
> 
> 
> Bugs: RANGER-2713
> https://issues.apache.org/jira/browse/RANGER-2713
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> XXPolicyRef objects have fields such as create time, update time, added by 
> user ID, updated by user ID, but there fields are entirely useless since they 
> are all copied from the XXPolicy object. In addition, while improving 
> performance for creation of policies with large numbers of users, we 
> discovered that a lot of time was being spent in JPA converting these Date 
> objects especially. After removing these fields we saw a significant 
> performance improvement (a secondary benefit is less database space usage).
> 
> We previously tried this commit and it caused a few issues (some typos on SQL 
> Server and SQL Anywhere, as well as upgrade scenarios - however this has been 
> fixed).
> 
> 
> Diffs
> -
> 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 
> 9a79fe8ad 
>   
> security-admin/db/mysql/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
> fd6cec9a7 
>   
> security-admin/db/oracle/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
> 4d5a8cedf 
>   
> security-admin/db/postgres/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   
> security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
>  3ed2a5b9c 
>   
> security-admin/db/sqlanywhere/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
> ca8f7da1f 
>   
> security-admin/db/sqlserver/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 
> 6cc3509d8 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java
>  6af8f99f4 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java
>  4f4409d6a 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefDataMaskType.java
>  cb926740e 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java 
> 32a1b9f24 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefResource.java
>  115064621 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefRole.java 
> 7aee502e0 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefUser.java 
> 8dfb92833 
>   
> security-admin/src/main/java/org/apache/ranger/service/XPortalUserService.java
>  85e457efa 
>   security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java 
> 691ab52b3 
> 
> 
> Diff: https://reviews.apache.org/r/74301/diff/2/
> 
> 
> Testing
> ---
> 
> Fresh install and upgrade tested on all 5 databases (tested service and 
> 

Re: Review Request 74245: RANGER-3899:Policy creation takes more time when there are more users, groups, roles

[Andrew Luo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74245/#review225167
---



I'm not a committer but I was the author of the original PR a few years back - 
happy to see this come back and hope it can be merged soon!

- Andrew Luo


On Jan. 4, 2023, 7:14 a.m., Ramachandran Krishnan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74245/
> ---
> 
> (Updated Jan. 4, 2023, 7:14 a.m.)
> 
> 
> Review request for ranger, Don Bosco Durai, Kirby Zhou, Abhay Kulkarni, 
> Madhan Neethiraj, Mehul Parikh, Nikhil P, Pradeep Agrawal, Ramesh Mani, 
> Selvamohan Neethiraj, Sailaja Polavarapu, Subhrat Chaudhary, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: RANGER-3899
> https://issues.apache.org/jira/browse/RANGER-3899
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Analysis is when policy has 500 users with 5 policy items.
> 
> we perform total 526 DB read calls and 504 DB write, 3 DB Batch write calls. 
> This is the reason for policy creation time where 1000 plus DB calls are done 
> sequentially.
> Time is taken to create the policy :4~8 seconds 
> 
> 1st#Improvement:
> 
> After Fix :After changing to batch create of XXPolicyRefUser  instead of 
> calling 1 DB write XXPolicyRefUser for every user of the policy:
> 
> When Policy contains 500 users,5 access Types (permissions),12 resources 
> (1DB,1Table,10 columns)
> 
> we perform total 526 DB read calls and 6 DB write calls, 5 DB Batch write 
> calls.
> 
> Time is taken to create the policy :2.6~4 seconds
> 
> 2nd#Improvement:
> Getting the ID for each role/group/user one-by-one is slow, especially for 
> large policies with many roles/groups/users.  Batching significantly improves 
> performance.
> 
> Fixing this part will reduce the DB read calls to a very minimum which will 
> improve performance drastically.
> 
> The policy contains 500 users,5 access Types (permissions),12 resources 
> (1DB,1Table,10 columns)
> 
> After fix:
> DB Read count:27
> 
> Time taken to create the policy: 1.4 to 2 secs (The policy contains 500 
> users,5 access Types (permissions),12 resources (1DB,1Table,10 columns))
> 
> DB read calls will be reduced to 27 from 526
> 
> Time is taken to create the policy :1.4~2 seconds
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 
> 6cc3509d8 
>   security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java 8ca4f62bb 
>   security-admin/src/main/java/org/apache/ranger/db/XXRoleDao.java 4e5b692b5 
>   security-admin/src/main/java/org/apache/ranger/db/XXUserDao.java 97bc2680a 
>   security-admin/src/main/resources/META-INF/jpa_named_queries.xml 85c8b6213 
> 
> 
> Diff: https://reviews.apache.org/r/74245/diff/2/
> 
> 
> Testing
> ---
> 
> Tested Created policies via load testing
> 
> 
> Thanks,
> 
> Ramachandran Krishnan
> 
>

Re: Review Request 74301: RANGER-2713: Remove audit fields from XXPolicyRef objects

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74301/#review225166
---



@Andrew - thank you for this patch to improve performance of policy changes. 
Changes in the patch look good.

I have few suggestions:

1. Column named guid in ref tables is not mapped to any field in respective 
Java entity class. It might help to remove this unused column.
2. DB patch to drop constriants look good. However, removing columns from 
tables would require following sequence of steps - to avoid earlier version 
Ranger running into errors while referring to these columns:
- stop Ranger server instances
- apply the DB patch to remove columns
- update Ranger admin server to newer version containing this patch
- start Ranger server instances

This will result in Ranger being not available for the duration of above steps. 
This will not be acceptable in deployments having Ranger in high availablity 
configuration.

I suggest not to remove columns that are used by current version of Ranger. 
Note that these columns can be removed from scripts that create Ranger schema 
in a new deployment (scripts under directory named optimized).

- Madhan Neethiraj


On Feb. 8, 2023, 4:09 p.m., Andrew Luo wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74301/
> ---
> 
> (Updated Feb. 8, 2023, 4:09 p.m.)
> 
> 
> Review request for ranger and Pradeep Agrawal.
> 
> 
> Bugs: RANGER-2713
> https://issues.apache.org/jira/browse/RANGER-2713
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> XXPolicyRef objects have fields such as create time, update time, added by 
> user ID, updated by user ID, but there fields are entirely useless since they 
> are all copied from the XXPolicy object. In addition, while improving 
> performance for creation of policies with large numbers of users, we 
> discovered that a lot of time was being spent in JPA converting these Date 
> objects especially. After removing these fields we saw a significant 
> performance improvement (a secondary benefit is less database space usage).
> 
> We previously tried this commit and it caused a few issues (some typos on SQL 
> Server and SQL Anywhere, as well as upgrade scenarios - however this has been 
> fixed).
> 
> 
> Diffs
> -
> 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 
> 9a79fe8ad 
>   
> security-admin/db/mysql/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
> fd6cec9a7 
>   
> security-admin/db/oracle/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
> 4d5a8cedf 
>   
> security-admin/db/postgres/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   
> security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
>  3ed2a5b9c 
>   
> security-admin/db/sqlanywhere/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
> ca8f7da1f 
>   
> security-admin/db/sqlserver/patches/061-drop-audit-columns-from-policy-ref-tables.sql
>  PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 
> 6cc3509d8 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java
>  6af8f99f4 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java
>  4f4409d6a 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefDataMaskType.java
>  cb926740e 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java 
> 32a1b9f24 
>   
> security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefResource.java
>  115064621 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefRole.java 
> 7aee502e0 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefUser.java 
> 8dfb92833 
>   
> security-admin/src/main/java/org/apache/ranger/service/XPortalUserService.java
>  85e457efa 
>   security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java 
> 691ab52b3 
> 
> 
> Diff: https://reviews.apache.org/r/74301/diff/2/
> 
> 
> Testing
> ---
> 
> Fresh install and upgrade tested on all 5 databases (tested service and 
> policy creation)
> 
> Also tested user deletion on MySQL:
> 
> 1. Created new user
> 2. Logged in as new user, created a policy
> 3. Logged back in as original user, delete user that created policy
> 4. Verified no errors, policy is still present/correct
> 
> Also searched the source code and JPA 

[jira] [Updated] (RANGER-4076) Support Java 17 for build and runtime

[Andrew Luo (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Luo updated RANGER-4076:
---
Attachment: (was: 0001-JDK-17.patch)

> Support Java 17 for build and runtime
> -
>
> Key: RANGER-4076
> URL: https://issues.apache.org/jira/browse/RANGER-4076
> Project: Ranger
>  Issue Type: New Feature
>  Components: admin, build-infra
>Reporter: Andrew Luo
>Priority: Major
> Attachments: 0001-JDK-17.patch
>
>
> Currently only Java 8 and 11 are supported.  Java 17 is a major LTS version 
> of Java and adding support would modernize our Java version support.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-4076) Support Java 17 for build and runtime

[Andrew Luo (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Luo updated RANGER-4076:
---
Attachment: 0001-JDK-17.patch

> Support Java 17 for build and runtime
> -
>
> Key: RANGER-4076
> URL: https://issues.apache.org/jira/browse/RANGER-4076
> Project: Ranger
>  Issue Type: New Feature
>  Components: admin, build-infra
>Reporter: Andrew Luo
>Priority: Major
> Attachments: 0001-JDK-17.patch
>
>
> Currently only Java 8 and 11 are supported.  Java 17 is a major LTS version 
> of Java and adding support would modernize our Java version support.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74302: RANGER-4076: Support Java 17 for build and runtime

[Andrew Luo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74302/
---

(Updated Feb. 8, 2023, 4:49 p.m.)


Review request for ranger.


Bugs: RANGER-4076
https://issues.apache.org/jira/browse/RANGER-4076


Repository: ranger


Description
---

This requires upgrading a few maven plugins, libraries, as well as changing 
from Nashorn to GraalJS.  This uses the latest version of GraalJS that supports 
JDK 8.

If preferred, I can split this into smaller commits.


Diffs (updated)
-

  .mvn/jvm.config PRE-CREATION 
  agents-common/pom.xml b753c1368 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/NashornScriptEngineCreator.java
 db620df92 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/ScriptEngineUtil.java 
8d76c1d81 
  hdfs-agent/pom.xml a23cdfea0 
  kms/pom.xml e97b993d7 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 7188b19b2 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java b6fc32950 
  plugin-nestedstructure/pom.xml 0e208f99c 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/RecordFilterJavaScript.java
 77767767c 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java
 9cb161b8d 
  pom.xml e402bcc5d 
  security-admin/pom.xml 2e5e21950 


Diff: https://reviews.apache.org/r/74302/diff/2/

Changes: https://reviews.apache.org/r/74302/diff/1-2/


Testing
---

Tested compile and run on both Java 8 and 17.


Thanks,

Andrew Luo

[jira] [Updated] (RANGER-4076) Support Java 17 for build and runtime

[Andrew Luo (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Luo updated RANGER-4076:
---
Attachment: (was: 0001-JDK-17.patch)

> Support Java 17 for build and runtime
> -
>
> Key: RANGER-4076
> URL: https://issues.apache.org/jira/browse/RANGER-4076
> Project: Ranger
>  Issue Type: New Feature
>  Components: admin, build-infra
>Reporter: Andrew Luo
>Priority: Major
> Attachments: 0001-JDK-17.patch
>
>
> Currently only Java 8 and 11 are supported.  Java 17 is a major LTS version 
> of Java and adding support would modernize our Java version support.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-4076) Support Java 17 for build and runtime

[Andrew Luo (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Luo updated RANGER-4076:
---
Attachment: 0001-JDK-17.patch

> Support Java 17 for build and runtime
> -
>
> Key: RANGER-4076
> URL: https://issues.apache.org/jira/browse/RANGER-4076
> Project: Ranger
>  Issue Type: New Feature
>  Components: admin, build-infra
>Reporter: Andrew Luo
>Priority: Major
> Attachments: 0001-JDK-17.patch
>
>
> Currently only Java 8 and 11 are supported.  Java 17 is a major LTS version 
> of Java and adding support would modernize our Java version support.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-4076) Support Java 17 for build and runtime

[Andrew Luo (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Luo updated RANGER-4076:
---
Attachment: 0001-JDK-17.patch

> Support Java 17 for build and runtime
> -
>
> Key: RANGER-4076
> URL: https://issues.apache.org/jira/browse/RANGER-4076
> Project: Ranger
>  Issue Type: New Feature
>  Components: admin, build-infra
>Reporter: Andrew Luo
>Priority: Major
> Attachments: 0001-JDK-17.patch
>
>
> Currently only Java 8 and 11 are supported.  Java 17 is a major LTS version 
> of Java and adding support would modernize our Java version support.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-4076) Support Java 17 for build and runtime

[Andrew Luo (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Luo updated RANGER-4076:
---
Attachment: (was: 0001-JDK-17.patch)

> Support Java 17 for build and runtime
> -
>
> Key: RANGER-4076
> URL: https://issues.apache.org/jira/browse/RANGER-4076
> Project: Ranger
>  Issue Type: New Feature
>  Components: admin, build-infra
>Reporter: Andrew Luo
>Priority: Major
> Attachments: 0001-JDK-17.patch
>
>
> Currently only Java 8 and 11 are supported.  Java 17 is a major LTS version 
> of Java and adding support would modernize our Java version support.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-4076) Support Java 17 for build and runtime

[Andrew Luo (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Luo updated RANGER-4076:
---
Attachment: 0001-JDK-17.patch

> Support Java 17 for build and runtime
> -
>
> Key: RANGER-4076
> URL: https://issues.apache.org/jira/browse/RANGER-4076
> Project: Ranger
>  Issue Type: New Feature
>  Components: admin, build-infra
>Reporter: Andrew Luo
>Priority: Major
> Attachments: 0001-JDK-17.patch
>
>
> Currently only Java 8 and 11 are supported.  Java 17 is a major LTS version 
> of Java and adding support would modernize our Java version support.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (RANGER-4076) Support Java 17 for build and runtime

[Andrew Luo (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685979#comment-17685979
 ] 

Andrew Luo commented on RANGER-4076:


[https://reviews.apache.org/r/74302/]

[https://github.com/apache/ranger/pull/212]

 

> Support Java 17 for build and runtime
> -
>
> Key: RANGER-4076
> URL: https://issues.apache.org/jira/browse/RANGER-4076
> Project: Ranger
>  Issue Type: New Feature
>  Components: admin, build-infra
>Reporter: Andrew Luo
>Priority: Major
> Attachments: 0001-JDK-17.patch
>
>
> Currently only Java 8 and 11 are supported.  Java 17 is a major LTS version 
> of Java and adding support would modernize our Java version support.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74302: RANGER-4076: Support Java 17 for build and runtime

[Andrew Luo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74302/
---

(Updated Feb. 8, 2023, 4:46 p.m.)


Review request for ranger.


Bugs: RANGER-4076
https://issues.apache.org/jira/browse/RANGER-4076


Repository: ranger


Description (updated)
---

This requires upgrading a few maven plugins, libraries, as well as changing 
from Nashorn to GraalJS.  This uses the latest version of GraalJS that supports 
JDK 8.

If preferred, I can split this into smaller commits.


Diffs
-

  .mvn/jvm.config PRE-CREATION 
  agents-common/pom.xml b753c1368 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/NashornScriptEngineCreator.java
 db620df92 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/ScriptEngineUtil.java 
8d76c1d81 
  hdfs-agent/pom.xml a23cdfea0 
  kms/pom.xml e97b993d7 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 7188b19b2 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java b6fc32950 
  plugin-nestedstructure/pom.xml 0e208f99c 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/RecordFilterJavaScript.java
 77767767c 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java
 9cb161b8d 
  pom.xml e402bcc5d 
  
ranger-trino-plugin-shim/src/main/java/org/apache/ranger/authorization/trino/authorizer/TrinoRangerPlugin.java
 aa6ce8620 
  security-admin/pom.xml 2e5e21950 


Diff: https://reviews.apache.org/r/74302/diff/1/


Testing
---

Tested compile and run on both Java 8 and 17.


Thanks,

Andrew Luo

Review Request 74302: RANGER-4076: Support Java 17 for build and runtime

[Andrew Luo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74302/
---

Review request for ranger.


Bugs: RANGER-4076
https://issues.apache.org/jira/browse/RANGER-4076


Repository: ranger


Description
---

This requires upgrading a few maven plugins, libraries, as well as changing 
from Nashorn to GraalJS.  This uses the latest version of GraalJS that supports 
JDK 8.


Diffs
-

  .mvn/jvm.config PRE-CREATION 
  agents-common/pom.xml b753c1368 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/NashornScriptEngineCreator.java
 db620df92 
  
agents-common/src/main/java/org/apache/ranger/plugin/util/ScriptEngineUtil.java 
8d76c1d81 
  hdfs-agent/pom.xml a23cdfea0 
  kms/pom.xml e97b993d7 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStore.java 7188b19b2 
  kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java b6fc32950 
  plugin-nestedstructure/pom.xml 0e208f99c 
  
plugin-nestedstructure/src/main/java/org/apache/ranger/authorization/nestedstructure/authorizer/RecordFilterJavaScript.java
 77767767c 
  
plugin-nestedstructure/src/test/java/org/apache/ranger/authorization/nestedstructure/authorizer/TestRecordFilterJavaScript.java
 9cb161b8d 
  pom.xml e402bcc5d 
  
ranger-trino-plugin-shim/src/main/java/org/apache/ranger/authorization/trino/authorizer/TrinoRangerPlugin.java
 aa6ce8620 
  security-admin/pom.xml 2e5e21950 


Diff: https://reviews.apache.org/r/74302/diff/1/


Testing
---

Tested compile and run on both Java 8 and 17.


Thanks,

Andrew Luo

[jira] [Created] (RANGER-4076) Support Java 17 for build and runtime

[Andrew Luo (Jira)]

Andrew Luo created RANGER-4076:
--

 Summary: Support Java 17 for build and runtime
 Key: RANGER-4076
 URL: https://issues.apache.org/jira/browse/RANGER-4076
 Project: Ranger
  Issue Type: New Feature
  Components: admin, build-infra
Reporter: Andrew Luo


Currently only Java 8 and 11 are supported.  Java 17 is a major LTS version of 
Java and adding support would modernize our Java version support.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (RANGER-2713) Remove audit fields from XXPolicyRef objects

[Andrew Luo (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685975#comment-17685975
 ] 

Andrew Luo commented on RANGER-2713:


https://github.com/apache/ranger/pull/211

> Remove audit fields from XXPolicyRef objects
> 
>
> Key: RANGER-2713
> URL: https://issues.apache.org/jira/browse/RANGER-2713
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin, audit
>Affects Versions: master
>Reporter: Andrew Luo
>Priority: Minor
> Attachments: 
> 0001-RANGER-2713-Remove-audit-fields-from-XXPolicyRef-obj.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> XXPolicyRef objects have fields such as create time, update time, added by 
> user ID, updated by user ID, but there fields are entirely useless since they 
> are all copied from the XXPolicy object.  In addition, while improving 
> performance for creation of policies with large numbers of users, we 
> discovered that a lot of time was being spent in JPA converting these Date 
> objects especially.  After removing these fields we saw a significant 
> performance improvement (a secondary benefit is less database space usage).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [ranger] andrewluotechnologies opened a new pull request, #211: RANGER-2713: Remove audit fields from XXPolicyRef objects

[via GitHub]

andrewluotechnologies opened a new pull request, #211:
URL: https://github.com/apache/ranger/pull/211

   XXPolicyRef objects have fields such as create time, update time, added by 
user ID, updated by user ID, but there fields are entirely useless since they 
are all copied from the XXPolicy object. In addition, while improving 
performance for creation of policies with large numbers of users, we discovered 
that a lot of time was being spent in JPA converting these Date objects 
especially. After removing these fields we saw a significant performance 
improvement (a secondary benefit is less database space usage).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (RANGER-2713) Remove audit fields from XXPolicyRef objects

[Andrew Luo (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685971#comment-17685971
 ] 

Andrew Luo commented on RANGER-2713:


https://reviews.apache.org/r/74301/

> Remove audit fields from XXPolicyRef objects
> 
>
> Key: RANGER-2713
> URL: https://issues.apache.org/jira/browse/RANGER-2713
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin, audit
>Affects Versions: master
>Reporter: Andrew Luo
>Priority: Minor
> Attachments: 
> 0001-RANGER-2713-Remove-audit-fields-from-XXPolicyRef-obj.patch
>
>
> XXPolicyRef objects have fields such as create time, update time, added by 
> user ID, updated by user ID, but there fields are entirely useless since they 
> are all copied from the XXPolicy object.  In addition, while improving 
> performance for creation of policies with large numbers of users, we 
> discovered that a lot of time was being spent in JPA converting these Date 
> objects especially.  After removing these fields we saw a significant 
> performance improvement (a secondary benefit is less database space usage).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-2713) Remove audit fields from XXPolicyRef objects

[Andrew Luo (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Luo updated RANGER-2713:
---
Attachment: 0001-RANGER-2713-Remove-audit-fields-from-XXPolicyRef-obj.patch

> Remove audit fields from XXPolicyRef objects
> 
>
> Key: RANGER-2713
> URL: https://issues.apache.org/jira/browse/RANGER-2713
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin, audit
>Affects Versions: master
>Reporter: Andrew Luo
>Priority: Minor
> Attachments: 
> 0001-RANGER-2713-Remove-audit-fields-from-XXPolicyRef-obj.patch
>
>
> XXPolicyRef objects have fields such as create time, update time, added by 
> user ID, updated by user ID, but there fields are entirely useless since they 
> are all copied from the XXPolicy object.  In addition, while improving 
> performance for creation of policies with large numbers of users, we 
> discovered that a lot of time was being spent in JPA converting these Date 
> objects especially.  After removing these fields we saw a significant 
> performance improvement (a secondary benefit is less database space usage).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-2713) Remove audit fields from XXPolicyRef objects

[Andrew Luo (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Luo updated RANGER-2713:
---
Attachment: (was: 
0001-RANGER-2713-Remove-audit-fields-from-XXPolicyRef-obj.patch)

> Remove audit fields from XXPolicyRef objects
> 
>
> Key: RANGER-2713
> URL: https://issues.apache.org/jira/browse/RANGER-2713
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin, audit
>Affects Versions: master
>Reporter: Andrew Luo
>Priority: Minor
> Attachments: 
> 0001-RANGER-2713-Remove-audit-fields-from-XXPolicyRef-obj.patch
>
>
> XXPolicyRef objects have fields such as create time, update time, added by 
> user ID, updated by user ID, but there fields are entirely useless since they 
> are all copied from the XXPolicy object.  In addition, while improving 
> performance for creation of policies with large numbers of users, we 
> discovered that a lot of time was being spent in JPA converting these Date 
> objects especially.  After removing these fields we saw a significant 
> performance improvement (a secondary benefit is less database space usage).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-2713) Remove audit fields from XXPolicyRef objects

[Andrew Luo (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Luo updated RANGER-2713:
---
Attachment: (was: 
0001-RANGER-2713-Remove-audit-fields-from-XXPolicyRef-obj.patch)

> Remove audit fields from XXPolicyRef objects
> 
>
> Key: RANGER-2713
> URL: https://issues.apache.org/jira/browse/RANGER-2713
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin, audit
>Affects Versions: master
>Reporter: Andrew Luo
>Priority: Minor
> Attachments: 
> 0001-RANGER-2713-Remove-audit-fields-from-XXPolicyRef-obj.patch
>
>
> XXPolicyRef objects have fields such as create time, update time, added by 
> user ID, updated by user ID, but there fields are entirely useless since they 
> are all copied from the XXPolicy object.  In addition, while improving 
> performance for creation of policies with large numbers of users, we 
> discovered that a lot of time was being spent in JPA converting these Date 
> objects especially.  After removing these fields we saw a significant 
> performance improvement (a secondary benefit is less database space usage).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74301: RANGER-2713: Remove audit fields from XXPolicyRef objects

[Andrew Luo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74301/
---

(Updated Feb. 8, 2023, 4:09 p.m.)


Review request for ranger and Pradeep Agrawal.


Repository: ranger


Description
---

XXPolicyRef objects have fields such as create time, update time, added by user 
ID, updated by user ID, but there fields are entirely useless since they are 
all copied from the XXPolicy object. In addition, while improving performance 
for creation of policies with large numbers of users, we discovered that a lot 
of time was being spent in JPA converting these Date objects especially. After 
removing these fields we saw a significant performance improvement (a secondary 
benefit is less database space usage).

We previously tried this commit and it caused a few issues (some typos on SQL 
Server and SQL Anywhere, as well as upgrade scenarios - however this has been 
fixed).


Diffs (updated)
-

  security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 9a79fe8ad 
  
security-admin/db/mysql/patches/061-drop-audit-columns-from-policy-ref-tables.sql
 PRE-CREATION 
  security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
fd6cec9a7 
  
security-admin/db/oracle/patches/061-drop-audit-columns-from-policy-ref-tables.sql
 PRE-CREATION 
  security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
4d5a8cedf 
  
security-admin/db/postgres/patches/061-drop-audit-columns-from-policy-ref-tables.sql
 PRE-CREATION 
  
security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 
3ed2a5b9c 
  
security-admin/db/sqlanywhere/patches/061-drop-audit-columns-from-policy-ref-tables.sql
 PRE-CREATION 
  security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
ca8f7da1f 
  
security-admin/db/sqlserver/patches/061-drop-audit-columns-from-policy-ref-tables.sql
 PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 
6cc3509d8 
  
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java
 6af8f99f4 
  
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java 
4f4409d6a 
  
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefDataMaskType.java
 cb926740e 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java 
32a1b9f24 
  
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefResource.java 
115064621 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefRole.java 
7aee502e0 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefUser.java 
8dfb92833 
  
security-admin/src/main/java/org/apache/ranger/service/XPortalUserService.java 
85e457efa 
  security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java 
691ab52b3 


Diff: https://reviews.apache.org/r/74301/diff/2/

Changes: https://reviews.apache.org/r/74301/diff/1-2/


Testing
---

Fresh install and upgrade tested on all 5 databases (tested service and policy 
creation)

Also tested user deletion on MySQL:

1. Created new user
2. Logged in as new user, created a policy
3. Logged back in as original user, delete user that created policy
4. Verified no errors, policy is still present/correct

Also searched the source code and JPA queries xml for other references to these 
fields


Thanks,

Andrew Luo

Review Request 74301: RANGER-2713: Remove audit fields from XXPolicyRef objects

[Andrew Luo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74301/
---

Review request for ranger and Pradeep Agrawal.


Repository: ranger


Description
---

XXPolicyRef objects have fields such as create time, update time, added by user 
ID, updated by user ID, but there fields are entirely useless since they are 
all copied from the XXPolicy object. In addition, while improving performance 
for creation of policies with large numbers of users, we discovered that a lot 
of time was being spent in JPA converting these Date objects especially. After 
removing these fields we saw a significant performance improvement (a secondary 
benefit is less database space usage).

We previously tried this commit and it caused a few issues (some typos on SQL 
Server and SQL Anywhere, as well as upgrade scenarios - however this has been 
fixed).


Diffs
-

  security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 9a79fe8ad 
  
security-admin/db/mysql/patches/061-drop-audit-columns-from-policy-ref-tables.sql
 PRE-CREATION 
  security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
fd6cec9a7 
  
security-admin/db/oracle/patches/061-drop-audit-columns-from-policy-ref-tables.sql
 PRE-CREATION 
  security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
4d5a8cedf 
  
security-admin/db/postgres/patches/061-drop-audit-columns-from-policy-ref-tables.sql
 PRE-CREATION 
  
security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 
3ed2a5b9c 
  
security-admin/db/sqlanywhere/patches/061-drop-audit-columns-from-policy-ref-tables.sql
 PRE-CREATION 
  security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
ca8f7da1f 
  
security-admin/db/sqlserver/patches/061-drop-audit-columns-from-policy-ref-tables.sql
 PRE-CREATION 
  security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 
6cc3509d8 
  
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java
 6af8f99f4 
  
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java 
4f4409d6a 
  
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefDataMaskType.java
 cb926740e 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java 
32a1b9f24 
  
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefResource.java 
115064621 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefRole.java 
7aee502e0 
  security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefUser.java 
8dfb92833 
  
security-admin/src/main/java/org/apache/ranger/service/XPortalUserService.java 
85e457efa 
  security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java 
691ab52b3 


Diff: https://reviews.apache.org/r/74301/diff/1/


Testing
---

Fresh install and upgrade tested on all 5 databases (tested service and policy 
creation)

Also tested user deletion on MySQL:

1. Created new user
2. Logged in as new user, created a policy
3. Logged back in as original user, delete user that created policy
4. Verified no errors, policy is still present/correct

Also searched the source code and JPA queries xml for other references to these 
fields


Thanks,

Andrew Luo

[jira] [Reopened] (RANGER-2713) Remove audit fields from XXPolicyRef objects

[Andrew Luo (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Luo reopened RANGER-2713:


> Remove audit fields from XXPolicyRef objects
> 
>
> Key: RANGER-2713
> URL: https://issues.apache.org/jira/browse/RANGER-2713
> Project: Ranger
>  Issue Type: Improvement
>  Components: admin, audit
>Affects Versions: master
>Reporter: Andrew Luo
>Priority: Minor
> Attachments: 
> 0001-RANGER-2713-Remove-audit-fields-from-XXPolicyRef-obj.patch, 
> 0001-RANGER-2713-Remove-audit-fields-from-XXPolicyRef-obj.patch
>
>
> XXPolicyRef objects have fields such as create time, update time, added by 
> user ID, updated by user ID, but there fields are entirely useless since they 
> are all copied from the XXPolicy object.  In addition, while improving 
> performance for creation of policies with large numbers of users, we 
> discovered that a lot of time was being spent in JPA converting these Date 
> objects especially.  After removing these fields we saw a significant 
> performance improvement (a secondary benefit is less database space usage).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74280: RANGER-4041 : Upgrade netty-all version to 4.1.86.Final

[Vishal Suvagia via Review Board]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74280/#review225165
---


Ship it!




Ship It!

- Vishal Suvagia


On Feb. 8, 2023, 6:56 a.m., Himanshu Maurya wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74280/
> ---
> 
> (Updated Feb. 8, 2023, 6:56 a.m.)
> 
> 
> Review request for ranger, bhavik patel, Dhaval Shah, Dineshkumar Yadav, 
> Harshal Chavan, Kishor Gollapalliwar, Madhan Neethiraj, Mehul Parikh, Nitin 
> Galave, Pradeep Agrawal, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4041
> https://issues.apache.org/jira/browse/RANGER-4041
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Upgraded netty-all version from 4.1.85.Final to 4.1.86.Final
> 
> 
> Diffs
> -
> 
>   pom.xml e402bcc5d 
> 
> 
> Diff: https://reviews.apache.org/r/74280/diff/1/
> 
> 
> Testing
> ---
> 
> Tested all CRUD operations like:-
> 1) Policies
> 2) Services
> 3) Zones
> 4) Users/Groups/Roles
> 5) Keys from KMS 
> 6) Checked all Audit event generate properly
> Also checked password and permission updation for users
> Run queries from backend for Hive, HBase, HDFS and YARN as different users 
> and checked the policies and plugins are working good
> 
> 
> Thanks,
> 
> Himanshu Maurya
> 
>

[jira] [Updated] (RANGER-4066) Add row draggable feature for audit filter table in service form in Ranger React.

[Brijesh Bhalala (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brijesh Bhalala updated RANGER-4066:

Fix Version/s: 3.0.0

> Add row draggable feature for audit filter table in service form in Ranger 
> React.
> -
>
> Key: RANGER-4066
> URL: https://issues.apache.org/jira/browse/RANGER-4066
> Project: Ranger
>  Issue Type: New Feature
>  Components: Ranger
>Reporter: Brijesh Bhalala
>Assignee: Brijesh Bhalala
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: 0001-RANGER-4066.patch
>
>
> 1)Provide a row drag feature for reordering audit filters table row in 
> service form.
> 2)Fixed the pageNo issue in policys, users/groups/roles listing tables.
> 3)Minor css improvement.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-4066) Add row draggable feature for audit filter table in service form in Ranger React.

[Brijesh Bhalala (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brijesh Bhalala updated RANGER-4066:

Attachment: 0001-RANGER-4066.patch

> Add row draggable feature for audit filter table in service form in Ranger 
> React.
> -
>
> Key: RANGER-4066
> URL: https://issues.apache.org/jira/browse/RANGER-4066
> Project: Ranger
>  Issue Type: New Feature
>  Components: Ranger
>Reporter: Brijesh Bhalala
>Assignee: Brijesh Bhalala
>Priority: Major
> Attachments: 0001-RANGER-4066.patch
>
>
> 1)Provide a row drag feature for reordering audit filters table row in 
> service form.
> 2)Fixed the pageNo issue in policys, users/groups/roles listing tables.
> 3)Minor css improvement.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (RANGER-4066) Add row draggable feature for audit filter table in service form in Ranger React.

[Brijesh Bhalala (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brijesh Bhalala updated RANGER-4066:

Attachment: (was: 0001-RANGER-4066.patch)

> Add row draggable feature for audit filter table in service form in Ranger 
> React.
> -
>
> Key: RANGER-4066
> URL: https://issues.apache.org/jira/browse/RANGER-4066
> Project: Ranger
>  Issue Type: New Feature
>  Components: Ranger
>Reporter: Brijesh Bhalala
>Assignee: Brijesh Bhalala
>Priority: Major
> Attachments: 0001-RANGER-4066.patch
>
>
> 1)Provide a row drag feature for reordering audit filters table row in 
> service form.
> 2)Fixed the pageNo issue in policys, users/groups/roles listing tables.
> 3)Minor css improvement.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Review Request 74300: RANGER-4066: Add row draggable feature for audit filter table in service form in Ranger React.

[Brijesh Bhalala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74300/
---

Review request for ranger, Dhaval Rajpara, Madhan Neethiraj, Mehul Parikh, 
Mugdha Varadkar, Nikunj Pansuriya, and Nitin Galave.


Bugs: RANGER-4066
https://issues.apache.org/jira/browse/RANGER-4066


Repository: ranger


Description
---

1)Provide a row drag feature for reordering audit filters table row in service 
form.
2)Fixed the pageNo issue in policys, users/groups/roles listing tables.
3)Minor css improvement.


Diffs
-

  security-admin/src/main/webapp/react-webapp/src/components/CreatableField.jsx 
66afce1ca 
  security-admin/src/main/webapp/react-webapp/src/components/Editable.jsx 
cf2292c82 
  security-admin/src/main/webapp/react-webapp/src/styles/style.css 6ffd55dfc 
  security-admin/src/main/webapp/react-webapp/src/utils/XAUtils.js 96657b57b 
  
security-admin/src/main/webapp/react-webapp/src/views/AuditEvent/AccessLogs.jsx 
73956bc9c 
  
security-admin/src/main/webapp/react-webapp/src/views/AuditEvent/AdminLogs/PolicyViewDetails.jsx
 accd48b0c 
  security-admin/src/main/webapp/react-webapp/src/views/Header.jsx b054dc21e 
  
security-admin/src/main/webapp/react-webapp/src/views/PermissionsModule/EditPermission.jsx
 0f14079f6 
  
security-admin/src/main/webapp/react-webapp/src/views/PolicyListing/AddUpdatePolicyForm.jsx
 1b163a724 
  
security-admin/src/main/webapp/react-webapp/src/views/PolicyListing/PolicyPermissionItem.jsx
 c088a95ef 
  
security-admin/src/main/webapp/react-webapp/src/views/ServiceManager/ServiceAuditFilter.jsx
 a92451fff 
  
security-admin/src/main/webapp/react-webapp/src/views/ServiceManager/ServiceDefinitions.jsx
 aa82bbf32 
  
security-admin/src/main/webapp/react-webapp/src/views/UserGroupRoleListing/groups_details/GroupForm.jsx
 6fc043a98 
  
security-admin/src/main/webapp/react-webapp/src/views/UserGroupRoleListing/role_details/RoleForm.jsx
 bb23a2727 
  
security-admin/src/main/webapp/react-webapp/src/views/UserGroupRoleListing/users_details/UserFormComp.jsx
 835291403 


Diff: https://reviews.apache.org/r/74300/diff/1/


Testing
---

1)Build and Verified Ranger Admin setup with this changes.
2)Verified the following things:-
  1)CRUD Operations for Policy, Services, Users/Groups/Roles.
  2)Policys, Users/Groups/Roles listing tables.


Thanks,

Brijesh Bhalala

[jira] [Created] (RANGER-4075) Add support for table functions to trino ranger plugin

[Mahebub Sayyed (Jira)]

Mahebub Sayyed created RANGER-4075:
--

 Summary: Add support for table functions to trino ranger plugin
 Key: RANGER-4075
 URL: https://issues.apache.org/jira/browse/RANGER-4075
 Project: Ranger
  Issue Type: Bug
  Components: plugins
Reporter: Mahebub Sayyed


Table functions were added to trino from 388 onwards and 
{{SystemAccessControl}} methods were updated to accommodate this change later. 
Therefore, to make the table function work with the Trino-Rangerplugin we need 
to re-implement method {{checkCanExecuteFunction}} with a new 
argument{{{}FunctionKind{}}}

Relevant links

Trino Slack: [https://trinodb.slack.com/archives/CGB0QHWSW/p1655456620535959]
[trinodb/trino#12544|https://github.com/trinodb/trino/pull/12544]
[trinodb/trino#12833|https://github.com/trinodb/trino/issues/12833]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (RANGER-3682) Unify the ways that rangerkeystore to encapsulate zonekey

[kirby zhou (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-3682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17685766#comment-17685766
 ] 

kirby zhou commented on RANGER-3682:


Any other idea?

> Unify the ways that rangerkeystore to encapsulate zonekey
> -
>
> Key: RANGER-3682
> URL: https://issues.apache.org/jira/browse/RANGER-3682
> Project: Ranger
>  Issue Type: Improvement
>  Components: kms
>Affects Versions: 3.0.0
>Reporter: kirby zhou
>Assignee: kirby zhou
>Priority: Major
>
> Unify the ways that rangerkeystore to encapsulate zonekey
> Now we have 2 styles of MasterKeyProvider:
>  # RangerMasterKey, RangerHSM, RangerSafenetKeySecure
>  # RangerAzureKeyVaultKeyGenerator, RangerGoogleCloudHSMProvider, 
> RangerTencentKMSProvider
> Style 1 can get out master key string from provider, Style 2 can not.
> In old, I add a flag KeyVaultEnabled to distinguish them. 
> KeyVaultEnabled=false means style1, true means style2
> RangerKeyStore with  style1 use SecretKeyEntry with SealedObject to store a 
> key and do encryption / decryption by itself.
> RangerKeyStore with  style2 use SecretKeyByteEntry to store a key and let MK 
> provider to encryption / decryption.
> These logics are hard-coded in the class RangerKeyStore. These are ugly and 
> hard to maintain. I refactor it by removing SecretKeyEntry, and let providers 
> of style1 do encryption / decryption. 
> Add a  common base class of RangerMasterKey, RangerHSM andd 
> RangerSafenetKeySecure, named AbstractRangerMasterKey. It provides the common 
> logic of encryptZoneKey and decryptZoneKey. AbstractRangerMasterKey encodes 
> SealedObject into byte[].
> So the new code does not change the actual storage format, and there is no 
> problem in compatibility.
> =
>  
> And, there is no unified method to initialize a master key provider. 
> Duplicate code is distributed in RangerKeyStoreProvider and a bunch of CLI 
> classes.
> I made a new RangerKMSMKIFactory class to unify it.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)