Re: Review Request 74958: RANGER-4775 : Ranger Kms is failing with oracle23 Database

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74958/#review226423
---


Ship it!




Ship It!

- Sailaja Polavarapu


On April 18, 2024, 8:57 a.m., Dhaval Shah wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74958/
> ---
> 
> (Updated April 18, 2024, 8:57 a.m.)
> 
> 
> Review request for ranger, bhavik patel, Dineshkumar Yadav, Jayendra Parab, 
> Kishor Gollapalliwar, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Pradeep 
> Agrawal, Ramesh Mani, Sailaja Polavarapu, Vikas Kumar, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: RANGER-4775
> https://issues.apache.org/jira/browse/RANGER-4775
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger KMS is failing with Oracle 23 database with below error.
> 
> ```
> 2024-04-01 07:25:27,032 INFO  org.apache.hadoop.crypto.key.RangerMasterKey: 
> Master Key doesn't exist in DB, Generating the Master Key
> 2024-04-01 07:25:27,169 ERROR org.apache.hadoop.crypto.key.RangerMasterKey: 
> Error while saving master key in Database!!!
> java.lang.ClassCastException: java.lang.String cannot be cast to java.sql.Clob
> at 
> org.eclipse.persistence.platform.database.oracle.Oracle8Platform.writeLOB(Oracle8Platform.java:194)
>  ~[eclipselink-2.7.12.jar:2.7.12.v20230209-e5c4074ef3]
> at 
> org.eclipse.persistence.internal.helper.LOBValueWriter.fetchLocatorAndWriteValue(LOBValueWriter.java:94)
>  ~[eclipselink-2.7.12.jar:2.7.12.v20230209-e5c4074ef3]
> at 
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.processResultSet(DatabaseAccessor.java:758)
>  ~[eclipselink-2.7.12.jar:2.7.12.v20230209-e5c4074ef3]
> at 
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:673)
>  ~[eclipselink-2.7.12.jar:2.7.12.v20230209-e5c4074ef3]
> at 
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:567)
>  ~[eclipselink-2.7.12.jar:2.7.12.v20230209-e5c4074ef3]
> at 
> org.eclipse.persistence.internal.helper.LOBValueWriter.buildAndExecuteCall(LOBValueWriter.java:79)
>  ~[eclipselink-2.7.12.jar:2.7.12.v20230209-e5c4074ef3]
> at 
> org.eclipse.persistence.internal.helper.LOBValueWriter.buildAndExecuteSelectCalls(LOBValueWriter.java:190)
>  ~[eclipselink-2.7.12.jar:2.7.12.v20230209-e5c4074ef3]
> at 
> org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.flushSelectCalls(DatabaseAccessor.java:175)
>  ~[eclipselink-2.7.12.jar:2.7.12.v20230209-e5c4074ef3]
> at 
> org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.insertObject(DatasourceCallQueryMechanism.java:457)
>  ~[eclipselink-2.7.12.jar:2.7.12.v20230209-e5c4074ef3]
> at 
> org.eclipse.persistence.internal.queries.StatementQueryMechanism.insertObject(StatementQueryMechanism.java:182)
>  ~[eclipselink-2.7.12.jar:2.7.12.v20230209-e5c4074ef3]
> at 
> org.eclipse.persistence.internal.queries.StatementQueryMechanism.insertObject(StatementQueryMechanism.java:197)
>  ~[eclipselink-2.7.12.jar:2.7.12.v20230209-e5c4074ef3]
> at 
> 
> ```
> 
> 
> Diffs
> -
> 
>   kms/src/main/java/org/apache/ranger/entity/XXRangerKeyStore.java 6b541eca5 
>   kms/src/main/java/org/apache/ranger/entity/XXRangerMasterKey.java 64fdd56e4 
> 
> 
> Diff: https://reviews.apache.org/r/74958/diff/1/
> 
> 
> Testing
> ---
> 
> Successful build with test cases.
> Successfully validated with all the supported DB Flavor
> 
> 
> Thanks,
> 
> Dhaval Shah
> 
>

Re: Review Request 74936: RANGER-4745: Enhance handling of subAccess authorization in Ranger HDFS plugin

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74936/#review226352
---


Ship it!




Ship It!

- Sailaja Polavarapu


On March 26, 2024, 4:18 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74936/
> ---
> 
> (Updated March 26, 2024, 4:18 p.m.)
> 
> 
> Review request for ranger, madhan, Madhan Neethiraj, Ramesh Mani, Sailaja 
> Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4745
> https://issues.apache.org/jira/browse/RANGER-4745
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> This describes how Ranger performs authorization of the HDFS commands which 
> require access to the hierarchy of files/directory rooted at the argument 
> passed to the HDFS command. Some examples of such commands are :
> 
> > hdfs dfs -count -q -h -v ;
> > hdfs dfs -R 
> 
> HDFS Authorization Interface
> 
> When these commands are invoked, HDFS Namenode builds a tree of i-nodes 
> corresponding to , and passes it to the authorizer with a flag 
> indicating that subAccess (access to the directory hierarchy rooted at 
> ) is to be checked.
> 
> Ranger implementation
> 
> For each directory in the hierarchy rooted at , Ranger code checks 
> if the requested permissions (typically read and execute) are allowed using 
> only Ranger policies. If any directory in the top-down path starting from 
>  does not allow access, then the authorization steps done until 
> then are discarded, and the HDFS default authorizer is called upon to check 
> the access with the same arguments. The default authorizer only checks the 
> HDFS ACLs (and not any Ranger policies) on each directory in the hierarchy to 
> determine the access.
> 
> Design of new Ranger implementation
> 
> For each directory in the hierarchy rooted at , new Ranger design 
> 1. Checks if the requested permissions are allowed using only Ranger policies
> 2. If the access is denied, the authorization steps done until this point are 
> discarded, and the HDFS default authorizer is called upon to check the access 
> with the original set of argument, and the result of default authorizer is 
> returned to Namenode.
> 3. Otherwise, if the access is not determined, a new set of arguments are 
> constructed for the directory being processed and HDFS default authorizer is 
> called to check the access with the modified set of arguments.
> 4. If the default authorizer does not allow the access, then the result is 
> returned to Namenode.
> 5. Otherwise, the processing continues with the next directory.
> 
> Performance considerations
> 
> The new implementation may have some impact on the performance. A few cases 
> are as follows.
> 1. There is a Ranger policy that allows requested permissions recursively to 
> some directory in the hierarchy. Depending on how deep this directory is in 
> the hierarchy, the number of directories for which the access evaluation is 
> requested will change. Higher this directory in the hierarchy, lesser the 
> number of evaluations. In the existing implementation, a short-circuiting of 
> calls for evaluating Ranger policies will, in general, happen earlier, and 
> the default authorizer will be called upon the handle the authorization.
> 2. In the worst case, if there is no Ranger policy for any directory in the 
> hierarchy, then each directory in the hierachy there will be a target of 
> access evaluation by Ranger and by the default authorizer (if the HDFS ACLs 
> for each directory allow requested accesses).
> 
> 
> Diffs
> -
> 
>   
> agents-common/src/main/java/org/apache/ranger/authorization/hadoop/constants/RangerHadoopConstants.java
>  a29390fd0 
>   
> hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java
>  9b410a185 
> 
> 
> Diff: https://reviews.apache.org/r/74936/diff/4/
> 
> 
> Testing
> ---
> 
> Tested with HDFS namenode; passes all unit tests.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Review Request 74936: RANGER-4745: Enhance handling of subAccess authorization in Ranger HDFS plugin

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74936/#review226348
---




hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java
Line 432 (original), 436 (patched)
<https://reviews.apache.org/r/74936/#comment314591>

Can we add this logic behind a flag so that we don't introduce regressions. 
We can have this logic will be disabled by default


- Sailaja Polavarapu


On March 12, 2024, 8:35 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74936/
> ---
> 
> (Updated March 12, 2024, 8:35 p.m.)
> 
> 
> Review request for ranger, madhan, Madhan Neethiraj, Ramesh Mani, Sailaja 
> Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-4745
> https://issues.apache.org/jira/browse/RANGER-4745
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> This describes how Ranger performs authorization of the HDFS commands which 
> require access to the hierarchy of files/directory rooted at the argument 
> passed to the HDFS command. Some examples of such commands are :
> 
> > hdfs dfs -count -q -h -v ;
> > hdfs dfs -R 
> 
> HDFS Authorization Interface
> 
> When these commands are invoked, HDFS Namenode builds a tree of i-nodes 
> corresponding to , and passes it to the authorizer with a flag 
> indicating that subAccess (access to the directory hierarchy rooted at 
> ) is to be checked.
> 
> Ranger implementation
> 
> For each directory in the hierarchy rooted at , Ranger code checks 
> if the requested permissions (typically read and execute) are allowed using 
> only Ranger policies. If any directory in the top-down path starting from 
>  does not allow access, then the authorization steps done until 
> then are discarded, and the HDFS default authorizer is called upon to check 
> the access with the same arguments. The default authorizer only checks the 
> HDFS ACLs (and not any Ranger policies) on each directory in the hierarchy to 
> determine the access.
> 
> Design of new Ranger implementation
> 
> For each directory in the hierarchy rooted at , new Ranger design 
> 1. Checks if the requested permissions are allowed using only Ranger policies
> 2. If the access is denied, the authorization steps done until this point are 
> discarded, and the HDFS default authorizer is called upon to check the access 
> with the original set of argument, and the result of default authorizer is 
> returned to Namenode.
> 3. Otherwise, if the access is not determined, a new set of arguments are 
> constructed for the directory being processed and HDFS default authorizer is 
> called to check the access with the modified set of arguments.
> 4. If the default authorizer does not allow the access, then the result is 
> returned to Namenode.
> 5. Otherwise, the processing continues with the next directory.
> 
> Performance considerations
> 
> The new implementation may have some impact on the performance. A few cases 
> are as follows.
> 1. There is a Ranger policy that allows requested permissions recursively to 
> some directory in the hierarchy. Depending on how deep this directory is in 
> the hierarchy, the number of directories for which the access evaluation is 
> requested will change. Higher this directory in the hierarchy, lesser the 
> number of evaluations. In the existing implementation, a short-circuiting of 
> calls for evaluating Ranger policies will, in general, happen earlier, and 
> the default authorizer will be called upon the handle the authorization.
> 2. In the worst case, if there is no Ranger policy for any directory in the 
> hierarchy, then each directory in the hierachy there will be a target of 
> access evaluation by Ranger and by the default authorizer (if the HDFS ACLs 
> for each directory allow requested accesses).
> 
> 
> Diffs
> -
> 
>   
> hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java
>  9b410a185 
> 
> 
> Diff: https://reviews.apache.org/r/74936/diff/3/
> 
> 
> Testing
> ---
> 
> Tested with HDFS namenode; passes all unit tests.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

[jira] [Created] (RANGER-4757) Use Ranger roles similar to RBAC for administering Ranger APIs instead of role assignment rules

[Sailaja Polavarapu (Jira)]

Sailaja Polavarapu created RANGER-4757:
--

 Summary: Use Ranger roles similar to RBAC for administering Ranger 
APIs instead of role assignment rules
 Key: RANGER-4757
 URL: https://issues.apache.org/jira/browse/RANGER-4757
 Project: Ranger
  Issue Type: Improvement
  Components: Ranger
Reporter: Sailaja Polavarapu


Currently users accessing/administering Ranger requires special privileges like 

ROLE_USER, ROLE_SYS_ADMIN, ROLE_KEY_ADMIN, ROLE_ADMIN_AUDITOR, or 
ROLE_KEY_ADMIN_AUDITOR. These roles are assigned to the user based on role 
assignment rules by Ranger Usersync process while syncing the users and groups 
from the sync source. This creates major overhead on Ranger User sync to 
compute the roles every time a group memberships are updated as well on Ranger 
admin to update the database to reflect the roles for each user.

As part of RANGER-2414, we introduced support for Role based authorization for 
plugins which basically checks the user/group in the authorization request is 
part of a role and uses that role information for authorization.

Proposal:
--
Create default special roles (ROLE_USER, ROLE_SYS_ADMIN, ROLE_KEY_ADMIN, 
ROLE_ADMIN_AUDITOR, or ROLE_KEY_ADMIN_AUDITOR) in Ranger DB as part of the 
setup and add default admin, rangerusersync, rangertagsync, and keyadmin users 
to the corresponding roles.
Similar to plugin role computation, when a user is accessing any Ranger API, 
get the groups that this user belongs to (from ranger DB) and check if the 
user/group is part of any of these special roles to authorize these APIs.

Pros: 
1. Role of a user is computed at the time of API access which in some cases can 
be dynamic
2. Reduces performance overhead on Ranger Usersync and Ranger Admin
3. Uniform user experience for Ranger APIs authorization as well as data 
authorization
4. Avoids inconsistency of setting a specific role for a user via Ranger UI and 
thru role assignment rules.
5. Can consolidate few DB tables that currently saves role information for a 
user

Cons:
1. Requires decent amount of code changes
2. Regression testing impact
3. Migrating role assignment rules to Roles as part of upgrade



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74900: RANGER-4722: HDFS authorization logic for directory hierarchy rooted at "/" is incorrect

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74900/#review226230
---


Ship it!




Ship It!

- Sailaja Polavarapu


On Feb. 20, 2024, 11:35 p.m., Abhay Kulkarni wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74900/
> ---
> 
> (Updated Feb. 20, 2024, 11:35 p.m.)
> 
> 
> Review request for ranger, madhan, Madhan Neethiraj, Ramesh Mani, and Sailaja 
> Polavarapu.
> 
> 
> Bugs: RANGER-4722
> https://issues.apache.org/jira/browse/RANGER-4722
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Ranger authorization logic for the HDFS commands that require authorization 
> of the entire directory hierarchy rooted at the specified directory argument 
> is incorrect as it does not correctly compute the sub-directory paths. The 
> paths of sub-directories that need to be authorized incorrectly contain an 
> extra '/' character, which leads to incorrect authorization result.
> 
> 
> Diffs
> -
> 
>   
> hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java
>  9b1279bcb 
> 
> 
> Diff: https://reviews.apache.org/r/74900/diff/1/
> 
> 
> Testing
> ---
> 
> Compiles clean and passes unit tests.
> 
> 
> Thanks,
> 
> Abhay Kulkarni
> 
>

Re: Planning for Apache Ranger 2.4.0 release

[Sailaja Polavarapu]

+1 for Apache Ranger 2.4.0 release.
Thanks Selva for initiating this.

- Sailaja

On Sat, Feb 4, 2023 at 2:42 PM Ramesh Mani  wrote:

> +1 for Apache Ranger 2.4.0 release activities.
>
> Thanks
> Ramesh
>
> On Tue, Jan 31, 2023 at 7:56 AM Selvamohan Neethiraj 
> wrote:
>
> > Dear Ranger Community members, There are various features and critical
> > bug fixes done in the Apache Ranger project since the release of Apache
> > Ranger 2.3.0. Around 3 New Features, 34 improvements and 60 bug fixes
> > were made from the last release. Now with that Ranger community is
> > expecting a release to adapt those changes and hence planning this
> > release. Please review and provide your opinion.
> >
> > Thanks, Selva-
> >
> > _*Improvements & New Features addressed in 2.4.0*_
> >
> > Improvement,RANGER-2928,[Ranger Zone REST API] Resources data is missing
> > in XML format Improvement,RANGER-3165,Upgrade Elasticsearch version in
> > Ranger to Elasticsearch 7.10.2 Improvement,RANGER-3534,Review of
> > RangerHiveAuditHandler Improvement,RANGER-3623,Add ability to enable
> > anonymous download of policy/role/tag Improvement,RANGER-3633,Remove
> > eclipse .project file from git Improvement,RANGER-3664,Ranger KMS : Add
> > refresh functionality on kms key listing page.
> > Improvement,RANGER-3763,The max limit of the requested entities is not
> > configurable in tagsync Improvement,RANGER-3767,Add text message in HDFS
> > and YARN policy pages to highlight the fallback ACL option
> > Improvement,RANGER-3787,Non-daemon threads started by
> > ElasticSearchAuditDestination cause Spark application hanging
> > Improvement,RANGER-3794,Improve performance of delete users/groups
> > utility Improvement,RANGER-3796,Enhancement to support multiple resource
> > sets in a policy Improvement,RANGER-3818,Upgrade Solr to 8.11.2
> > Improvement,RANGER-3822,RangerService outputs password information in
> > plaintext Improvement,RANGER-3837,"Allow Ranger non-admins to get,
> > create, edit and delete roles" Improvement,RANGER-3856,Ranger admin
> > client option to work with non-kerberized server
> > Improvement,RANGER-3865,support for using user attributes in masking
> > expressions Improvement,RANGER-3900,"Roles deletion Takes time in Apache
> > Ranger when there are more users,groups,roles"
> > Improvement,RANGER-3902,dbLoadTime is not added correctly in
> > RangerServicePoliciesCache Improvement,RANGER-3910,API Documentation is
> > broken for knox sso Improvement,RANGER-3934,improve tag cache handling
> > to reduce resource usage Improvement,RANGER-3940,"Add javascript
> > includes(), intersects() polyfills for array prototype to
> > RangerCommonConstants" Improvement,RANGER-3948,update serialization to
> > skip empty values Improvement,RANGER-3951,optimize memory used for tags
> > in plugins and server Improvement,RANGER-3955,optimization to reduce
> > duplicate strings Improvement,RANGER-3973,LDAP incremental search not
> > always available Improvement,RANGER-3978,Docker setup to run Ranger KMS
> > Improvement,RANGER-3982,Python client for Ranger KMS REST APIs
> > Improvement,RANGER-3983,Support getColumnMasks and getRowFilters in
> > Trino SPI 376+ Improvement,RANGER-3985,Trino plugin: Check table name
> > when creating tables Improvement,RANGER-3986,Upgrade trino guice
> > dependency to 5.1.0 Improvement,RANGER-3988,Trino plugin should
> > differntiate between views and tables Improvement,RANGER-3997,option to
> > use default value when user/group/tag does not have the attribute
> > Improvement,RANGER-4004,"During the service deletion also, we can clear
> > the in-memory cache for that service which got deleted on the ranger
> > side" Improvement,RANGER-4011,option to disable creation of default
> > policies per hierarchy New Feature,RANGER-3852,Performance and
> > scalability analyzer tool for Ranger New
> > Feature,RANGER-3855,RangerExternalUserStoreRetriever class New
> > Feature,RANGER-4028,Ranger - Upgrade bootbox.js.
> >
> > _*BUGS addressed in 2.4.0*_
> >
> > Bug,RANGER-2737,Ranger REST API returns different infomation when GET
> > user by id and by name Bug,RANGER-3080,"A service administrator should
> > be allowed to set ""excludes"" flag for a policy resource"
> > Bug,RANGER-3108,NPE in RangerPolicyRepository.init
> > Bug,RANGER-3387,Ranger Admin Header Validation. Bug,RANGER-3394,Too much
> > `varchar(4000)` causes table to exceed ROW SIZE limit in MySQL
> > Bug,RANGER-3500,Ranger policy list doesn't support sorting by field
> > Bug,RANGER-3670,Policy update creates unnecessary entries in transaction
> > log table Bug,RANGER-3680,mysql ErrorCode:1118 when Importing DB schema
> > to database Bug,RANGER-3719,Can not create mysql table with charset
> > utf8mb4. Bug,RANGER-3775,Logback.xml has been incorrectly modified by
> > RANGER-3704. Bug,RANGER-3789,Upgrade Handlebars version to 4.7.7
> > Bug,RANGER-3790,Ranger tagsync module should not depend on kafka server
> > Bug,RANGER-3798,"Ranger API Resource Metrics REST ""Up time 

Re: Review Request 74281: RANGER-4026: Allow sync source updates for existing users synced via different sync sources

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74281/#review225133
---


Ship it!




Ship It!

- Sailaja Polavarapu


On Jan. 15, 2023, 11:13 p.m., Abhishek  Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74281/
> ---
> 
> (Updated Jan. 15, 2023, 11:13 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Sailaja Polavarapu.
> 
> 
> Bugs: RANGER-4026
> https://issues.apache.org/jira/browse/RANGER-4026
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> The change allows existing external users and groups in ranger to sync from a 
> different sync source.
> 
> 
> Diffs
> -
> 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/config/UserGroupSyncConfig.java
>  531e35a95 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
>  0aa419f3f 
> 
> 
> Diff: https://reviews.apache.org/r/74281/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean package runs fine.
> 
> 
> Thanks,
> 
> Abhishek  Kumar
> 
>

Re: Review Request 74279: RANGER-4042: Remove redundant ugsync poc package

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74279/#review225101
---


Ship it!




Ship It!

- Sailaja Polavarapu


On Jan. 15, 2023, 11:01 p.m., Abhishek  Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74279/
> ---
> 
> (Updated Jan. 15, 2023, 11:01 p.m.)
> 
> 
> Review request for ranger, Madhan Neethiraj and Sailaja Polavarapu.
> 
> 
> Bugs: RANGER-4042
> https://issues.apache.org/jira/browse/RANGER-4042
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> org.apache.ranger.unixusersync.poc package and its classes are redundant and 
> unreferenced. This may be removed.
> 
> 
> Diffs
> -
> 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/poc/InvalidGroupException.java
>  86197bfd6 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/poc/InvalidUserException.java
>  0dca55e00 
>   ugsync/src/main/java/org/apache/ranger/unixusersync/poc/ListRangerUser.java 
> 4fa1ad05d 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/poc/ListRangerUserGroup.java
>  8d5bf2a17 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/poc/ListUserGroupTest.java
>  6fc4c2db7 
>   ugsync/src/main/java/org/apache/ranger/unixusersync/poc/ListUserTest.java 
> 04f70d05d 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/poc/RangerClientUserGroupMapping.java
>  88a6c5d72 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/poc/RangerJSONParser.java 
> 0a3d6d7b8 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/poc/RangerUserGroupMapping.java
>  d1851277f 
>   ugsync/src/main/java/org/apache/ranger/unixusersync/poc/RestClientPost.java 
> b4b69ecc2 
> 
> 
> Diff: https://reviews.apache.org/r/74279/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean package runs fine.
> No references found in the codebase.
> 
> 
> Thanks,
> 
> Abhishek  Kumar
> 
>

[jira] [Created] (RANGER-4026) Provide option to update group memberships when same users/groups are synced from different sync sources

[Sailaja Polavarapu (Jira)]

Sailaja Polavarapu created RANGER-4026:
--

 Summary: Provide option to update group memberships when same 
users/groups are synced from different sync sources
 Key: RANGER-4026
 URL: https://issues.apache.org/jira/browse/RANGER-4026
 Project: Ranger
  Issue Type: Improvement
  Components: usersync
Reporter: Sailaja Polavarapu


RANGER-3254 implemented a change in user/group mapping so that sync source is 
taken into account when a group name matches multiple sources. LDAP users 
belonging to a group like "CN=mygroup" will not be synced in Ranger if there is 
an existing "mygroup" that was imported by UnixUserGroupBuilder.

This breaks a very common use case where posix users and groups are synced to 
the OS from an LDAP backend using SSSD, Centrify, or similar utilities. In 
those cases, both the linux OS and LDAP/AD are using the same identity 
repository. If Ranger imported a set of users and groups from one sync source, 
and then later switches to another, group mappings break and users don't get 
all of their groups.

Provide an option for customers to treat users/groups from multiple sync 
sources as same for updating group memberships.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (RANGER-4026) Provide option to update group memberships when same users/groups are synced from different sync sources

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-4026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu reassigned RANGER-4026:
--

Assignee: Sailaja Polavarapu

> Provide option to update group memberships when same users/groups are synced 
> from different sync sources
> 
>
> Key: RANGER-4026
> URL: https://issues.apache.org/jira/browse/RANGER-4026
> Project: Ranger
>  Issue Type: Improvement
>  Components: usersync
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
>
> RANGER-3254 implemented a change in user/group mapping so that sync source is 
> taken into account when a group name matches multiple sources. LDAP users 
> belonging to a group like "CN=mygroup" will not be synced in Ranger if there 
> is an existing "mygroup" that was imported by UnixUserGroupBuilder.
> This breaks a very common use case where posix users and groups are synced to 
> the OS from an LDAP backend using SSSD, Centrify, or similar utilities. In 
> those cases, both the linux OS and LDAP/AD are using the same identity 
> repository. If Ranger imported a set of users and groups from one sync 
> source, and then later switches to another, group mappings break and users 
> don't get all of their groups.
> Provide an option for customers to treat users/groups from multiple sync 
> sources as same for updating group memberships.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (RANGER-3920) When sync'ing users from Ldap, intermittent User/Group/UserGroup membership is missing

[Sailaja Polavarapu (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-3920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607239#comment-17607239
 ] 

Sailaja Polavarapu commented on RANGER-3920:


Hi [~ankita] , This might be duplicate for RANGER-3469. Can you please check 
the below review request and may be merge both if needed?

https://reviews.apache.org/r/73677/

> When sync'ing users from Ldap,  intermittent User/Group/UserGroup membership  
> is missing
> 
>
> Key: RANGER-3920
> URL: https://issues.apache.org/jira/browse/RANGER-3920
> Project: Ranger
>  Issue Type: Bug
>  Components: admin, Ranger, usersync
>Reporter: Ankita Sinha
>Assignee: Ankita Sinha
>Priority: Major
> Fix For: 2.4.0
>
>
> Problem Statement:
> Currently, Usersync sync's users/groups/usergroup in batches. In doing so, 
> the user/group/usergroup at the end of the batch is not sync'ed. 
> For example consider batch is of size 1000, so batch starts from 0-999 and  
> then  next batch starts from 1001-1999. So in each batch one 
> user/group/usergroup is missing.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 73896: RANGER-3657: Support for recursive ACL check for subpaths in Ozone plugin

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73896/
---

(Updated Aug. 4, 2022, 9:47 p.m.)


Review request for ranger, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, 
Ramesh Mani, and Velmurugan Periasamy.


Changes
---

Added code to retrieve all the children for likely matching resources for 
recursive delete evaluation and incorporated some review comments


Bugs: RANGER-3657
https://issues.apache.org/jira/browse/RANGER-3657


Repository: ranger


Description
---

Added code to support recursive delete operations for Ranger Ozone plugin 
including upgrade patch for service def changes. Also added plugin side change 
to retrieve owner information from the request context as an extention to 
https://reviews.apache.org/r/73051/
Updated Ozone version.


Diffs (updated)
-

  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/PolicyEngine.java
 f44570623 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessRequest.java
 6a38747f4 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java
 3ae0add51 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerResourceTrie.java
 504acd3a2 
  
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java
 dc51078a7 
  agents-common/src/main/resources/service-defs/ranger-servicedef-ozone.json 
4b899736b 
  
agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestPathResourceTrie.java
 30a7215a6 
  plugin-ozone/pom.xml 264197acf 
  
plugin-ozone/src/main/java/org/apache/ranger/authorization/ozone/authorizer/RangerOzoneAuthorizer.java
 12e647ca3 
  pom.xml 0945f4b1d 
  ranger-ozone-plugin-shim/pom.xml ab3dc4e3f 
  security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 833ffa0e0 
  security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
0d79d30f2 
  security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
d0e6a3824 
  
security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 
20ab7c224 
  security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
827e982af 
  security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyAdminImpl.java 
97a384f30 
  
security-admin/src/main/java/org/apache/ranger/patch/PatchForOzoneServiceDefUpdate_J10057.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/73896/diff/2/

Changes: https://reviews.apache.org/r/73896/diff/1-2/


Testing
---

Patched cluster and verified functionality with recursive delete operations and 
few regression tests.
Also verified manually upgrade patch changes.


Thanks,

Sailaja Polavarapu

Re: Review Request 74058: RANGER-3829: Incremental Sync Value to be read from usersync config

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74058/#review224576
---


Ship it!




Ship It!

- Sailaja Polavarapu


On July 19, 2022, 1:51 a.m., Abhishek  Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74058/
> ---
> 
> (Updated July 19, 2022, 1:51 a.m.)
> 
> 
> Review request for ranger, Sailaja Polavarapu and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3829
> https://issues.apache.org/jira/browse/RANGER-3829
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Config parameter ranger.usersync.ldap.deltasync is hardcoded to true, the 
> review allows the param to be read from config file.
> 
> 
> Diffs
> -
> 
>   
> ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapUserGroupBuilder.java
>  550775f65 
> 
> 
> Diff: https://reviews.apache.org/r/74058/diff/2/
> 
> 
> Testing
> ---
> 
> Tested on remote cluster.
> 
> 
> Thanks,
> 
> Abhishek  Kumar
> 
>

Re: Review Request 74042: Fix ConcurrentModificationException in UnixUserGroupBuilder

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74042/#review224561
---


Ship it!




Ship It!

- Sailaja Polavarapu


On July 14, 2022, 12:17 a.m., Abhishek  Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74042/
> ---
> 
> (Updated July 14, 2022, 12:17 a.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, and 
> Sailaja Polavarapu.
> 
> 
> Bugs: RANGER-3813
> https://issues.apache.org/jira/browse/RANGER-3813
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Line number 426 in 
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/UnixUserGroupBuilder.java
>  updates the map while iteration which raises the exception 
> ConcurrentModificationException.
> 
> 
> Steps to reproduce the issue:
> 1. Set nss and enumerateGroupMembers to true.
> 2. Create a user with userid < minimumUserId.
> 3. Add it to a group with groupId >= minimumGroupId.
> 4. Ensure the user is part of multiple groups and any one group that the user 
> is part of does not show the user as its member on executing: getent group
> 
> 
> Diffs
> -
> 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/UnixUserGroupBuilder.java
>  7653dfdbe 
> 
> 
> Diff: https://reviews.apache.org/r/74042/diff/5/
> 
> 
> Testing
> ---
> 
> Tested the fix on a live cluster.
> 
> 
> Thanks,
> 
> Abhishek  Kumar
> 
>

[jira] [Comment Edited] (RANGER-3387) Ranger Admin Header Validation.

[Sailaja Polavarapu (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17566563#comment-17566563
 ] 

Sailaja Polavarapu edited comment on RANGER-3387 at 7/14/22 12:19 AM:
--

Merged follow-up changes to master to handle requests from Knox gateway


was (Author: spolavarapu):
Merged changes to master.

> Ranger Admin Header Validation.
> ---
>
> Key: RANGER-3387
> URL: https://issues.apache.org/jira/browse/RANGER-3387
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 2.2.0
>Reporter: Mateen N Mansoori
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: 
> 0001-RANGER-3387-Added-extra-validation-for-handling-PUT-.patch, 
> 0001-RANGER-3387-Ranger-Admin-Header-Validation.patch
>
>
> Ranger Admin Header Validation.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: Review Request 74042: Fix ConcurrentModificationException in UnixUserGroupBuilder

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74042/#review224539
---




ugsync/src/main/java/org/apache/ranger/unixusersync/process/UnixUserGroupBuilder.java
Line 246 (original), 240 (patched)
<https://reviews.apache.org/r/74042/#comment313272>

It is better to use Iterator for traversal. Please don't change this.



ugsync/src/main/java/org/apache/ranger/unixusersync/process/UnixUserGroupBuilder.java
Line 373 (original), 359 (patched)
<https://reviews.apache.org/r/74042/#comment313273>

I think it will be sufficient to use Iterator for traversal in order to fix 
the ConcurrentModificationException. Please check



ugsync/src/main/java/org/apache/ranger/unixusersync/process/UnixUserGroupBuilder.java
Line 382 (original)
<https://reviews.apache.org/r/74042/#comment313274>

I think we still need this block. This could be the case where getent 
groups (as part of parseMembers()) have returned some users that we don't want 
to process isn't it?



ugsync/src/main/java/org/apache/ranger/unixusersync/process/UnixUserGroupBuilder.java
Line 549 (original)
<https://reviews.apache.org/r/74042/#comment313275>

It doesn't harm to check for null?



ugsync/src/main/java/org/apache/ranger/unixusersync/process/UnixUserGroupBuilder.java
Line 583 (original)
<https://reviews.apache.org/r/74042/#comment313276>

Same as above. Please check


- Sailaja Polavarapu


On July 1, 2022, 7:40 p.m., Abhishek  Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74042/
> ---
> 
> (Updated July 1, 2022, 7:40 p.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, and 
> Sailaja Polavarapu.
> 
> 
> Bugs: RANGER-3813
> https://issues.apache.org/jira/browse/RANGER-3813
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Line number 426 in 
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/UnixUserGroupBuilder.java
>  updates the map while iteration which raises the exception 
> ConcurrentModificationException.
> 
> 
> Steps to reproduce the issue:
> 1. Set nss and enumerateGroupMembers to true.
> 2. Create a user with userid < minimumUserId.
> 3. Add it to a group with groupId >= minimumGroupId.
> 4. Ensure the user is part of multiple groups and any one group that the user 
> is part of does not show the user as its member on executing: getent group
> 
> 
> Diffs
> -
> 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/UnixUserGroupBuilder.java
>  7653dfdbe 
> 
> 
> Diff: https://reviews.apache.org/r/74042/diff/2/
> 
> 
> Testing
> ---
> 
> Pending.
> 
> 
> Thanks,
> 
> Abhishek  Kumar
> 
>

Re: [VOTE] Release Apache Ranger version 2.3.0 - rc3

[Sailaja Polavarapu]

+1
Thanks Ramesh for putting this together. Verified build and signatures
- Sailaja.

On Wed, Jun 29, 2022 at 9:53 AM Velmurugan Periasamy  wrote:

> +1
>
> Verified build and signatures.
>
> Thanks Ramesh for putting 2.3.0 release together.
>
> > On Jun 28, 2022, at 9:19 PM, Bhavik Patel 
> wrote:
> >
> > +1
> > - Verified 2.3.0-rc3 builds successfully
> >
> >
> > Thanks,
> > Bhavik Patel
> > +91-7208744109
> >
> > On Wed, Jun 29, 2022, 2:42 AM Abhishek Kumar <
> abhishekkumar100...@gmail.com >
> wrote:
> > +1
> > - Verified 2.3.0-rc3 builds successfully
> >
> > On Tue, Jun 28, 2022 at 11:58 AM Nixon Rodrigues  > wrote:
> >
> > > +1 for rc3 release.
> > >
> > > Verified the build and signatures.
> > >
> > > Thanks Ramesh for putting this together for release
> > >
> > > On Tue, 28 Jun 2022 at 11:56, Selvamohan Neethiraj <
> > > sneet...@appsdefense.com >
> > > wrote:
> > >
> > > > +1
> > > >
> > > >   * verified signature
> > > >   * Builds successfully
> > > >
> > > > Thanks Ramesh for driving the release.
> > > >
> > > > Selva-
> > > >
> > > > On 6/25/22 3:00 AM, Ramesh Mani wrote:
> > > > > Dear Rangers,
> > > > >
> > > > > Apache Ranger 2.3.0 release candidate #3 is now available for
> > > > > a vote within the dev community. Links to the release artifacts are
> > > > > given below. Please review and vote.
> > > > >
> > > > > The vote will be open for at least 72 hours or until
> > > > > necessary votes are reached.
> > > > >   [ ] +1 approve
> > > > >   [ ] +0 no opinion
> > > > >   [ ] -1 disapprove (and reason why)
> > > > >
> > > > > Thanks,
> > > > > Ramesh
> > > > >
> > > > > List of all issues / improvements addressed in this release:
> > > > > https://issues.apache.org/jira/issues/?jql=project=RANGER <
> https://issues.apache.org/jira/issues/?jql=project=RANGER> AND
> > > > > status=Resolved AND fixVersion=2.3.0 ORDER BY key DESC
> > > > >
> > > > > Git tag for the release:
> > > > > https://github.com/apache/ranger/tree/release-2.3.0-rc3 <
> https://github.com/apache/ranger/tree/release-2.3.0-rc3>
> > > > >
> > > > > Sources for the release:
> > > > >
> > > >
> > >
> https://dist.apache.org/repos/dist/dev/ranger/2.3.0-rc3/apache-ranger-2.3.0.tar.gz
> <
> https://dist.apache.org/repos/dist/dev/ranger/2.3.0-rc3/apache-ranger-2.3.0.tar.gz
> >
> > > > >
> > > > > Source release verification:
> > > > >   PGP Signature:
> > > > >
> > > >
> > >
> https://dist.apache.org/repos/dist/dev/ranger/2.3.0-rc3/apache-ranger-2.3.0.tar.gz.asc
> <
> https://dist.apache.org/repos/dist/dev/ranger/2.3.0-rc3/apache-ranger-2.3.0.tar.gz.asc
> >
> > > > >   SHA256 Hash:
> > > > >
> > > >
> > >
> https://dist.apache.org/repos/dist/dev/ranger/2.3.0-rc3/apache-ranger-2.3.0.tar.gz.sha256
> <
> https://dist.apache.org/repos/dist/dev/ranger/2.3.0-rc3/apache-ranger-2.3.0.tar.gz.sha256
> >
> > > > >   SHA512 Hash:
> > > > >
> > > >
> > >
> https://dist.apache.org/repos/dist/dev/ranger/2.3.0-rc3/apache-ranger-2.3.0.tar.gz.sha512
> <
> https://dist.apache.org/repos/dist/dev/ranger/2.3.0-rc3/apache-ranger-2.3.0.tar.gz.sha512
> >
> > > > >
> > > > > Keys to verify the signature of the release artifacts are available
> > > > > at: https://dist.apache.org/repos/dist/release/ranger/KEYS <
> https://dist.apache.org/repos/dist/release/ranger/KEYS>
> > > > >
> > > > > New features/enhancements:
> > > > >
> > > > > RANGER-2846 Add support for resource[volume, bucket, key] look up
> in
> > > > > ozone plugin
> > > > > RANGER-2967 Add support for Amazon CloudWatch Logs as an Audit
> Store
> > > > > RANGER-3023 Permission tab takes longer time to load with large
> number
> > > > > of users and group_users data
> > > > > RANGER-3030 Replace Findbugs with Spotbugs maven plugin
> > > > > RANGER-3182 Prestosql is renamed to Trino
> > > > > RANGER-3221 Improve logging in Presto plugin
> > > > > RANGER-3276 Remove duplicate code from buildks.java
> > > > > RANGER-3290 ArrayIndexOutOfBoundsException if solr is down
> > > > > RANGER-3299 Upgrading the bouncycastle version for bcprov-jdk15on
> > > > > RANGER-3298 Add coarse URI check for Hive Agent
> > > > > RANGER-3389 Swagger UI Support for Ranger REST API
> > > > > RANGER-3435 Add unique index on guid, service and zone_id column of
> > > > > x_policy table
> > > > > RANGER-3439 Add rest api to get or delete ranger policy based on
> guid
> > > > > RANGER-3455 [Logout-Ranger] Should either be disabled/ should
> redirect
> > > > > to knox logout page
> > > > > RANGER-3459 Upgrade Ranger's Kafka dependency to 2.8
> > > > > RANGER-3475 Promote TagRest endpoints to /public/v2
> > > > > RANGER-3487 Update underscore js with latest version.
> > > > > RANGER-3493 Add unique index on service and resource_signature
> column
> > > > > of x_policy table
> > > > > RANGER-3498 RANGER : Remove log4j1 dependencies.
> > > > > RANGER-3504 Create framework to execute DB patch dependent on Java
> > > patch.
> > > > > 

Re: Review Request 74039: RANGER-3806: Group's users mapping entry failing whenever primary key auto-increment is not set to 1 in db

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74039/#review224536
---


Ship it!




Ship It!

- Sailaja Polavarapu


On June 24, 2022, 10:12 a.m., Pradeep Agrawal wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/74039/
> ---
> 
> (Updated June 24, 2022, 10:12 a.m.)
> 
> 
> Review request for ranger, bhavik patel, Dhaval Shah, Abhay Kulkarni, Madhan 
> Neethiraj, Mehul Parikh, Ramesh Mani, Sailaja Polavarapu, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: RANGER-3806
> https://issues.apache.org/jira/browse/RANGER-3806
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> **Problem Statement:** 
> Ranger db table `x_group_users` has columns "added_by_id" and "upd_by_id" 
> which should be filled with the id of logged-in user. Whenever group's-users 
> mapping is pushed from usersync module at that time these values are 
> considered from a config "ranger.xuser.createdByUserId" which is set to 1 in 
> ranger-default-admin-site.xml and code as well.
> If users have set ranger db table's primary key's auto-increment offset to 
> any number other than 1 and have not updated the same accordingly in the 
> config "ranger.xuser.createdByUserId" then request may fail due to Foreignkey 
> constraint violation.
> 
> Example Case: if primary key's auto-increment start value has been set to 2 
> in db engine then first entry in table x_portal_user will have primary-key id 
> 2. while if config  "ranger.xuser.createdByUserId" is set to 1 then due to 
> non availablity of userID 1 when it will try to insert group-user mappinng 
> entry in `x_group_users` table with value 1 for the column "added_by_id" and 
> "upd_by_id" it will cause to fail the request.
> 
> **Proposed solution:** Rather directly putting the config's 
> "ranger.xuser.createdByUserId" value in "added_by_id" and "upd_by_id" field 
> of `x_group_users` table we can check whether user id exist in x_portal_user 
> or not. if it does not exist then do not set the value received from config 
> "ranger.xuser.createdByUserId" and avoid the Foreign-key Constraint Violation 
> Exception.
> 
> 
> Diffs
> -
> 
>   
> security-admin/src/main/java/org/apache/ranger/service/XGroupUserService.java 
> 9345e8180 
> 
> 
> Diff: https://reviews.apache.org/r/74039/diff/1/
> 
> 
> Testing
> ---
> 
> With the patch issue is not reproducing when i have set the 
> auto_increment_increment to 3 in mysql server env.
> 
> 
> Thanks,
> 
> Pradeep Agrawal
> 
>

Re: Planning for Apache Ranger 2.3.0 release

[Sailaja Polavarapu]

Hi Ramesh,
 +1 for Ranger 2.3 release. Thanks for the initiative.
- Sailaja

On Thu, Apr 28, 2022 at 7:21 AM Ramesh Mani  wrote:

> zhoutianling,
>
> Thanks for the review. These Jiras are part of the Apache Ranger 2.3 apache
> release, it's not pulled in this published list as this may not have
> correct fixed version maintained or it is part of KMS as a component. I
> shall add it to the list in the release note.
>
> Thanks,
> Ramesh
>
> On Thu, Apr 28, 2022 at 2:19 AM KirbY ZhoU 
> wrote:
>
> > Missed some commit
> > For example:
> >
> > RANGER-3299
> > RANGER-3580
> > RANGER-3600
> > RANGER-3619
> > RANGER-3669
> >
> > 在 2022/4/27 14:49，“Ramesh Mani” 写入:
> >
> > Dear Ranger Community members,
> >
> > There are various features and critical bug fixes done in the Apache
> > Ranger
> > project since the release of Apache Ranger 2.2.0.
> > Around 55 improvements, 45 bug fixes and a total of 527 commits were
> > made
> > from the last release.
> > Now with that Ranger community is expecting a release to adapt those
> > changes and hence planning this release.
> >
> > Please review and provide your opinion.
> >
> > Thanks,
> > Ramesh
> >
> > *Improvements:*
> >
> > RANGER-3687 Password Policy Best Practices for Strong Security
> > RANGER-3667 Improve feedback in policy creation UI when resource
> > does
> > not exist
> > RANGER-3659 Ranger Admin goes to OOM when usersync is trying to
> > delete
> > existing group mappings from ranger DB
> > RANGER-3459 Upgrade Ranger's Kafka dependency to 2.8
> > RANGER-3551 Analyze & optimize module permissions related API
> > RANGER-3539 Add jacoco-maven-plugin for code coverage
> > RANGER-3562 Redesign post commit tasks for updating ref-tables
> when
> > policy/role is updated
> > RANGER-3540 Add support to read audit logs from Amazon CloudWatch
> > RANGER-3030 Replace Findbugs with Spotbugs maven plugin
> > RANGER-3538 Reduce the granularity of locking when
> > building/retrieving
> > a policy-engine within Ranger admin service
> > RANGER-3518 Limit the query size stored in Audit logs
> > RANGER-3276 Remove duplicate code from buildks.java
> > RANGER-3515 Enhance Ranger Java client SSL config to be
> configured
> > using serviceType and AppId
> > RANGER-3504 Create framework to execute DB patch dependent on
> Java
> > patch.
> > RANGER-3023 Permission tab takes longer time to load with large
> > number
> > of users and group_users data
> > RANGER-3487 Update underscore js with latest version.
> > RANGER-3548 Update performance engine test scripts
> > RANGER-3556 Ranger tagsync logs unnecessary messages
> > RANGER-3573 Add vim in docker base image
> > RANGER-3578 Simplify code for policy label creation
> > RANGER-3675 Upgrade tomcat due to intermittent READ TIMEOUT
> > RANGER-3686 Docker setup to run Ranger with MySQL database
> > RANGER-3628 Support fine grain authorization for different solr
> > objects
> > RANGER-3629 RANGER -  Handle solr permissions during upgrade
> > RANGER-3665 "No Data Found !!" messages in Ranger admin UI alarm
> > users
> > RANGER-3662 There should be pause button for error popup
> > RANGER-3660 [Ranger Admin UI] Improvements in tooltip hints for
> > better
> > user experience
> > RANGER-3649 Represent the Solr admin object types on the Ranger
> UI
> > RANGER-3658 Docker: Ranger containers to run as user=ranger
> > RANGER-3603 HDFS audit files rollover improvement to trigger
> > rollover
> > in monitoring thread
> > RANGER-3651 Remove jersey 1.x version dependency for knox plugin
> > RANGER-3621 Optimise Tag/Policy iterator
> > RANGER-3521 Ranger KMS IS NOT ENFORCING HSTS ON SSL PORT DEFINED
> > BY RFC
> > 6797
> > RANGER-3455 [Logout-Ranger] Should either be disabled/ should
> > redirect
> > to knox logout page
> > RANGER-3630 Support wildcards, group short names, and list of
> > memberof
> > attribute DNs for computing user search filter
> > RANGER-3597 User role should not be able to modify the Policy
> > RANGER-3512 Create Java patch to update policy guid to unique
> > value.
> > RANGER-3511 Create Java patch to update policy resource-signature
> > to
> > unique value.
> > RANGER-3493 Add unique index on service and resource_signature
> > column
> > of x_policy table
> > RANGER-3435 Add unique index on guid, service and zone_id column
> of
> > x_policy table
> > RANGER-3439 Add rest api to get or delete ranger policy based on
> > guid
> > RANGER-3498 RANGER : Remove log4j1 dependencies.
> > RANGER-3475 Promote TagRest endpoints to /public/v2
> > RANGER-3698 Ranger - Upgrade kylin to 3.1.3
> > 

Re: Review Request 73794: RANGER-3387: Ranger Admin Header Validation

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73794/
---

(Updated April 20, 2022, 6:02 a.m.)


Review request for ranger, Mateen Mansoori, Mehul Parikh, Pradeep Agrawal, and 
Ramesh Mani.


Changes
---

Added null checks as per the review comments


Bugs: RANGER-3387
https://issues.apache.org/jira/browse/RANGER-3387


Repository: ranger


Description
---

Added extra validation for handling PUT/POST requests coming from KNOX proxy 
with different CSRF header than the one set by the client.


Diffs (updated)
-

  
security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerCSRFPreventionFilter.java
 e02813fec 
  
security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKRBAuthenticationFilter.java
 2d3308b99 


Diff: https://reviews.apache.org/r/73794/diff/2/

Changes: https://reviews.apache.org/r/73794/diff/1-2/


Testing
---

1. Verified functionality with Knox trusted proxy on both HA and non-HA 
clusters.
2. Also verified couple of regression test cases.


Thanks,

Sailaja Polavarapu

Re: Review Request 73853: RANGER-3628 : Support fine grain authorization for different solr objects

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73853/#review224166
---


Ship it!




Ship It!

- Sailaja Polavarapu


On March 10, 2022, 11:15 a.m., Mateen Mansoori wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73853/
> ---
> 
> (Updated March 10, 2022, 11:15 a.m.)
> 
> 
> Review request for ranger, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, 
> Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3628
> https://issues.apache.org/jira/browse/RANGER-3628
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Modifying ranger solr plugin to allow granting the following privileges:
>   
>- QUERY - read only privilege on an object
>- UPDATE - write only privilege on an object
>- All - read and write access
> 
> Privileges can be defined on the following objects:
> 
> - admin
> - collections
> - cores
> - metrics
> - autoscaling
> - security
> - collection
> - config
> - schema
> 
> 
> Diffs
> -
> 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-solr.json 
> dfaa2f701 
>   
> plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuditHandler.java
>  359211cb2 
>   
> plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
>  d4dd7b0ec 
>   
> plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/SolrAuthzUtil.java
>  PRE-CREATION 
>   
> plugin-solr/src/main/java/org/apache/ranger/services/solr/RangerServiceSolr.java
>  97909ae54 
>   
> plugin-solr/src/main/java/org/apache/ranger/services/solr/RangerSolrConstants.java
>  PRE-CREATION 
>   
> plugin-solr/src/main/java/org/apache/ranger/services/solr/client/ServiceSolrClient.java
>  5f7b9b924 
> 
> 
> Diff: https://reviews.apache.org/r/73853/diff/4/
> 
> 
> Testing
> ---
> 
> Tested on cluster with by covering test cases as per new implementation.
> 
> 
> Thanks,
> 
> Mateen Mansoori
> 
>

[jira] [Updated] (RANGER-3659) Ranger Admin goes to OOM when usersync is trying to delete existing group mappings from ranger DB

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu updated RANGER-3659:
---
Attachment: 0001-RANGER-3659-Improved-create-update-or-delete-group-u.patch

> Ranger Admin goes to OOM when usersync is trying to delete existing group 
> mappings from ranger DB
> -
>
> Key: RANGER-3659
> URL: https://issues.apache.org/jira/browse/RANGER-3659
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Attachments: 
> 0001-RANGER-3659-Improved-create-update-or-delete-group-u.patch
>
>
> Ranger Admin goes to OOM when usersync is trying to delete existing group 
> mappings from ranger DB. 
> From heapdump the thread that is running OOM:
> {code:java}
> https-jsse-nio-6182-exec-3
> at java.lang.OutOfMemoryError.()V (OutOfMemoryError.java:48)
> at java.util.IdentityHashMap.resize(I)Z (IdentityHashMap.java:472)
> at 
> java.util.IdentityHashMap.put(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
>  (IdentityHashMap.java:441)
> at 
> org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.calculateChanges(Ljava/util/Map;Lorg/eclipse/persistence/internal/sessions/UnitOfWorkChangeSet;ZZ)Lorg/eclipse/persistence/internal/sessions/UnitOfWorkChangeSet;
>  (UnitOfWorkImpl.java:731)
> at 
> org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork.writeChanges()V
>  (RepeatableWriteUnitOfWork.java:443)
> at org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush()V 
> (EntityManagerImpl.java:975)
> at org.eclipse.persistence.internal.jpa.QueryImpl.performPreQueryFlush()V 
> (QueryImpl.java:976)
> at 
> org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery()Ljava/lang/Object;
>  (QueryImpl.java:211)
> at 
> org.eclipse.persistence.internal.jpa.QueryImpl.getSingleResult()Ljava/lang/Object;
>  (QueryImpl.java:530)
> at 
> org.eclipse.persistence.internal.jpa.EJBQueryImpl.getSingleResult()Ljava/lang/Object;
>  (EJBQueryImpl.java:404)
> at 
> org.apache.ranger.db.XXUserDao.findByUserName(Ljava/lang/String;)Lorg/apache/ranger/entity/XXUser;
>  (XXUserDao.java:52)
> at 
> org.apache.ranger.service.XUserService.getXUserByUserName(Ljava/lang/String;)Lorg/apache/ranger/view/VXUser;
>  (XUserService.java:150)
> at 
> org.apache.ranger.biz.XUserMgr.deleteXGroupAndXUser(Ljava/lang/String;Ljava/lang/String;)V
>  (XUserMgr.java:855)
> at 
> org.apache.ranger.biz.XUserMgr.createOrDeleteXGroupUsers(Lorg/apache/ranger/ugsyncutil/model/GroupUserInfo;Ljava/util/Map;)V
>  (XUserMgr.java:2835)
> at 
> org.apache.ranger.biz.XUserMgr.createOrDeleteXGroupUserList(Ljava/util/List;)I
>  (XUserMgr.java:2860)
> at 
> org.apache.ranger.biz.XUserMgr$$FastClassBySpringCGLIB$$57c6d473.invoke(ILjava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
>  (Unknown Source)
> at 
> org.springframework.cglib.proxy.MethodProxy.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
>  (MethodProxy.java:204)
> at 
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;Lorg/springframework/cglib/proxy/MethodProxy;)Ljava/lang/Object;
>  (CglibAopProxy.java:668)
> at 
> org.apache.ranger.biz.XUserMgr$$EnhancerBySpringCGLIB$$b2c8ac3b.createOrDeleteXGroupUserList(Ljava/util/List;)I
>  (Unknown Source)
> at 
> org.apache.ranger.rest.XUserREST.addOrUpdateGroupUsersList(Ljava/util/List;)I 
> (XUserREST.java:1464)
> at 
> org.apache.ranger.rest.XUserREST$$FastClassBySpringCGLIB$$b2a65360.invoke(ILjava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
>  (Unknown Source)
> at 
> org.springframework.cglib.proxy.MethodProxy.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
>  (MethodProxy.java:204)
> at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint()Ljava/lang/Object;
>  (CglibAopProxy.java:737){code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Review Request 73897: RANGER-3659: Ranger Admin goes to OOM when usersync is trying to delete existing group mappings from ranger DB

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73897/
---

Review request for ranger, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, 
Ramesh Mani, and Velmurugan Periasamy.


Bugs: RANGER-3659
https://issues.apache.org/jira/browse/RANGER-3659


Repository: ranger


Description
---

Improved create, update, or delete group user mappings API call to reduce 
memory usage in ranger admin. These APIs are called by usersync for updating 
group memberships from sync source to ranger admin. Also added code to use 
common DB transaction framework.


Diffs
-

  security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 8109968e4 
  security-admin/src/main/java/org/apache/ranger/service/XGroupUserService.java 
2b863b006 


Diff: https://reviews.apache.org/r/73897/diff/1/


Testing
---

Patched cluster and tested with ~200k group mappings in Ranger DB.
Also verified basic functionality for regression testing.


Thanks,

Sailaja Polavarapu

[jira] [Updated] (RANGER-3657) Support for recursive ACL check for subpaths in Ozone plugin

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu updated RANGER-3657:
---
Attachment: 0001-RANGER-3657-Recursive-delete-support-with-end-to-end.patch

> Support for recursive ACL check for subpaths in Ozone plugin
> 
>
> Key: RANGER-3657
> URL: https://issues.apache.org/jira/browse/RANGER-3657
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Attachments: 
> 0001-RANGER-3657-Recursive-delete-support-with-end-to-end.patch
>
>
> This task is to implement {{recursive}} ACL check in 
> {{{}RangerOzoneAuthorizer{}}}, using the below interface. OzoneRanger can 
> build an optimized recursive ACL check similar to HDFS - 
> [Reference|https://github.com/apache/ranger/blob/master/hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java#L444]
>  
> {code:java}
> IAccessAuthorizer#checkAccess(IOzoneObj ozoneObject, RequestContext context)
> public class RequestContext {
>  …...
>  /**
>   * Represents recursive access check required for all the sub-paths of 
> the
>   * given path. If the given path is not a directory, there is no effect 
> for
>   * this flag. A true value represents recursive check, false represents
>   * non-recursive check.
>   */
>   private final boolean recursiveAccessCheck; // introducing new 
> attribute for recursive access checks.
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Review Request 73896: RANGER-3657: Support for recursive ACL check for subpaths in Ozone plugin

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73896/
---

Review request for ranger, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, 
Ramesh Mani, and Velmurugan Periasamy.


Bugs: RANGER-3657
https://issues.apache.org/jira/browse/RANGER-3657


Repository: ranger


Description
---

Added code to support recursive delete operations for Ranger Ozone plugin 
including upgrade patch for service def changes. Also added plugin side change 
to retrieve owner information from the request context as an extention to 
https://reviews.apache.org/r/73051/
Updated Ozone version.


Diffs
-

  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/PolicyEngine.java
 f44570623 
  
agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java
 41ad8936d 
  
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java
 12fecd030 
  agents-common/src/main/resources/service-defs/ranger-servicedef-ozone.json 
4b899736b 
  plugin-ozone/pom.xml 264197acf 
  
plugin-ozone/src/main/java/org/apache/ranger/authorization/ozone/authorizer/RangerOzoneAuthorizer.java
 12e647ca3 
  
plugin-ozone/src/main/java/org/apache/ranger/services/ozone/RangerServiceOzone.java
 704412246 
  pom.xml 49a06411d 
  ranger-ozone-plugin-shim/pom.xml ab3dc4e3f 
  security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql b603f96cd 
  security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
c111a28f6 
  security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
854a2c676 
  
security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 
b45eace3b 
  security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
adec99857 
  security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyAdminImpl.java 
df75db11a 
  
security-admin/src/main/java/org/apache/ranger/patch/PatchForOzoneServiceDefUpdate_J10056.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/73896/diff/1/


Testing
---

Patched cluster and verified functionality with recursive delete operations and 
few regression tests.
Also verified manually upgrade patch changes.


Thanks,

Sailaja Polavarapu

[jira] [Assigned] (RANGER-3659) Ranger Admin goes to OOM when usersync is trying to delete existing group mappings from ranger DB

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu reassigned RANGER-3659:
--

Assignee: Sailaja Polavarapu

> Ranger Admin goes to OOM when usersync is trying to delete existing group 
> mappings from ranger DB
> -
>
> Key: RANGER-3659
> URL: https://issues.apache.org/jira/browse/RANGER-3659
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
>
> Ranger Admin goes to OOM when usersync is trying to delete existing group 
> mappings from ranger DB. 
> From heapdump the thread that is running OOM:
> {code:java}
> https-jsse-nio-6182-exec-3
> at java.lang.OutOfMemoryError.()V (OutOfMemoryError.java:48)
> at java.util.IdentityHashMap.resize(I)Z (IdentityHashMap.java:472)
> at 
> java.util.IdentityHashMap.put(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
>  (IdentityHashMap.java:441)
> at 
> org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.calculateChanges(Ljava/util/Map;Lorg/eclipse/persistence/internal/sessions/UnitOfWorkChangeSet;ZZ)Lorg/eclipse/persistence/internal/sessions/UnitOfWorkChangeSet;
>  (UnitOfWorkImpl.java:731)
> at 
> org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork.writeChanges()V
>  (RepeatableWriteUnitOfWork.java:443)
> at org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush()V 
> (EntityManagerImpl.java:975)
> at org.eclipse.persistence.internal.jpa.QueryImpl.performPreQueryFlush()V 
> (QueryImpl.java:976)
> at 
> org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery()Ljava/lang/Object;
>  (QueryImpl.java:211)
> at 
> org.eclipse.persistence.internal.jpa.QueryImpl.getSingleResult()Ljava/lang/Object;
>  (QueryImpl.java:530)
> at 
> org.eclipse.persistence.internal.jpa.EJBQueryImpl.getSingleResult()Ljava/lang/Object;
>  (EJBQueryImpl.java:404)
> at 
> org.apache.ranger.db.XXUserDao.findByUserName(Ljava/lang/String;)Lorg/apache/ranger/entity/XXUser;
>  (XXUserDao.java:52)
> at 
> org.apache.ranger.service.XUserService.getXUserByUserName(Ljava/lang/String;)Lorg/apache/ranger/view/VXUser;
>  (XUserService.java:150)
> at 
> org.apache.ranger.biz.XUserMgr.deleteXGroupAndXUser(Ljava/lang/String;Ljava/lang/String;)V
>  (XUserMgr.java:855)
> at 
> org.apache.ranger.biz.XUserMgr.createOrDeleteXGroupUsers(Lorg/apache/ranger/ugsyncutil/model/GroupUserInfo;Ljava/util/Map;)V
>  (XUserMgr.java:2835)
> at 
> org.apache.ranger.biz.XUserMgr.createOrDeleteXGroupUserList(Ljava/util/List;)I
>  (XUserMgr.java:2860)
> at 
> org.apache.ranger.biz.XUserMgr$$FastClassBySpringCGLIB$$57c6d473.invoke(ILjava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
>  (Unknown Source)
> at 
> org.springframework.cglib.proxy.MethodProxy.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
>  (MethodProxy.java:204)
> at 
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;Lorg/springframework/cglib/proxy/MethodProxy;)Ljava/lang/Object;
>  (CglibAopProxy.java:668)
> at 
> org.apache.ranger.biz.XUserMgr$$EnhancerBySpringCGLIB$$b2c8ac3b.createOrDeleteXGroupUserList(Ljava/util/List;)I
>  (Unknown Source)
> at 
> org.apache.ranger.rest.XUserREST.addOrUpdateGroupUsersList(Ljava/util/List;)I 
> (XUserREST.java:1464)
> at 
> org.apache.ranger.rest.XUserREST$$FastClassBySpringCGLIB$$b2a65360.invoke(ILjava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
>  (Unknown Source)
> at 
> org.springframework.cglib.proxy.MethodProxy.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
>  (MethodProxy.java:204)
> at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint()Ljava/lang/Object;
>  (CglibAopProxy.java:737){code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (RANGER-3659) Ranger Admin goes to OOM when usersync is trying to delete existing group mappings from ranger DB

[Sailaja Polavarapu (Jira)]

Sailaja Polavarapu created RANGER-3659:
--

 Summary: Ranger Admin goes to OOM when usersync is trying to 
delete existing group mappings from ranger DB
 Key: RANGER-3659
 URL: https://issues.apache.org/jira/browse/RANGER-3659
 Project: Ranger
  Issue Type: Improvement
  Components: Ranger
Reporter: Sailaja Polavarapu


Ranger Admin goes to OOM when usersync is trying to delete existing group 
mappings from ranger DB. 

>From heapdump the thread that is running OOM:
{code:java}
https-jsse-nio-6182-exec-3
at java.lang.OutOfMemoryError.()V (OutOfMemoryError.java:48)
at java.util.IdentityHashMap.resize(I)Z (IdentityHashMap.java:472)
at 
java.util.IdentityHashMap.put(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object;
 (IdentityHashMap.java:441)
at 
org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.calculateChanges(Ljava/util/Map;Lorg/eclipse/persistence/internal/sessions/UnitOfWorkChangeSet;ZZ)Lorg/eclipse/persistence/internal/sessions/UnitOfWorkChangeSet;
 (UnitOfWorkImpl.java:731)
at 
org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork.writeChanges()V
 (RepeatableWriteUnitOfWork.java:443)
at org.eclipse.persistence.internal.jpa.EntityManagerImpl.flush()V 
(EntityManagerImpl.java:975)
at org.eclipse.persistence.internal.jpa.QueryImpl.performPreQueryFlush()V 
(QueryImpl.java:976)
at 
org.eclipse.persistence.internal.jpa.QueryImpl.executeReadQuery()Ljava/lang/Object;
 (QueryImpl.java:211)
at 
org.eclipse.persistence.internal.jpa.QueryImpl.getSingleResult()Ljava/lang/Object;
 (QueryImpl.java:530)
at 
org.eclipse.persistence.internal.jpa.EJBQueryImpl.getSingleResult()Ljava/lang/Object;
 (EJBQueryImpl.java:404)
at 
org.apache.ranger.db.XXUserDao.findByUserName(Ljava/lang/String;)Lorg/apache/ranger/entity/XXUser;
 (XXUserDao.java:52)
at 
org.apache.ranger.service.XUserService.getXUserByUserName(Ljava/lang/String;)Lorg/apache/ranger/view/VXUser;
 (XUserService.java:150)
at 
org.apache.ranger.biz.XUserMgr.deleteXGroupAndXUser(Ljava/lang/String;Ljava/lang/String;)V
 (XUserMgr.java:855)
at 
org.apache.ranger.biz.XUserMgr.createOrDeleteXGroupUsers(Lorg/apache/ranger/ugsyncutil/model/GroupUserInfo;Ljava/util/Map;)V
 (XUserMgr.java:2835)
at 
org.apache.ranger.biz.XUserMgr.createOrDeleteXGroupUserList(Ljava/util/List;)I 
(XUserMgr.java:2860)
at 
org.apache.ranger.biz.XUserMgr$$FastClassBySpringCGLIB$$57c6d473.invoke(ILjava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
 (Unknown Source)
at 
org.springframework.cglib.proxy.MethodProxy.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
 (MethodProxy.java:204)
at 
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;Lorg/springframework/cglib/proxy/MethodProxy;)Ljava/lang/Object;
 (CglibAopProxy.java:668)
at 
org.apache.ranger.biz.XUserMgr$$EnhancerBySpringCGLIB$$b2c8ac3b.createOrDeleteXGroupUserList(Ljava/util/List;)I
 (Unknown Source)
at 
org.apache.ranger.rest.XUserREST.addOrUpdateGroupUsersList(Ljava/util/List;)I 
(XUserREST.java:1464)
at 
org.apache.ranger.rest.XUserREST$$FastClassBySpringCGLIB$$b2a65360.invoke(ILjava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
 (Unknown Source)
at 
org.springframework.cglib.proxy.MethodProxy.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
 (MethodProxy.java:204)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint()Ljava/lang/Object;
 (CglibAopProxy.java:737){code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu updated RANGER-3630:
---
Fix Version/s: 3.0.0

> Support wildcards, group short names, and list of memberof attribute DNs for 
> computing user search filter
> -
>
> Key: RANGER-3630
> URL: https://issues.apache.org/jira/browse/RANGER-3630
> Project: Ranger
>  Issue Type: New Feature
>  Components: Ranger, usersync
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: 
> 0001-RANGER-3630-Added-code-to-support-wildcards-group-sh.patch, 
> RANGER-3630_proposal.pdf
>
>
> Ranger Usersync provides multiple configuration properties to sync users & 
> groups from AD/LDAP. One of the key configuration properties is the User 
> Search filter (ranger.usersync.ldap.user.searchfilter). Currently, the value 
> of user search filter must be a valid ldap search filter and is used by 
> ranger usersync “as is” to limit the no. of users to be sync’d from AD/LDAP. 
> Example values include:
>  # samaccountname=*  
>  ** Syncs all users from a given user search base
>  # (|(memberof=CN=finance,ou=Hadoop 
> Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop 
> Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop 
> Groups,dc=apache,dc=org))
>  ** Sync users that are members of finance, eng_dev, and eng_testing groups
> According to [Microsoft 
> documentation|https://social.technet.microsoft.com/wiki/contents/articles/5392.active-directory-ldap-syntax-filters.aspx],
>  the wildcard character * is not allowed when the  is a DN 
> attribute. Examples of DN attributes are distinguishedName, manager, 
> directReports, member, and memberOf. If users need to be sync'd from multiple 
> Active Directory groups with memberOf filters, this value can quickly become 
> a long string of OR concatenated group DNs. A single misplaced character in 
> this cryptic string results in all users failing to sync. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Assigned] (RANGER-3657) Support for recursive ACL check for subpaths in Ozone plugin

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu reassigned RANGER-3657:
--

Assignee: Sailaja Polavarapu

> Support for recursive ACL check for subpaths in Ozone plugin
> 
>
> Key: RANGER-3657
> URL: https://issues.apache.org/jira/browse/RANGER-3657
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
>
> This task is to implement {{recursive}} ACL check in 
> {{{}RangerOzoneAuthorizer{}}}, using the below interface. OzoneRanger can 
> build an optimized recursive ACL check similar to HDFS - 
> [Reference|https://github.com/apache/ranger/blob/master/hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java#L444]
>  
> {code:java}
> IAccessAuthorizer#checkAccess(IOzoneObj ozoneObject, RequestContext context)
> public class RequestContext {
>  …...
>  /**
>   * Represents recursive access check required for all the sub-paths of 
> the
>   * given path. If the given path is not a directory, there is no effect 
> for
>   * this flag. A true value represents recursive check, false represents
>   * non-recursive check.
>   */
>   private final boolean recursiveAccessCheck; // introducing new 
> attribute for recursive access checks.
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (RANGER-3657) Support for recursive ACL check for subpaths in Ozone plugin

[Sailaja Polavarapu (Jira)]

Sailaja Polavarapu created RANGER-3657:
--

 Summary: Support for recursive ACL check for subpaths in Ozone 
plugin
 Key: RANGER-3657
 URL: https://issues.apache.org/jira/browse/RANGER-3657
 Project: Ranger
  Issue Type: Improvement
  Components: Ranger
Reporter: Sailaja Polavarapu


This task is to implement {{recursive}} ACL check in 
{{{}RangerOzoneAuthorizer{}}}, using the below interface. OzoneRanger can build 
an optimized recursive ACL check similar to HDFS - 
[Reference|https://github.com/apache/ranger/blob/master/hdfs-agent/src/main/java/org/apache/ranger/authorization/hadoop/RangerHdfsAuthorizer.java#L444]

 
{code:java}
IAccessAuthorizer#checkAccess(IOzoneObj ozoneObject, RequestContext context)

public class RequestContext {
 …...
 /**
  * Represents recursive access check required for all the sub-paths of the
  * given path. If the given path is not a directory, there is no effect for
  * this flag. A true value represents recursive check, false represents
  * non-recursive check.
  */
  private final boolean recursiveAccessCheck; // introducing new attribute 
for recursive access checks.
}{code}



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: Review Request 73677: RANGER-3469: Off-By-One Error in XUser Syncing

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73677/#review224145
---




ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
Line 1023 (original), 1024 (patched)
<https://reviews.apache.org/r/73677/#comment313076>

I think uploadedCount should be initialized to 0 instead of -1
Same for lines 1118 & 1204


- Sailaja Polavarapu


On Dec. 10, 2021, 4 p.m., David Mollitor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73677/
> ---
> 
> (Updated Dec. 10, 2021, 4 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-3285: Off-By-One Error in XUser Syncing
> 
> 
> Diffs
> -
> 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
>  60445043f 
> 
> 
> Diff: https://reviews.apache.org/r/73677/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> David Mollitor
> 
>

Re: Review Request 73867: RANGER-3630: Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter

[Sailaja Polavarapu]

> On March 3, 2022, 5:21 a.m., Pradeep Agrawal wrote:
> > ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapUserGroupBuilder.java
> > Lines 1282 (patched)
> > <https://reviews.apache.org/r/73867/diff/1/?file=2266547#file2266547line1284>
> >
> > Do we need to handle closing exception or it will be handled by own.

If there is an exception, currently we are passing to the caller and in this 
case, usersync process will exit. I kept this behavior intentionally as the 
exception is at the init stage.


- Sailaja


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73867/#review224118
---


On Feb. 25, 2022, 4:12 a.m., Sailaja Polavarapu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73867/
> ---
> 
> (Updated Feb. 25, 2022, 4:12 a.m.)
> 
> 
> Review request for ranger, Mateen Mansoori, Mehul Parikh, Pradeep Agrawal, 
> and Ramesh Mani.
> 
> 
> Bugs: RANGER-3630
> https://issues.apache.org/jira/browse/RANGER-3630
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Introduced new usersync configuration "ranger.usersync.ldap.groupnames" that 
> accepts ";" separated list of group names with wildcards, shortname, or DN 
> format. During startup of usersync added logic to read this configuration to 
> compute the user search filter. Also added new unit tests to cover some 
> functional and error cases.
> 
> 
> Diffs
> -
> 
>   
> ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapUserGroupBuilder.java
>  dae78e9f2 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/config/UserGroupSyncConfig.java
>  5f301651b 
>   ugsync/src/test/java/org/apache/ranger/usergroupsync/TestLdapUserGroup.java 
> 78bc56cd9 
> 
> 
> Diff: https://reviews.apache.org/r/73867/diff/1/
> 
> 
> Testing
> ---
> 
> 1. Patched cluster and ran some functional tests to verify the new logic
> 2. Ran few regression tests with AD/LDAP sync source
> 
> 
> Thanks,
> 
> Sailaja Polavarapu
> 
>

Re: Review Request 73854: RANGER-3629 : Handle solr permissions during upgrade

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73854/#review224101
---


Ship it!




Ship It!

- Sailaja Polavarapu


On Feb. 22, 2022, 4:22 p.m., Mateen Mansoori wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73854/
> ---
> 
> (Updated Feb. 22, 2022, 4:22 p.m.)
> 
> 
> Review request for ranger, Jayendra Parab, Mehul Parikh, Sailaja Polavarapu, 
> and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3629
> https://issues.apache.org/jira/browse/RANGER-3629
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Handling permissions during upgrades
> Since we are moving to a finer grained privilege model, a 100% mapping will 
> likely not be possible.
> 
> When a user has solr_admin access type privileges on all collections (*), we 
> are mapping it to:
> - admin=*->(QERY and UPDATE)
> - collection=*->(QERY and UPDATE)
> - schemas=*->(QERY and UPDATE)
> - configs=*->(QERY and UPDATE)
> 
> When a user has solr_admin access type on a particular collection 
> collection_name, we are mapping it to
> - collection=collection_name->(QERY and UPDATE)
> - schemas=collection_name->(QERY and UPDATE)
> 
> One should verify their permissions after the upgrade.
> 
> 
> Diffs
> -
> 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 
> b603f96cd 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
> c111a28f6 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
> 854a2c676 
>   
> security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
>  b45eace3b 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
> adec99857 
>   
> security-admin/src/main/java/org/apache/ranger/patch/PatchForSolrSvcDefAndPoliciesUpdate_J10055.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/73854/diff/1/
> 
> 
> Testing
> ---
> 
> Verified upgrade on local/cluser - policies are getting migrated as per 
> description.
> 
> 
> Thanks,
> 
> Mateen Mansoori
> 
>

Re: Review Request 73853: RANGER-3628 : Support fine grain authorization for different solr objects

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73853/#review224100
---




plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
Line 580 (original), 627 (patched)
<https://reviews.apache.org/r/73853/#comment313054>

Can you please check the indentation?



plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
Lines 648 (patched)
<https://reviews.apache.org/r/73853/#comment313055>

Can you please check the indentation?



plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
Line 601 (original), 660 (patched)
<https://reviews.apache.org/r/73853/#comment313056>

Same as above


- Sailaja Polavarapu


On Feb. 22, 2022, 4:26 p.m., Mateen Mansoori wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73853/
> ---
> 
> (Updated Feb. 22, 2022, 4:26 p.m.)
> 
> 
> Review request for ranger, Mehul Parikh, Sailaja Polavarapu, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: RANGER-3628
> https://issues.apache.org/jira/browse/RANGER-3628
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Modifying ranger solr plugin to allow granting the following privileges:
>   
>- QUERY - read only privilege on an object
>- UPDATE - write only privilege on an object
>- All - read and write access
> 
> Privileges can be defined on the following objects:
> 
> - admin
> - collections
> - cores
> - metrics
> - autoscaling
> - security
> - collection
> - config
> - schema
> 
> 
> Diffs
> -
> 
>   agents-common/src/main/resources/service-defs/ranger-servicedef-solr.json 
> dfaa2f701 
>   
> plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuditHandler.java
>  359211cb2 
>   
> plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
>  d4dd7b0ec 
>   
> plugin-solr/src/main/java/org/apache/ranger/authorization/solr/authorizer/SolrAuthzUtil.java
>  PRE-CREATION 
>   
> plugin-solr/src/main/java/org/apache/ranger/services/solr/RangerServiceSolr.java
>  97909ae54 
>   
> plugin-solr/src/main/java/org/apache/ranger/services/solr/RangerSolrConstants.java
>  PRE-CREATION 
>   
> plugin-solr/src/main/java/org/apache/ranger/services/solr/client/ServiceSolrClient.java
>  5f7b9b924 
> 
> 
> Diff: https://reviews.apache.org/r/73853/diff/1/
> 
> 
> Testing
> ---
> 
> Tested on cluster with by covering test cases as per new implementation.
> 
> 
> Thanks,
> 
> Mateen Mansoori
> 
>

[jira] [Updated] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu updated RANGER-3630:
---
Attachment: 0001-RANGER-3630-Added-code-to-support-wildcards-group-sh.patch

> Support wildcards, group short names, and list of memberof attribute DNs for 
> computing user search filter
> -
>
> Key: RANGER-3630
> URL: https://issues.apache.org/jira/browse/RANGER-3630
> Project: Ranger
>  Issue Type: New Feature
>  Components: Ranger, usersync
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Attachments: 
> 0001-RANGER-3630-Added-code-to-support-wildcards-group-sh.patch, 
> RANGER-3630_proposal.pdf
>
>
> Ranger Usersync provides multiple configuration properties to sync users & 
> groups from AD/LDAP. One of the key configuration properties is the User 
> Search filter (ranger.usersync.ldap.user.searchfilter). Currently, the value 
> of user search filter must be a valid ldap search filter and is used by 
> ranger usersync “as is” to limit the no. of users to be sync’d from AD/LDAP. 
> Example values include:
>  # samaccountname=*  
>  ** Syncs all users from a given user search base
>  # (|(memberof=CN=finance,ou=Hadoop 
> Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop 
> Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop 
> Groups,dc=apache,dc=org))
>  ** Sync users that are members of finance, eng_dev, and eng_testing groups
> According to [Microsoft 
> documentation|https://social.technet.microsoft.com/wiki/contents/articles/5392.active-directory-ldap-syntax-filters.aspx],
>  the wildcard character * is not allowed when the  is a DN 
> attribute. Examples of DN attributes are distinguishedName, manager, 
> directReports, member, and memberOf. If users need to be sync'd from multiple 
> Active Directory groups with memberOf filters, this value can quickly become 
> a long string of OR concatenated group DNs. A single misplaced character in 
> this cryptic string results in all users failing to sync. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Review Request 73867: RANGER-3630: Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73867/
---

Review request for ranger, Mateen Mansoori, Mehul Parikh, Pradeep Agrawal, and 
Ramesh Mani.


Bugs: RANGER-3630
https://issues.apache.org/jira/browse/RANGER-3630


Repository: ranger


Description
---

Introduced new usersync configuration "ranger.usersync.ldap.groupnames" that 
accepts ";" separated list of group names with wildcards, shortname, or DN 
format. During startup of usersync added logic to read this configuration to 
compute the user search filter. Also added new unit tests to cover some 
functional and error cases.


Diffs
-

  
ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapUserGroupBuilder.java
 dae78e9f2 
  
ugsync/src/main/java/org/apache/ranger/unixusersync/config/UserGroupSyncConfig.java
 5f301651b 
  ugsync/src/test/java/org/apache/ranger/usergroupsync/TestLdapUserGroup.java 
78bc56cd9 


Diff: https://reviews.apache.org/r/73867/diff/1/


Testing
---

1. Patched cluster and ran some functional tests to verify the new logic
2. Ran few regression tests with AD/LDAP sync source


Thanks,

Sailaja Polavarapu

Re: Review Request 73863: RANGER-3638: Solr Ranger document level security breaks solr if collection is reloaded

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73863/
---

(Updated Feb. 25, 2022, 1:56 a.m.)


Review request for ranger, Abhay Kulkarni, Mateen Mansoori, Mehul Parikh, and 
Ramesh Mani.


Changes
---

incorporated review comments


Bugs: RANGER-3638
https://issues.apache.org/jira/browse/RANGER-3638


Repository: ranger


Description
---

RangerSolrAuthorizer is a common implementation call for both SearchComponent 
(for Document level Authorization) and AuthorizationPlugin (for collection 
level Authorization). RangerSolrAuthorizer implementation close() shutdowns the 
plugin and should be avoided when the call is for SearchComponent. Added check 
to get the authorizer class name to determine if the call is for 
SearchComponent of for the authorization plugin.


Diffs (updated)
-

  
ranger-solr-plugin-shim/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
 dfa219670 


Diff: https://reviews.apache.org/r/73863/diff/3/

Changes: https://reviews.apache.org/r/73863/diff/2-3/


Testing
---

1. Patched cluster with the changes and verified the end to end functionality 
with and without Document level authorization
2. Verified basic functional tests and audits for regression testing.


Thanks,

Sailaja Polavarapu

Re: Review Request 73863: RANGER-3638: Solr Ranger document level security breaks solr if collection is reloaded

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73863/
---

(Updated Feb. 24, 2022, 2:31 p.m.)


Review request for ranger, Abhay Kulkarni, Mateen Mansoori, Mehul Parikh, and 
Ramesh Mani.


Changes
---

Placed comments before checking the authorizer class name


Bugs: RANGER-3638
https://issues.apache.org/jira/browse/RANGER-3638


Repository: ranger


Description
---

RangerSolrAuthorizer is a common implementation call for both SearchComponent 
(for Document level Authorization) and AuthorizationPlugin (for collection 
level Authorization). RangerSolrAuthorizer implementation close() shutdowns the 
plugin and should be avoided when the call is for SearchComponent. Added check 
to get the authorizer class name to determine if the call is for 
SearchComponent of for the authorization plugin.


Diffs (updated)
-

  
ranger-solr-plugin-shim/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
 dfa219670 


Diff: https://reviews.apache.org/r/73863/diff/2/

Changes: https://reviews.apache.org/r/73863/diff/1-2/


Testing
---

1. Patched cluster with the changes and verified the end to end functionality 
with and without Document level authorization
2. Verified basic functional tests and audits for regression testing.


Thanks,

Sailaja Polavarapu

[jira] [Updated] (RANGER-3638) Solr Ranger document level security breaks solr if collection is reloaded

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu updated RANGER-3638:
---
Attachment: 0001-RANGER-3638-Avoid-calling-rangerSolrAuthorizerImpl-c.patch

> Solr Ranger document level security breaks solr if collection is reloaded
> -
>
> Key: RANGER-3638
> URL: https://issues.apache.org/jira/browse/RANGER-3638
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 2.2.0
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Attachments: 
> 0001-RANGER-3638-Avoid-calling-rangerSolrAuthorizerImpl-c.patch
>
>
> Solr stops working if using a collection with document level security enabled 
> when:
> adding fields to collection schema
> making changes to collection config
> reloading collection
> This does not happens if the reload is done to the non document level 
> security collection, with other collection with document level security.
> -solr can no longer send ranger audits to solr-infra showing 403 errror from 
> solr-infra
> -solr can no longer save audits to hdfs showing issue with the hdfs sink.
> -solr answer 403 to any query to the collection with document level security 
> and other collections, admin commands too
> -zookeeper znodes warnings are seen on the logs.
> After solr restart all collections with and without document level security 
> are available again and answer queries.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Review Request 73863: RANGER-3638: Solr Ranger document level security breaks solr if collection is reloaded

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73863/
---

Review request for ranger, Abhay Kulkarni, Mateen Mansoori, Mehul Parikh, and 
Ramesh Mani.


Bugs: RANGER-3638
https://issues.apache.org/jira/browse/RANGER-3638


Repository: ranger


Description
---

RangerSolrAuthorizer is a common implementation call for both SearchComponent 
(for Document level Authorization) and AuthorizationPlugin (for collection 
level Authorization). RangerSolrAuthorizer implementation close() shutdowns the 
plugin and should be avoided when the call is for SearchComponent. Added check 
to get the authorizer class name to determine if the call is for 
SearchComponent of for the authorization plugin.


Diffs
-

  
ranger-solr-plugin-shim/src/main/java/org/apache/ranger/authorization/solr/authorizer/RangerSolrAuthorizer.java
 dfa219670 


Diff: https://reviews.apache.org/r/73863/diff/1/


Testing
---

1. Patched cluster with the changes and verified the end to end functionality 
with and without Document level authorization
2. Verified basic functional tests and audits for regression testing.


Thanks,

Sailaja Polavarapu

[jira] [Created] (RANGER-3638) Solr Ranger document level security breaks solr if collection is reloaded

[Sailaja Polavarapu (Jira)]

Sailaja Polavarapu created RANGER-3638:
--

 Summary: Solr Ranger document level security breaks solr if 
collection is reloaded
 Key: RANGER-3638
 URL: https://issues.apache.org/jira/browse/RANGER-3638
 Project: Ranger
  Issue Type: Bug
  Components: Ranger
Affects Versions: 2.2.0
Reporter: Sailaja Polavarapu


Solr stops working if using a collection with document level security enabled 
when:
adding fields to collection schema
making changes to collection config
reloading collection
This does not happens if the reload is done to the non document level security 
collection, with other collection with document level security.

-solr can no longer send ranger audits to solr-infra showing 403 errror from 
solr-infra
-solr can no longer save audits to hdfs showing issue with the hdfs sink.
-solr answer 403 to any query to the collection with document level security 
and other collections, admin commands too
-zookeeper znodes warnings are seen on the logs.

After solr restart all collections with and without document level security are 
available again and answer queries.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Assigned] (RANGER-3638) Solr Ranger document level security breaks solr if collection is reloaded

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu reassigned RANGER-3638:
--

Assignee: Sailaja Polavarapu

> Solr Ranger document level security breaks solr if collection is reloaded
> -
>
> Key: RANGER-3638
> URL: https://issues.apache.org/jira/browse/RANGER-3638
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 2.2.0
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
>
> Solr stops working if using a collection with document level security enabled 
> when:
> adding fields to collection schema
> making changes to collection config
> reloading collection
> This does not happens if the reload is done to the non document level 
> security collection, with other collection with document level security.
> -solr can no longer send ranger audits to solr-infra showing 403 errror from 
> solr-infra
> -solr can no longer save audits to hdfs showing issue with the hdfs sink.
> -solr answer 403 to any query to the collection with document level security 
> and other collections, admin commands too
> -zookeeper znodes warnings are seen on the logs.
> After solr restart all collections with and without document level security 
> are available again and answer queries.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu updated RANGER-3630:
---
Attachment: RANGER-3630_proposal.pdf

> Support wildcards, group short names, and list of memberof attribute DNs for 
> computing user search filter
> -
>
> Key: RANGER-3630
> URL: https://issues.apache.org/jira/browse/RANGER-3630
> Project: Ranger
>  Issue Type: New Feature
>  Components: Ranger, usersync
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Attachments: RANGER-3630_proposal.pdf
>
>
> Ranger Usersync provides multiple configuration properties to sync users & 
> groups from AD/LDAP. One of the key configuration properties is the User 
> Search filter (ranger.usersync.ldap.user.searchfilter). Currently, the value 
> of user search filter must be a valid ldap search filter and is used by 
> ranger usersync “as is” to limit the no. of users to be sync’d from AD/LDAP. 
> Example values include:
>  # samaccountname=*  
>  ** Syncs all users from a given user search base
>  # (|(memberof=CN=finance,ou=Hadoop 
> Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop 
> Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop 
> Groups,dc=apache,dc=org))
>  ** Sync users that are members of finance, eng_dev, and eng_testing groups
> According to [Microsoft 
> documentation|https://social.technet.microsoft.com/wiki/contents/articles/5392.active-directory-ldap-syntax-filters.aspx],
>  the wildcard character * is not allowed when the  is a DN 
> attribute. Examples of DN attributes are distinguishedName, manager, 
> directReports, member, and memberOf. If users need to be sync'd from multiple 
> Active Directory groups with memberOf filters, this value can quickly become 
> a long string of OR concatenated group DNs. A single misplaced character in 
> this cryptic string results in all users failing to sync. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter

[Sailaja Polavarapu (Jira)]

[ https://issues.apache.org/jira/browse/RANGER-3630 ]


Sailaja Polavarapu deleted comment on RANGER-3630:


was (Author: spolavarapu):
 *Proposal:*

Majority of the use cases to filter users using the “memberof” attribute fall 
into two categories:
 # Unique pattern for the group name  - example, “eng_dev” and “finance”
 # Group names with wildcard character - example, “eng_dev” and “eng_testing”

As noted down earlier, since Active directory doesn’t support either wildcards 
or short names of the groups with memberof attributes, Ranger usersync must be 
improved to generate user search filter internally by taking list of individual 
group names or group names with wildcard character.

Instead of configuring user search filter as one big string, a new 
configuration “ranger.usersync.ldap.groupnames” can be introduced for usersync. 
Values can be either list of DN of the groups, list of short name of the 
groups, or list of group names with wildcard character with ";" separated like 
below:
 # DN of the groups - "memberof=CN=finance,ou=Hadoop 
Groups,dc=apache,dc=org{color:#ff}{*};{*}{color:#172b4d}memberof=CN=eng_dev,ou=Hadoop
 
Groups,dc=apache,dc=org{color}{color}{color:#de350b}*;*{color}memberof=CN=eng_testing,ou=Hadoop
 Groups,dc=apache,dc=org"
 # Short names of the groups - 
"CN=finance{color:#de350b}*;*{color}CN=eng_dev{color:#de350b}*;*{color}CN=eng_testing"
 # Group names with wildcard character - 
CN=eng*{color:#de350b}*;*{color}CN=finance"

*Usersync Changes:*

Usersync reads these new configuration values and determines the format of the 
specified values as DN of the groups, short names of the groups, or group names 
with wildcard character.
 # Values specified as DN of the groups
 # In this case ranger usersync builds the user search filter by concatenating 
each DN with an OR (|) operator 
 # Example - (|(memberof=CN=finance,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop 
Groups,dc=apache,dc=org))

 # Values specified as short names of the groups or with wildcard character
 # In this case ranger usersync first contacts AD/LDAP server to retrieve the 
DN of the specified groups.
 # Build user search filter by prepending each DN with “memberof=” and 
concatenating with and OR(|) operator

Notes: 
 # This new configuration(ranger.usersync.ldap.groupnames) is read by usersync 
only when “ranger.usersync.ldap.user.searchfilter” configuration value is 
empty. 
 # When “ranger.usersync.ldap.user.searchfilter” configuration value is not 
empty, then usersync will ignore the value for 
“ranger.usersync.ldap.groupnames” configuration.
 # All the configured group names(ranger.usersync.ldap.groupnames) are 
concatenated with only OR (|) operator and are hardcoded for “memberof” 
attribute.

> Support wildcards, group short names, and list of memberof attribute DNs for 
> computing user search filter
> -
>
> Key: RANGER-3630
> URL: https://issues.apache.org/jira/browse/RANGER-3630
> Project: Ranger
>  Issue Type: New Feature
>      Components: Ranger, usersync
>    Reporter: Sailaja Polavarapu
>Assignee: Sailaja Polavarapu
>Priority: Major
>
> Ranger Usersync provides multiple configuration properties to sync users & 
> groups from AD/LDAP. One of the key configuration properties is the User 
> Search filter (ranger.usersync.ldap.user.searchfilter). Currently, the value 
> of user search filter must be a valid ldap search filter and is used by 
> ranger usersync “as is” to limit the no. of users to be sync’d from AD/LDAP. 
> Example values include:
>  # samaccountname=*  
>  ** Syncs all users from a given user search base
>  # (|(memberof=CN=finance,ou=Hadoop 
> Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop 
> Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop 
> Groups,dc=apache,dc=org))
>  ** Sync users that are members of finance, eng_dev, and eng_testing groups
> According to [Microsoft 
> documentation|https://social.technet.microsoft.com/wiki/contents/articles/5392.active-directory-ldap-syntax-filters.aspx],
>  the wildcard character * is not allowed when the  is a DN 
> attribute. Examples of DN attributes are distinguishedName, manager, 
> directReports, member, and memberOf. If users need to be sync'd from multiple 
> Active Directory groups with memberOf filters, this value can quickly become 
> a long string of OR concatenated group DNs. A single misplaced character in 
> this cryptic string results in all users failing to sync. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Comment Edited] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter

[Sailaja Polavarapu (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493992#comment-17493992
 ] 

Sailaja Polavarapu edited comment on RANGER-3630 at 2/17/22, 3:05 PM:
--

 *Proposal:*

Majority of the use cases to filter users using the “memberof” attribute fall 
into two categories:
 # Unique pattern for the group name  - example, “eng_dev” and “finance”
 # Group names with wildcard character - example, “eng_dev” and “eng_testing”

As noted down earlier, since Active directory doesn’t support either wildcards 
or short names of the groups with memberof attributes, Ranger usersync must be 
improved to generate user search filter internally by taking list of individual 
group names or group names with wildcard character.

Instead of configuring user search filter as one big string, a new 
configuration “ranger.usersync.ldap.groupnames” can be introduced for usersync. 
Values can be either list of DN of the groups, list of short name of the 
groups, or list of group names with wildcard character with ";" separated like 
below:
 # DN of the groups - "memberof=CN=finance,ou=Hadoop 
Groups,dc=apache,dc=org{color:#ff}*;* 
{color:#172b4d}memberof=CN=eng_dev,ou=Hadoop 
Groups,dc=apache,dc=org{color}{color}{color:#de350b}*;*{color}memberof=CN=eng_testing,ou=Hadoop
 Groups,dc=apache,dc=org"
# Short names of the groups - "CN=finance{*};{*}CN=eng_dev{*};{*}CN=eng_testing"
 # Group names with wildcard character - 
CN=eng*{color:#de350b}*;*{color}CN=finance"

*Usersync Changes:*

Usersync reads these new configuration values and determines the format of the 
specified values as DN of the groups, short names of the groups, or group names 
with wildcard character.
 # Values specified as DN of the groups
 # In this case ranger usersync builds the user search filter by concatenating 
each DN with an OR (|) operator 
 # Example - (|(memberof=CN=finance,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop 
Groups,dc=apache,dc=org))

 # Values specified as short names of the groups or with wildcard character
 # In this case ranger usersync first contacts AD/LDAP server to retrieve the 
DN of the specified groups.
 # Build user search filter by prepending each DN with “memberof=” and 
concatenating with and OR(|) operator

Notes: 
 # This new configuration(ranger.usersync.ldap.groupnames) is read by usersync 
only when “ranger.usersync.ldap.user.searchfilter” configuration value is 
empty. 
 # When “ranger.usersync.ldap.user.searchfilter” configuration value is not 
empty, then usersync will ignore the value for 
“ranger.usersync.ldap.groupnames” configuration.
 # All the configured group names(ranger.usersync.ldap.groupnames) are 
concatenated with only OR (|) operator and are hardcoded for “memberof” 
attribute.


was (Author: spolavarapu):
 *Proposal:*

Majority of the use cases to filter users using the “memberof” attribute fall 
into two categories:
 # Unique pattern for the group name  - example, “eng_dev” and “finance”
 # Group names with wildcard character - example, “eng_dev” and “eng_testing”

As noted down earlier, since Active directory doesn’t support either wildcards 
or short names of the groups with memberof attributes, Ranger usersync must be 
improved to generate user search filter internally by taking list of individual 
group names or group names with wildcard character.

Instead of configuring user search filter as one big string, a new 
configuration “ranger.usersync.ldap.groupnames” can be introduced for usersync. 
Values can be either list of DN of the groups, list of short name of the 
groups, or list of group names with wildcard character with ";" separated like 
below:
 # DN of the groups - "memberof=CN=finance,ou=Hadoop 
Groups,dc=apache,dc=org{color:#ff}{*};{*}{color:#172b4d}memberof=CN=eng_dev,ou=Hadoop
 Groups,dc=apache,dc=org{color}{color}{*};{*}memberof=CN=eng_testing,ou=Hadoop 
Groups,dc=apache,dc=org"
# Short names of the groups - 
"CN=finance{*};{*}CN=eng_dev{color:#ff}*;*{color}CN=eng_testing"
 # Group names with wildcard character - 
CN=eng*{color:#de350b}*;*{color}CN=finance"

*Usersync Changes:*

Usersync reads these new configuration values and determines the format of the 
specified values as DN of the groups, short names of the groups, or group names 
with wildcard character.
 # Values specified as DN of the groups
 # In this case ranger usersync builds the user search filter by concatenating 
each DN with an OR (|) operator 
 # Example - (|(memberof=CN=finance,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop 
Groups,dc=apache,dc=org))

 # Values specified as short names of the groups or with wildcard character
 # In this 

[jira] [Comment Edited] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter

[Sailaja Polavarapu (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493992#comment-17493992
 ] 

Sailaja Polavarapu edited comment on RANGER-3630 at 2/17/22, 3:05 PM:
--

 *Proposal:*

Majority of the use cases to filter users using the “memberof” attribute fall 
into two categories:
 # Unique pattern for the group name  - example, “eng_dev” and “finance”
 # Group names with wildcard character - example, “eng_dev” and “eng_testing”

As noted down earlier, since Active directory doesn’t support either wildcards 
or short names of the groups with memberof attributes, Ranger usersync must be 
improved to generate user search filter internally by taking list of individual 
group names or group names with wildcard character.

Instead of configuring user search filter as one big string, a new 
configuration “ranger.usersync.ldap.groupnames” can be introduced for usersync. 
Values can be either list of DN of the groups, list of short name of the 
groups, or list of group names with wildcard character with ";" separated like 
below:
 # DN of the groups - "memberof=CN=finance,ou=Hadoop 
Groups,dc=apache,dc=org{color:#ff}{*};{*}{color:#172b4d}memberof=CN=eng_dev,ou=Hadoop
 
Groups,dc=apache,dc=org{color}{color}{color:#de350b}*;*{color}memberof=CN=eng_testing,ou=Hadoop
 Groups,dc=apache,dc=org"
 # Short names of the groups - 
"CN=finance{color:#de350b}*;*{color}CN=eng_dev{color:#de350b}*;*{color}CN=eng_testing"
 # Group names with wildcard character - 
CN=eng*{color:#de350b}*;*{color}CN=finance"

*Usersync Changes:*

Usersync reads these new configuration values and determines the format of the 
specified values as DN of the groups, short names of the groups, or group names 
with wildcard character.
 # Values specified as DN of the groups
 # In this case ranger usersync builds the user search filter by concatenating 
each DN with an OR (|) operator 
 # Example - (|(memberof=CN=finance,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop 
Groups,dc=apache,dc=org))

 # Values specified as short names of the groups or with wildcard character
 # In this case ranger usersync first contacts AD/LDAP server to retrieve the 
DN of the specified groups.
 # Build user search filter by prepending each DN with “memberof=” and 
concatenating with and OR(|) operator

Notes: 
 # This new configuration(ranger.usersync.ldap.groupnames) is read by usersync 
only when “ranger.usersync.ldap.user.searchfilter” configuration value is 
empty. 
 # When “ranger.usersync.ldap.user.searchfilter” configuration value is not 
empty, then usersync will ignore the value for 
“ranger.usersync.ldap.groupnames” configuration.
 # All the configured group names(ranger.usersync.ldap.groupnames) are 
concatenated with only OR (|) operator and are hardcoded for “memberof” 
attribute.


was (Author: spolavarapu):
 *Proposal:*

Majority of the use cases to filter users using the “memberof” attribute fall 
into two categories:
 # Unique pattern for the group name  - example, “eng_dev” and “finance”
 # Group names with wildcard character - example, “eng_dev” and “eng_testing”

As noted down earlier, since Active directory doesn’t support either wildcards 
or short names of the groups with memberof attributes, Ranger usersync must be 
improved to generate user search filter internally by taking list of individual 
group names or group names with wildcard character.

Instead of configuring user search filter as one big string, a new 
configuration “ranger.usersync.ldap.groupnames” can be introduced for usersync. 
Values can be either list of DN of the groups, list of short name of the 
groups, or list of group names with wildcard character with ";" separated like 
below:
 # DN of the groups - "memberof=CN=finance,ou=Hadoop 
Groups,dc=apache,dc=org{color:#ff}*;* 
{color:#172b4d}memberof=CN=eng_dev,ou=Hadoop 
Groups,dc=apache,dc=org{color}{color}{color:#de350b}*;*{color}memberof=CN=eng_testing,ou=Hadoop
 Groups,dc=apache,dc=org"
# Short names of the groups - "CN=finance{*};{*}CN=eng_dev{*};{*}CN=eng_testing"
 # Group names with wildcard character - 
CN=eng*{color:#de350b}*;*{color}CN=finance"

*Usersync Changes:*

Usersync reads these new configuration values and determines the format of the 
specified values as DN of the groups, short names of the groups, or group names 
with wildcard character.
 # Values specified as DN of the groups
 # In this case ranger usersync builds the user search filter by concatenating 
each DN with an OR (|) operator 
 # Example - (|(memberof=CN=finance,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop 
Groups,dc=apache,dc=org))

 # Values specified as short names of the groups or wit

[jira] [Comment Edited] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter

[Sailaja Polavarapu (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493992#comment-17493992
 ] 

Sailaja Polavarapu edited comment on RANGER-3630 at 2/17/22, 3:04 PM:
--

 *Proposal:*

Majority of the use cases to filter users using the “memberof” attribute fall 
into two categories:
 # Unique pattern for the group name  - example, “eng_dev” and “finance”
 # Group names with wildcard character - example, “eng_dev” and “eng_testing”

As noted down earlier, since Active directory doesn’t support either wildcards 
or short names of the groups with memberof attributes, Ranger usersync must be 
improved to generate user search filter internally by taking list of individual 
group names or group names with wildcard character.

Instead of configuring user search filter as one big string, a new 
configuration “ranger.usersync.ldap.groupnames” can be introduced for usersync. 
Values can be either list of DN of the groups, list of short name of the 
groups, or list of group names with wildcard character with ";" separated like 
below:
 # DN of the groups - "memberof=CN=finance,ou=Hadoop 
Groups,dc=apache,dc=org{color:#ff}{*};{*}{color:#172b4d}memberof=CN=eng_dev,ou=Hadoop
 Groups,dc=apache,dc=org{color}{color}{*};{*}memberof=CN=eng_testing,ou=Hadoop 
Groups,dc=apache,dc=org"
# Short names of the groups - 
"CN=finance{*};{*}CN=eng_dev{color:#ff}*;*{color}CN=eng_testing"
 # Group names with wildcard character - 
CN=eng*{color:#de350b}*;*{color}CN=finance"

*Usersync Changes:*

Usersync reads these new configuration values and determines the format of the 
specified values as DN of the groups, short names of the groups, or group names 
with wildcard character.
 # Values specified as DN of the groups
 # In this case ranger usersync builds the user search filter by concatenating 
each DN with an OR (|) operator 
 # Example - (|(memberof=CN=finance,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop 
Groups,dc=apache,dc=org))

 # Values specified as short names of the groups or with wildcard character
 # In this case ranger usersync first contacts AD/LDAP server to retrieve the 
DN of the specified groups.
 # Build user search filter by prepending each DN with “memberof=” and 
concatenating with and OR(|) operator

Notes: 
 # This new configuration(ranger.usersync.ldap.groupnames) is read by usersync 
only when “ranger.usersync.ldap.user.searchfilter” configuration value is 
empty. 
 # When “ranger.usersync.ldap.user.searchfilter” configuration value is not 
empty, then usersync will ignore the value for 
“ranger.usersync.ldap.groupnames” configuration.
 # All the configured group names(ranger.usersync.ldap.groupnames) are 
concatenated with only OR (|) operator and are hardcoded for “memberof” 
attribute.


was (Author: spolavarapu):
 *Proposal:*

Majority of the use cases to filter users using the “memberof” attribute fall 
into two categories:
 # Unique pattern for the group name  - example, “eng_dev” and “finance”
 # Group names with wildcard character - example, “eng_dev” and “eng_testing”

As noted down earlier, since Active directory doesn’t support either wildcards 
or short names of the groups with memberof attributes, Ranger usersync must be 
improved to generate user search filter internally by taking list of individual 
group names or group names with wildcard character.

Instead of configuring user search filter as one big string, a new 
configuration “ranger.usersync.ldap.groupnames” can be introduced for usersync. 
Values can be either list of DN of the groups, list of short name of the 
groups, or list of group names with wildcard character with ";" separated like 
below:
 # DN of the groups - "memberof=CN=finance,ou=Hadoop 
Groups,dc=apache,dc=org{color:#FF}{*};{*}{color:#172b4d}memberof=CN=eng_dev,ou=Hadoop
 
Groups,dc=apache,dc=org{color}{color}{color:#FF}{*};{*}{color:#172b4d}memberof=CN=eng_testing,ou=Hadoop
 Groups,dc=apache,dc=org{color}{color}"
 # Short names of the groups - 
"CN=finance{color:#FF}*;*{color}CN=eng_dev{color:#FF}*;*{color}CN=eng_testing"
 # Group names with wildcard character - 
CN=eng*{color:#de350b}*;*{color}CN=finance"


*Usersync Changes:*

Usersync reads these new configuration values and determines the format of the 
specified values as DN of the groups, short names of the groups, or group names 
with wildcard character.
 # Values specified as DN of the groups
 # In this case ranger usersync builds the user search filter by concatenating 
each DN with an OR (|) operator 
 # Example - (|(memberof=CN=finance,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop 
Groups,dc=apache,dc=org))


 # Values specified a

[jira] [Commented] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter

[Sailaja Polavarapu (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17493992#comment-17493992
 ] 

Sailaja Polavarapu commented on RANGER-3630:


 *Proposal:*

Majority of the use cases to filter users using the “memberof” attribute fall 
into two categories:
 # Unique pattern for the group name  - example, “eng_dev” and “finance”
 # Group names with wildcard character - example, “eng_dev” and “eng_testing”

As noted down earlier, since Active directory doesn’t support either wildcards 
or short names of the groups with memberof attributes, Ranger usersync must be 
improved to generate user search filter internally by taking list of individual 
group names or group names with wildcard character.

Instead of configuring user search filter as one big string, a new 
configuration “ranger.usersync.ldap.groupnames” can be introduced for usersync. 
Values can be either list of DN of the groups, list of short name of the 
groups, or list of group names with wildcard character with ";" separated like 
below:
 # DN of the groups - "memberof=CN=finance,ou=Hadoop 
Groups,dc=apache,dc=org{color:#FF}{*};{*}{color:#172b4d}memberof=CN=eng_dev,ou=Hadoop
 
Groups,dc=apache,dc=org{color}{color}{color:#FF}{*};{*}{color:#172b4d}memberof=CN=eng_testing,ou=Hadoop
 Groups,dc=apache,dc=org{color}{color}"
 # Short names of the groups - 
"CN=finance{color:#FF}*;*{color}CN=eng_dev{color:#FF}*;*{color}CN=eng_testing"
 # Group names with wildcard character - 
CN=eng*{color:#de350b}*;*{color}CN=finance"


*Usersync Changes:*

Usersync reads these new configuration values and determines the format of the 
specified values as DN of the groups, short names of the groups, or group names 
with wildcard character.
 # Values specified as DN of the groups
 # In this case ranger usersync builds the user search filter by concatenating 
each DN with an OR (|) operator 
 # Example - (|(memberof=CN=finance,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop 
Groups,dc=apache,dc=org))


 # Values specified as short names of the groups or with wildcard character
 # In this case ranger usersync first contacts AD/LDAP server to retrieve the 
DN of the specified groups.
 # Build user search filter by prepending each DN with “memberof=” and 
concatenating with and OR(|) operator

Notes: 
 # This new configuration(ranger.usersync.ldap.groupnames) is read by usersync 
only when “ranger.usersync.ldap.user.searchfilter” configuration value is 
empty. 
 # When “ranger.usersync.ldap.user.searchfilter” configuration value is not 
empty, then usersync will ignore the value for 
“ranger.usersync.ldap.groupnames” configuration.
 # All the configured group names(ranger.usersync.ldap.groupnames) are 
concatenated with only OR (|) operator and are hardcoded for “memberof” 
attribute.

> Support wildcards, group short names, and list of memberof attribute DNs for 
> computing user search filter
> -
>
> Key: RANGER-3630
> URL: https://issues.apache.org/jira/browse/RANGER-3630
> Project: Ranger
>  Issue Type: New Feature
>      Components: Ranger, usersync
>    Reporter: Sailaja Polavarapu
>Assignee: Sailaja Polavarapu
>Priority: Major
>
> Ranger Usersync provides multiple configuration properties to sync users & 
> groups from AD/LDAP. One of the key configuration properties is the User 
> Search filter (ranger.usersync.ldap.user.searchfilter). Currently, the value 
> of user search filter must be a valid ldap search filter and is used by 
> ranger usersync “as is” to limit the no. of users to be sync’d from AD/LDAP. 
> Example values include:
>  # samaccountname=*  
>  ** Syncs all users from a given user search base
>  # (|(memberof=CN=finance,ou=Hadoop 
> Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop 
> Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop 
> Groups,dc=apache,dc=org))
>  ** Sync users that are members of finance, eng_dev, and eng_testing groups
> According to [Microsoft 
> documentation|https://social.technet.microsoft.com/wiki/contents/articles/5392.active-directory-ldap-syntax-filters.aspx],
>  the wildcard character * is not allowed when the  is a DN 
> attribute. Examples of DN attributes are distinguishedName, manager, 
> directReports, member, and memberOf. If users need to be sync'd from multiple 
> Active Directory groups with memberOf filters, this value can quickly become 
> a long string of OR concatenated group DNs. A single misplaced character in 
> this cryptic string results in all users failing to sync. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter

[Sailaja Polavarapu (Jira)]

Sailaja Polavarapu created RANGER-3630:
--

 Summary: Support wildcards, group short names, and list of 
memberof attribute DNs for computing user search filter
 Key: RANGER-3630
 URL: https://issues.apache.org/jira/browse/RANGER-3630
 Project: Ranger
  Issue Type: New Feature
  Components: Ranger, usersync
Reporter: Sailaja Polavarapu


Ranger Usersync provides multiple configuration properties to sync users & 
groups from AD/LDAP. One of the key configuration properties is the User Search 
filter (ranger.usersync.ldap.user.searchfilter). Currently, the value of user 
search filter must be a valid ldap search filter and is used by ranger usersync 
“as is” to limit the no. of users to be sync’d from AD/LDAP. 

Example values include:
 # samaccountname=*  
 ** Syncs all users from a given user search base
 # (|(memberof=CN=finance,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop 
Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop 
Groups,dc=apache,dc=org))
 ** Sync users that are members of finance, eng_dev, and eng_testing groups

According to [Microsoft 
documentation|https://social.technet.microsoft.com/wiki/contents/articles/5392.active-directory-ldap-syntax-filters.aspx],
 the wildcard character * is not allowed when the  is a DN 
attribute. Examples of DN attributes are distinguishedName, manager, 
directReports, member, and memberOf. If users need to be sync'd from multiple 
Active Directory groups with memberOf filters, this value can quickly become a 
long string of OR concatenated group DNs. A single misplaced character in this 
cryptic string results in all users failing to sync. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Assigned] (RANGER-3630) Support wildcards, group short names, and list of memberof attribute DNs for computing user search filter

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu reassigned RANGER-3630:
--

Assignee: Sailaja Polavarapu

> Support wildcards, group short names, and list of memberof attribute DNs for 
> computing user search filter
> -
>
> Key: RANGER-3630
> URL: https://issues.apache.org/jira/browse/RANGER-3630
> Project: Ranger
>  Issue Type: New Feature
>  Components: Ranger, usersync
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
>
> Ranger Usersync provides multiple configuration properties to sync users & 
> groups from AD/LDAP. One of the key configuration properties is the User 
> Search filter (ranger.usersync.ldap.user.searchfilter). Currently, the value 
> of user search filter must be a valid ldap search filter and is used by 
> ranger usersync “as is” to limit the no. of users to be sync’d from AD/LDAP. 
> Example values include:
>  # samaccountname=*  
>  ** Syncs all users from a given user search base
>  # (|(memberof=CN=finance,ou=Hadoop 
> Groups,dc=apache,dc=org)(memberof=CN=eng_dev,ou=Hadoop 
> Groups,dc=apache,dc=org)(memberof=CN=eng_testing,ou=Hadoop 
> Groups,dc=apache,dc=org))
>  ** Sync users that are members of finance, eng_dev, and eng_testing groups
> According to [Microsoft 
> documentation|https://social.technet.microsoft.com/wiki/contents/articles/5392.active-directory-ldap-syntax-filters.aspx],
>  the wildcard character * is not allowed when the  is a DN 
> attribute. Examples of DN attributes are distinguishedName, manager, 
> directReports, member, and memberOf. If users need to be sync'd from multiple 
> Active Directory groups with memberOf filters, this value can quickly become 
> a long string of OR concatenated group DNs. A single misplaced character in 
> this cryptic string results in all users failing to sync. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Re: Review Request 73793: RANGER-3403: Ranger usersync role based rules not working as expected

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73793/
---

(Updated Jan. 26, 2022, 11:53 p.m.)


Review request for ranger, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, 
Ramesh Mani, and Velmurugan Periasamy.


Changes
---

Incorporated review comments and fixed one regression case where order of roles 
is not honored properly


Bugs: RANGER-3403
https://issues.apache.org/jira/browse/RANGER-3403


Repository: ranger


Description
---

1. Introduced new usersync configuration to treat service users like 
"rangerusersync", "admin", "rangertagsync" that require "ROLE_SYS_ADMIN" 
privileges. 
2. Fixed issue where removing a role assignment rule from usersync 
configuration is not updating the role of the affected users.
3. Also fixed an issue where the role (other than “ROLE_USER”) assigned to an 
external user manually from Ranger UI, is not reset automatically when usersync 
is restarted.


Diffs (updated)
-

  security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 1c2ef52ef 
  security-admin/src/main/java/org/apache/ranger/db/XXPortalUserDao.java 
d383cf596 
  security-admin/src/main/resources/META-INF/jpa_named_queries.xml d25fdc020 
  security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java 
eb2d6b383 
  
ugsync-util/src/main/java/org/apache/ranger/ugsyncutil/model/UsersGroupRoleAssignments.java
 ce0b9dd8b 
  
ugsync/src/main/java/org/apache/ranger/unixusersync/config/UserGroupSyncConfig.java
 47beecf21 
  
ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
 693d90894 


Diff: https://reviews.apache.org/r/73793/diff/2/

Changes: https://reviews.apache.org/r/73793/diff/1-2/


Testing
---

1. Verified functional test cases with new configuration and existing role 
assignment configuration.
2. Verified few regression test cases with default options
3. Verified all the existing unit test cases pass.


Thanks,

Sailaja Polavarapu

[jira] [Assigned] (RANGER-3574) Group-role mapping in Ranger Admin doesn't work

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu reassigned RANGER-3574:
--

Assignee: Sailaja Polavarapu

> Group-role mapping in Ranger Admin doesn't work
> ---
>
> Key: RANGER-3574
> URL: https://issues.apache.org/jira/browse/RANGER-3574
> Project: Ranger
>  Issue Type: Bug
>  Components: admin
>Reporter: Andre Araujo
>    Assignee: Sailaja Polavarapu
>Priority: Major
>
> The intent of the Ranger Admin properties 
> {{{}ranger.ldap.group.searchbase{}}}, {{ranger.ldap.group.searchfilter}} and 
> {{ranger.ldap.group.roleattribute}} is to provide a mechanism to map the 
> user's LDAP groups to Ranger roles dynamically. For example, if the user 
> belongs to the LDAP group {{{}sys_admin{}}}, it will automatically be 
> assigned to the {{ROLE_SYS_ADMIN}} role in Ranger.
> The 
> [{{RangerAuthenticationProvider.getLdapAuthentication()}}|https://github.com/apache/ranger/blob/e9b1e1d5f2009e90c6bbf912d5039bdafe319a5c/security-admin/src/main/java/org/apache/ranger/security/handler/RangerAuthenticationProvider.java#L185]
>  and 
> [{{RangerAuthenticationProvider.getLdapBindAuthentication()}}|https://github.com/apache/ranger/blob/e9b1e1d5f2009e90c6bbf912d5039bdafe319a5c/security-admin/src/main/java/org/apache/ranger/security/handler/RangerAuthenticationProvider.java#L512]
>  methods define the parameters above and use them to correctly configure a 
> {{{}DefaultLdapAuthoritiesPopulator{}}}.
> The configured populator, though, is never used to fetch the 
> {{GrantedAuthorities}} from LDAP. The [{{getAuthorities()}} 
> method|https://github.com/apache/ranger/blob/e9b1e1d5f2009e90c6bbf912d5039bdafe319a5c/security-admin/src/main/java/org/apache/ranger/security/handler/RangerAuthenticationProvider.java#L639-L646]
>  simply returns the roles assigned in Ranger and completely ignores the 
> populator configured previously.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (RANGER-3387) Ranger Admin Header Validation.

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu updated RANGER-3387:
---
Attachment: 0001-RANGER-3387-Added-extra-validation-for-handling-PUT-.patch

> Ranger Admin Header Validation.
> ---
>
> Key: RANGER-3387
> URL: https://issues.apache.org/jira/browse/RANGER-3387
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 2.2.0
>Reporter: Mateen N Mansoori
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 
> 0001-RANGER-3387-Added-extra-validation-for-handling-PUT-.patch, 
> 0001-RANGER-3387-Ranger-Admin-Header-Validation.patch
>
>
> Ranger Admin Header Validation.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Review Request 73794: RANGER-3387: Ranger Admin Header Validation

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73794/
---

Review request for ranger, Mateen Mansoori, Mehul Parikh, Pradeep Agrawal, and 
Ramesh Mani.


Bugs: RANGER-3387
https://issues.apache.org/jira/browse/RANGER-3387


Repository: ranger


Description
---

Added extra validation for handling PUT/POST requests coming from KNOX proxy 
with different CSRF header than the one set by the client.


Diffs
-

  
security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerCSRFPreventionFilter.java
 254f225d4 
  
security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKRBAuthenticationFilter.java
 2d3308b99 


Diff: https://reviews.apache.org/r/73794/diff/1/


Testing
---

1. Verified functionality with Knox trusted proxy on both HA and non-HA 
clusters.
2. Also verified couple of regression test cases.


Thanks,

Sailaja Polavarapu

[jira] [Reopened] (RANGER-3387) Ranger Admin Header Validation.

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu reopened RANGER-3387:

  Assignee: Sailaja Polavarapu  (was: Mateen Mansoori)

Reopening as the fix needs to be updated to handle requests coming from Knox 
(as gateway proxy)

> Ranger Admin Header Validation.
> ---
>
> Key: RANGER-3387
> URL: https://issues.apache.org/jira/browse/RANGER-3387
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Affects Versions: 2.2.0
>Reporter: Mateen N Mansoori
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Attachments: 0001-RANGER-3387-Ranger-Admin-Header-Validation.patch
>
>
> Ranger Admin Header Validation.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (RANGER-3403) Ranger usersync role based rules not working as expected

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu updated RANGER-3403:
---
Attachment: 0001-RANGER-3403-Made-code-changes-to-handle-role-assignm.patch

> Ranger usersync role based rules not working as expected
> 
>
> Key: RANGER-3403
> URL: https://issues.apache.org/jira/browse/RANGER-3403
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger, usersync
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Attachments: 
> 0001-RANGER-3403-Fixed-issue-where-role-assignments-are-n.patch, 
> 0001-RANGER-3403-Made-code-changes-to-handle-role-assignm.patch
>
>
> When add a group in Ranger Usersync Group based role assignment Rules, all 
> members in the group will have the admin role, but if remove the group from 
> Ranger Usersync Group based role assignment Rules, the members will still be 
> admin. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Comment Edited] (RANGER-3403) Ranger usersync role based rules not working as expected

[Sailaja Polavarapu (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17412260#comment-17412260
 ] 

Sailaja Polavarapu edited comment on RANGER-3403 at 1/8/22, 12:13 AM:
--

Review request available at:

https://reviews.apache.org/r/73793/


was (Author: spolavarapu):
Review request available at:

https://reviews.apache.org/r/73575/

> Ranger usersync role based rules not working as expected
> 
>
> Key: RANGER-3403
> URL: https://issues.apache.org/jira/browse/RANGER-3403
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger, usersync
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Attachments: 
> 0001-RANGER-3403-Fixed-issue-where-role-assignments-are-n.patch, 
> 0001-RANGER-3403-Made-code-changes-to-handle-role-assignm.patch
>
>
> When add a group in Ranger Usersync Group based role assignment Rules, all 
> members in the group will have the admin role, but if remove the group from 
> Ranger Usersync Group based role assignment Rules, the members will still be 
> admin. 



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Review Request 73793: RANGER-3403: Ranger usersync role based rules not working as expected

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73793/
---

Review request for ranger, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, 
Ramesh Mani, and Velmurugan Periasamy.


Bugs: RANGER-3403
https://issues.apache.org/jira/browse/RANGER-3403


Repository: ranger


Description
---

1. Introduced new usersync configuration to treat service users like 
"rangerusersync", "admin", "rangertagsync" that require "ROLE_SYS_ADMIN" 
privileges. 
2. Fixed issue where removing a role assignment rule from usersync 
configuration is not updating the role of the affected users.
3. Also fixed an issue where the role (other than “ROLE_USER”) assigned to an 
external user manually from Ranger UI, is not reset automatically when usersync 
is restarted.


Diffs
-

  security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 0eb582c8e 
  security-admin/src/main/java/org/apache/ranger/db/XXPortalUserDao.java 
d383cf596 
  security-admin/src/main/resources/META-INF/jpa_named_queries.xml c19ec495e 
  security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java 
945bba48d 
  
ugsync-util/src/main/java/org/apache/ranger/ugsyncutil/model/UsersGroupRoleAssignments.java
 ce0b9dd8b 
  
ugsync/src/main/java/org/apache/ranger/unixusersync/config/UserGroupSyncConfig.java
 ea6b47cd0 
  
ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
 60445043f 


Diff: https://reviews.apache.org/r/73793/diff/1/


Testing
---

1. Verified functional test cases with new configuration and existing role 
assignment configuration.
2. Verified few regression test cases with default options
3. Verified all the existing unit test cases pass.


Thanks,

Sailaja Polavarapu

Re: Review Request 73677: RANGER-3469: Off-By-One Error in XUser Syncing

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73677/#review223819
---




ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
Line 932 (original), 932 (patched)
<https://reviews.apache.org/r/73677/#comment312880>

Also while going through the code, I noticed that similar check is missing 
for getGroups, getGroupUsers, and updateUserRoles. Can you please add?


- Sailaja Polavarapu


On Nov. 22, 2021, 9:40 p.m., David Mollitor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73677/
> ---
> 
> (Updated Nov. 22, 2021, 9:40 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-3285: Off-By-One Error in XUser Syncing
> 
> 
> Diffs
> -
> 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
>  60445043f 
> 
> 
> Diff: https://reviews.apache.org/r/73677/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> David Mollitor
> 
>

Re: Review Request 73677: RANGER-3469: Off-By-One Error in XUser Syncing

[Sailaja Polavarapu]

> On Nov. 22, 2021, 10:35 p.m., Abhishek  Kumar wrote:
> > ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
> > Lines 919 (patched)
> > 
> >
> > Why is max required here ?
> 
> David Mollitor wrote:
> Hello, I am not sure, but this was the behavior of the original code and 
> I did not want to change behavior in this way for this particular issue.
> 
> David Mollitor wrote:
> Any additional questions or thoughts on this matter?

David,
 I agree that this behavior is from the original code. you can drop this 
comment. But please make similar changes for getGroups, getGroupUsers, and 
updateUserRoles where needed as commented by Abhishek.


- Sailaja


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73677/#review223761
---


On Nov. 22, 2021, 9:40 p.m., David Mollitor wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73677/
> ---
> 
> (Updated Nov. 22, 2021, 9:40 p.m.)
> 
> 
> Review request for ranger.
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> RANGER-3285: Off-By-One Error in XUser Syncing
> 
> 
> Diffs
> -
> 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
>  60445043f 
> 
> 
> Diff: https://reviews.apache.org/r/73677/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> David Mollitor
> 
>

Re: Review Request 73713: RANGER-3468: When multiple Ranger tabs are opened, Some tabs are not redirecting to Knox Logout page

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73713/
---

(Updated Dec. 1, 2021, 10:34 p.m.)


Review request for ranger, Kishor Gollapalliwar, Abhay Kulkarni, Mehul Parikh, 
Ramesh Mani, and Velmurugan Periasamy.


Bugs: RANGER-3468
https://issues.apache.org/jira/browse/RANGER-3468


Repository: ranger


Description
---

Fixed an issue where inactivity timeout request is not handled properly when 
the requested sessionid is invalid. Also added Content-Type header with value 
as application/x-http-headers in the logout redirect response


Diffs (updated)
-

  
security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKRBAuthenticationFilter.java
 c0ff06ea9 


Diff: https://reviews.apache.org/r/73713/diff/2/

Changes: https://reviews.apache.org/r/73713/diff/1-2/


Testing
---

Patched and verified few tests for regressions.


Thanks,

Sailaja Polavarapu

Re: Review Request 73716: RANGER-3514: Java patch to update sync source on upgrades

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73716/#review223758
---


Ship it!




- Sailaja Polavarapu


On Nov. 19, 2021, 8:03 a.m., Abhishek  Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73716/
> ---
> 
> (Updated Nov. 19, 2021, 8:03 a.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, 
> Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3514
> https://issues.apache.org/jira/browse/RANGER-3514
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> This patch updates the sync source field in later releases to values obtained 
> from otherAttributes in previous releases in a upgrade so that sync source is 
> consistent.
> It also updates and marks users/groups as external if there the user/group 
> contains otherAttribues.
> 
> 
> Diffs
> -
> 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 
> 9e2892a1c 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
> 65f9ad2f6 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
> a5bcf488b 
>   
> security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
>  a48f2348c 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
> 5cbe6d5a7 
>   
> security-admin/src/main/java/org/apache/ranger/patch/PatchForSyncSourceUpdate_J10054.java
>  PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/73716/diff/1/
> 
> 
> Testing
> ---
> 
> Tested for upgrades on remote cluster.
> 
> 
> Thanks,
> 
> Abhishek  Kumar
> 
>

[jira] [Updated] (RANGER-3468) When multiple Ranger tabs are opened, Some tabs are not redirecting to Knox Logout page

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu updated RANGER-3468:
---
Attachment: 0001-RANGER-3468-Fixed-an-issue-where-inactivity-timeout-.patch

> When multiple Ranger tabs are opened, Some tabs are not redirecting to Knox 
> Logout page 
> 
>
> Key: RANGER-3468
> URL: https://issues.apache.org/jira/browse/RANGER-3468
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Attachments: 
> 0001-RANGER-3468-Fixed-an-issue-where-inactivity-timeout-.patch
>
>
> When multiple Ranger tabs are open and session timeout happens some tabs are 
> not getting redirected to Knox logout page but we see new session has been 
> created
> Steps
> 1.Open 3 tabs of Ranger
> 2.Wait for idle session timeout
> 3.When checked all the tabs some tabs are not redirected to knox logout page.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

Review Request 73713: RANGER-3468: When multiple Ranger tabs are opened, Some tabs are not redirecting to Knox Logout page

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73713/
---

Review request for ranger, Kishor Gollapalliwar, Abhay Kulkarni, Mehul Parikh, 
Ramesh Mani, and Velmurugan Periasamy.


Bugs: RANGER-3468
https://issues.apache.org/jira/browse/RANGER-3468


Repository: ranger


Description
---

Fixed an issue where inactivity timeout request is not handled properly when 
the requested sessionid is invalid. Also added Content-Type header with value 
as application/x-http-headers in the logout redirect response


Diffs
-

  
security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKRBAuthenticationFilter.java
 c0ff06ea9 


Diff: https://reviews.apache.org/r/73713/diff/1/


Testing
---

Patched and verified few tests for regressions.


Thanks,

Sailaja Polavarapu

[jira] [Updated] (RANGER-3503) Make "hadoop.security.authorization" config as an optional config in cm_ozone service definition

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu updated RANGER-3503:
---
Fix Version/s: 3.0.0

> Make "hadoop.security.authorization" config as an optional config in cm_ozone 
> service definition
> 
>
> Key: RANGER-3503
> URL: https://issues.apache.org/jira/browse/RANGER-3503
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Sailaja Polavarapu
>Assignee: Sailaja Polavarapu
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: 
> 0001-RANGER-3503-Updated-Ozone-servicedef-to-make-hadoop..patch
>
>
> Today, the "hadoop.security.authorization" config is defined as mandatory in 
> cm_ozone service def, which prevents Ozone Ranger plugin to be setup in a 
> pure Ozone cluster (without HDFS) when "hadoop.security.authorization" is set 
> to empty.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Updated] (RANGER-3505) Ranger usersync fails to sync users when a duplicate user exists in ranger

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu updated RANGER-3505:
---
Attachment: 0001-RANGER-3505-modified-code-to-ignore-case-while-valid.patch

> Ranger usersync fails to sync users when a duplicate user exists in ranger
> --
>
> Key: RANGER-3505
> URL: https://issues.apache.org/jira/browse/RANGER-3505
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger, usersync
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Attachments: 
> 0001-RANGER-3505-modified-code-to-ignore-case-while-valid.patch
>
>
> Ranger is having issue with syncing user with case sensitive when the backend 
> database for ranger is MySQL / Maria DB. Initially users are sync'd with 
> default configuration (without any case conversion). Later usersync is 
> configured to sync users with upper case. At this time, usersync fails to 
> sync any users throwing duplicate user exception. For resolving this, 
> existing user from ranger must be deleted and then restarted the usersync 
> service. Instead, ranger code should update existing user with case 
> conversion and continue.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Review Request 73685: RANGER-3505: Ranger usersync fails to sync users when a duplicate user exists in ranger

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73685/
---

Review request for ranger, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, 
Ramesh Mani, and Velmurugan Periasamy.


Bugs: RANGER-3505
https://issues.apache.org/jira/browse/RANGER-3505


Repository: ranger


Description
---

modified code to ignore case while validating a user for update


Diffs
-

  security-admin/src/main/java/org/apache/ranger/service/XUserService.java 
f22bcec75 


Diff: https://reviews.apache.org/r/73685/diff/1/


Testing
---

Patched cluster with Mysql DB as backed for ranger and verified few test cases.
Also verified with postgres DB as backend for ranger for quick regression tests.


Thanks,

Sailaja Polavarapu

[jira] [Created] (RANGER-3505) Ranger usersync fails to sync users when a duplicate user exists in ranger

[Sailaja Polavarapu (Jira)]

Sailaja Polavarapu created RANGER-3505:
--

 Summary: Ranger usersync fails to sync users when a duplicate user 
exists in ranger
 Key: RANGER-3505
 URL: https://issues.apache.org/jira/browse/RANGER-3505
 Project: Ranger
  Issue Type: Bug
  Components: Ranger, usersync
Reporter: Sailaja Polavarapu


Ranger is having issue with syncing user with case sensitive when the backend 
database for ranger is MySQL / Maria DB. Initially users are sync'd with 
default configuration (without any case conversion). Later usersync is 
configured to sync users with upper case. At this time, usersync fails to sync 
any users throwing duplicate user exception. For resolving this, existing user 
from ranger must be deleted and then restarted the usersync service. Instead, 
ranger code should update existing user with case conversion and continue.

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (RANGER-3505) Ranger usersync fails to sync users when a duplicate user exists in ranger

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu reassigned RANGER-3505:
--

Assignee: Sailaja Polavarapu

> Ranger usersync fails to sync users when a duplicate user exists in ranger
> --
>
> Key: RANGER-3505
> URL: https://issues.apache.org/jira/browse/RANGER-3505
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger, usersync
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
>
> Ranger is having issue with syncing user with case sensitive when the backend 
> database for ranger is MySQL / Maria DB. Initially users are sync'd with 
> default configuration (without any case conversion). Later usersync is 
> configured to sync users with upper case. At this time, usersync fails to 
> sync any users throwing duplicate user exception. For resolving this, 
> existing user from ranger must be deleted and then restarted the usersync 
> service. Instead, ranger code should update existing user with case 
> conversion and continue.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (RANGER-3503) Make "hadoop.security.authorization" config as an optional config in cm_ozone service definition

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu updated RANGER-3503:
---
Attachment: 0001-RANGER-3503-Updated-Ozone-servicedef-to-make-hadoop..patch

> Make "hadoop.security.authorization" config as an optional config in cm_ozone 
> service definition
> 
>
> Key: RANGER-3503
> URL: https://issues.apache.org/jira/browse/RANGER-3503
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>Reporter: Sailaja Polavarapu
>Assignee: Sailaja Polavarapu
>Priority: Major
> Attachments: 
> 0001-RANGER-3503-Updated-Ozone-servicedef-to-make-hadoop..patch
>
>
> Today, the "hadoop.security.authorization" config is defined as mandatory in 
> cm_ozone service def, which prevents Ozone Ranger plugin to be setup in a 
> pure Ozone cluster (without HDFS) when "hadoop.security.authorization" is set 
> to empty.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Review Request 73678: RANGER-3503: Make "hadoop.security.authorization" config as an optional config in cm_ozone service definition

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73678/
---

Review request for ranger, Abhay Kulkarni, Mehul Parikh, Pradeep Agrawal, 
Ramesh Mani, and Velmurugan Periasamy.


Bugs: RANGER-3503
https://issues.apache.org/jira/browse/RANGER-3503


Repository: ranger


Description
---

Updated Ozone servicedef to make hadoop.security.authorization config an 
optional property. Also added corresponding upgrade patch.


Diffs
-

  agents-common/src/main/resources/service-defs/ranger-servicedef-ozone.json 
7bbfeeb7b 
  security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql c9cb79b73 
  security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
581ffe381 
  security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
960681a22 
  
security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 
079574985 
  security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
5048ea901 
  
security-admin/src/main/java/org/apache/ranger/patch/PatchForOzoneServiceDefConfigUpdate_J10051.java
 PRE-CREATION 


Diff: https://reviews.apache.org/r/73678/diff/1/


Testing
---

1. Verified manually updating the service def with the changes.
2. Also patched cluster with upgrade patch and verified the patch is applied 
successfuly during initial ranger setup.


Thanks,

Sailaja Polavarapu

[jira] [Created] (RANGER-3503) Make "hadoop.security.authorization" config as an optional config in cm_ozone service definition

[Sailaja Polavarapu (Jira)]

Sailaja Polavarapu created RANGER-3503:
--

 Summary: Make "hadoop.security.authorization" config as an 
optional config in cm_ozone service definition
 Key: RANGER-3503
 URL: https://issues.apache.org/jira/browse/RANGER-3503
 Project: Ranger
  Issue Type: Bug
  Components: Ranger
Reporter: Sailaja Polavarapu
Assignee: Sailaja Polavarapu


Today, the "hadoop.security.authorization" config is defined as mandatory in 
cm_ozone service def, which prevents Ozone Ranger plugin to be setup in a pure 
Ozone cluster (without HDFS) when "hadoop.security.authorization" is set to 
empty.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Resolved] (RANGER-3319) Ranger usersync cookie default duration for sync

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu resolved RANGER-3319.

Fix Version/s: 2.2.0
   Resolution: Fixed

> Ranger usersync cookie default duration for sync
> 
>
> Key: RANGER-3319
> URL: https://issues.apache.org/jira/browse/RANGER-3319
> Project: Ranger
>  Issue Type: Bug
>  Components: usersync
>Affects Versions: 2.1.0, 2.0.1, 2.2.0, 2.1.1
>Reporter: Konstantin Tsypin
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Fix For: 2.2.0
>
>
> Hi!
> At this moment we cant initial sync out of the box our LDAP with 10k+ users & 
> groups.
> It's because sync works as three steps:
> 1) Sync groups
> 2) Sync users
> 3) Map users as one request and send it to rangeradmin
> From time to time our third step on initial sync generate this single request 
> for a long time
> It can be easily three or four hours.
> Acrossing this timegate we have an error with timeout usersync cookie (that 
> by default is 60 minutes) and failed 3rd step.
>  
> The workaround - is 
> ranger_admin_directory/ews/webapp/WEB-INF/web.xml
> change 
> default 
> 60 
> to just in case
> 1440
> BUT im was really frustrated with this behavior whan faced it first time, and 
> i want to have a mechanism to split mapping step for a smaller part, and 
> update cookie from time to time. 
>  
> Thank you.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Assigned] (RANGER-3319) Ranger usersync cookie default duration for sync

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu reassigned RANGER-3319:
--

Assignee: Sailaja Polavarapu

> Ranger usersync cookie default duration for sync
> 
>
> Key: RANGER-3319
> URL: https://issues.apache.org/jira/browse/RANGER-3319
> Project: Ranger
>  Issue Type: Bug
>  Components: usersync
>Affects Versions: 2.1.0, 2.0.1, 2.2.0, 2.1.1
>Reporter: Konstantin Tsypin
>    Assignee: Sailaja Polavarapu
>Priority: Major
>
> Hi!
> At this moment we cant initial sync out of the box our LDAP with 10k+ users & 
> groups.
> It's because sync works as three steps:
> 1) Sync groups
> 2) Sync users
> 3) Map users as one request and send it to rangeradmin
> From time to time our third step on initial sync generate this single request 
> for a long time
> It can be easily three or four hours.
> Acrossing this timegate we have an error with timeout usersync cookie (that 
> by default is 60 minutes) and failed 3rd step.
>  
> The workaround - is 
> ranger_admin_directory/ews/webapp/WEB-INF/web.xml
> change 
> default 
> 60 
> to just in case
> 1440
> BUT im was really frustrated with this behavior whan faced it first time, and 
> i want to have a mechanism to split mapping step for a smaller part, and 
> update cookie from time to time. 
>  
> Thank you.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Comment Edited] (RANGER-3319) Ranger usersync cookie default duration for sync

[Sailaja Polavarapu (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434898#comment-17434898
 ] 

Sailaja Polavarapu edited comment on RANGER-3319 at 10/27/21, 3:08 PM:
---

[~fullhouse],

>> 3) Map users as one request and send it to rangeradmin

>From time to time our third step on initial sync generate this single request 
>for a long time

It can be easily three or four hours.

>>>

This issue is fixed as part of RANGER-3374

 Please give it a try and re-open if the issue still exists.


was (Author: spolavarapu):
[~fullhouse],

>> 3) Map users as one request and send it to rangeradmin

>From time to time our third step on initial sync generate this single request 
>for a long time

It can be easily three or four hours.

>>>

This issue is fixed as part of RANGER-3374

 

> Ranger usersync cookie default duration for sync
> 
>
> Key: RANGER-3319
> URL: https://issues.apache.org/jira/browse/RANGER-3319
> Project: Ranger
>  Issue Type: Bug
>  Components: usersync
>Affects Versions: 2.1.0, 2.0.1, 2.2.0, 2.1.1
>Reporter: Konstantin Tsypin
>Priority: Major
>
> Hi!
> At this moment we cant initial sync out of the box our LDAP with 10k+ users & 
> groups.
> It's because sync works as three steps:
> 1) Sync groups
> 2) Sync users
> 3) Map users as one request and send it to rangeradmin
> From time to time our third step on initial sync generate this single request 
> for a long time
> It can be easily three or four hours.
> Acrossing this timegate we have an error with timeout usersync cookie (that 
> by default is 60 minutes) and failed 3rd step.
>  
> The workaround - is 
> ranger_admin_directory/ews/webapp/WEB-INF/web.xml
> change 
> default 
> 60 
> to just in case
> 1440
> BUT im was really frustrated with this behavior whan faced it first time, and 
> i want to have a mechanism to split mapping step for a smaller part, and 
> update cookie from time to time. 
>  
> Thank you.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (RANGER-3319) Ranger usersync cookie default duration for sync

[Sailaja Polavarapu (Jira)]

[ 
https://issues.apache.org/jira/browse/RANGER-3319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17434898#comment-17434898
 ] 

Sailaja Polavarapu commented on RANGER-3319:


[~fullhouse],

>> 3) Map users as one request and send it to rangeradmin

>From time to time our third step on initial sync generate this single request 
>for a long time

It can be easily three or four hours.

>>>

This issue is fixed as part of RANGER-3374

 

> Ranger usersync cookie default duration for sync
> 
>
> Key: RANGER-3319
> URL: https://issues.apache.org/jira/browse/RANGER-3319
> Project: Ranger
>  Issue Type: Bug
>  Components: usersync
>Affects Versions: 2.1.0, 2.0.1, 2.2.0, 2.1.1
>Reporter: Konstantin Tsypin
>Priority: Major
>
> Hi!
> At this moment we cant initial sync out of the box our LDAP with 10k+ users & 
> groups.
> It's because sync works as three steps:
> 1) Sync groups
> 2) Sync users
> 3) Map users as one request and send it to rangeradmin
> From time to time our third step on initial sync generate this single request 
> for a long time
> It can be easily three or four hours.
> Acrossing this timegate we have an error with timeout usersync cookie (that 
> by default is 60 minutes) and failed 3rd step.
>  
> The workaround - is 
> ranger_admin_directory/ews/webapp/WEB-INF/web.xml
> change 
> default 
> 60 
> to just in case
> 1440
> BUT im was really frustrated with this behavior whan faced it first time, and 
> i want to have a mechanism to split mapping step for a smaller part, and 
> update cookie from time to time. 
>  
> Thank you.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [VOTE] Release Apache Ranger version 2.2.0 - rc2

[Sailaja Polavarapu]

+1
Thanks Ramesh for putting Apache ranger 2.2.0 rc2.
- Verified pgp signature & hashes.
- Verified clean build with unit test and packaging
- Verified usersync and ranger admin started successfully

Thanks,
Sailaja.

On Tue, Oct 26, 2021 at 8:54 AM vishal suvagia
 wrote:

> Thank-you Ramesh for putting the Apache Ranger 2.2.0 release candidate #2.
>
> +1 for the Apache Ranger 2.2.0 release candidate #2.
> clean build Ranger from the source tar succeeded without any issues.
> verified hashes and signature matching the source file.
> verified that Ranger Admin portal came up fine.
>
>
> Thanks
> Vishal Suvagia.
> --
>
> On Tuesday, 26 October, 2021, 11:39:33 am IST, Mehul Parikh <
> xsme...@gmail.com> wrote:
>
>  +1 for Apache Ranger 2.2.0 rc2
>
>
>   - Verified signatures
>   - Built ranger-2.2-rc2 from sources successfully
>   - Started Ranger-admin successfully
>   - Verified policy download by hdfs / yarn / hbase / hive / kafka /
>   plugins.
>
>
> On Mon, Oct 25, 2021 at 3:34 AM Abhay Kulkarni  wrote:
>
> > +1
> >
> > - Verified pgp, sha signatures
> > - Built ranger-2.2-rc2 from sources successfully
> > - Started Ranger-admin successfully
> > - Verified that the changes to tag policies are downloaded to
> > hdfs/hbase/hive/kafka/yarn plugins
> >
> > Thanks!
> > -Abhay
> >
> > On Sat, Oct 23, 2021 at 5:31 PM Madhan Neethiraj 
> > wrote:
> > >
> > > +1 for Apache Ranger 2.2.0 rc2
> > >
> > >  - verified signature
> > >  - verified 2.2.0-rc2 builds successfully
> > >  - verified Ranger admin startup; created services/policies/security
> > zones
> > >  - sanity testing of HDFS/Hive/HBase/Kafka/YARN plugins
> > >  - verified audit logs from plugins, audit-filters
> > >
> > > Thank you Ramesh for putting this release together.
> > >
> > > Thanks,
> > > Madhan
> > >
> > >
> > > On 10/23/21, 12:33 AM, "Ramesh Mani"  wrote:
> > >
> > >Dear Rangers,
> > >
> > >Apache Ranger 2.2.0 release candidate #2 is now available for a vote
> > within
> > >the dev community. Links to the release artifacts are given below.
> > Please
> > >review and vote.
> > >
> > >The vote will be open for at least 72 hours or until necessary votes
> > are
> > >reached.
> > >  [ ] +1 approve
> > >  [ ] +0 no opinion
> > >  [ ] -1 disapprove (and reason why)
> > >
> > >Thanks,
> > >Ramesh
> > >
> > >List of all the issues addressed in this release:
> > >https://issues.apache.org/jira/issues/?jql=project=RANGER AND
> > >status=Resolved AND fixVersion=2.2.0 ORDER BY key DESC
> > >
> > >Git tag for the release:
> > >https://github.com/apache/ranger/tree/release-2.2.0-rc2
> > >
> > >Sources for the release:
> > >
> >
> https://dist.apache.org/repos/dist/dev/ranger/2.2.0-rc2/apache-ranger-2.2.0.tar.gz
> > >
> > >Source release verification:
> > >  PGP Signature:
> > >
> >
> https://dist.apache.org/repos/dist/dev/ranger/2.2.0-rc2/apache-ranger-2.2.0.tar.gz.asc
> > >  SHA256 Hash:
> > >
> >
> https://dist.apache.org/repos/dist/dev/ranger/2.2.0-rc2/apache-ranger-2.2.0.tar.gz.SHA256
> > >  SHA512 Hash:
> > >
> >
> https://dist.apache.org/repos/dist/dev/ranger/2.2.0-rc2/apache-ranger-2.2.0.tar.gz.SHA512
> > >
> > >Keys to verify the signature of the release artifacts are available
> > at:
> > >https://dist.apache.org/repos/dist/release/ranger/KEYS
> > >
> > >New features/enhancements:
> > >
> > >
> > >-
> > >
> > >Schema changes to improve performance of chained plugin
> features.
> > >RANGER-3067
> > >
> > >
> > >-
> > >
> > >Support delegation-admin for specific permissions.RANGER-3122
> > >-
> > >
> > >Kafka Client improvement to use Kafka AdminClient API instead of
> > >Zookeeper. RANGER-3001
> > >-
> > >
> > >GET API service/xusers/users response time improvement.
> > RANGER-3027/
> > >RANGER-3024
> > >-
> > >
> > >Improvement in Ranger Latest UI's Edit Policy Page. RANGER-3130
> > >-
> > >
> > >Ranger UI Search by object name in page /reports/audit/admin.
> > RANGER-3052
> > >-
> > >
> > >Enhancement to trace additional information on resources.
> > RANGER-3065
> > >-
> > >
> > >Improve audit log for Role operations in Ranger Hive authorizer.
> > >RANGER-3170
> > >-
> > >
> > >Audit-filter feature implementation to help reduce volume of
> > audit logs
> > >generated.RANGER-3000
> > >-
> > >
> > >Need feature to make the access log file name configurable for
> > >user.RANGER-3242/RANGER-3241
> > >-
> > >
> > >Upgrade solr version in Ranger to Solr 8.6.3 for better
> > >performance.RANGER-3091
> > >-
> > >
> > >Enhance Ranger admin REST Client to use cookie for policy, tag
> > and role
> > >download.RANGER-3283
> > >-
> > >

Re: Review Request 73629: RANGER-2846 RANGER-2847: Fix TestConnection and Resource Lookup(Bucket) for Ozone Plugin

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73629/#review223640
---


Ship it!




Ship It!

- Sailaja Polavarapu


On Oct. 20, 2021, 6:45 p.m., Abhishek  Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73629/
> ---
> 
> (Updated Oct. 20, 2021, 6:45 p.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, 
> Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2846 and RANGER-2847
> https://issues.apache.org/jira/browse/RANGER-2846
> https://issues.apache.org/jira/browse/RANGER-2847
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Test connection for ozone service now works on the Ranger UI.
> Also, fixes ozone bucket lookup while trying to create a new ozone policy in 
> Ranger.
> 
> 
> Diffs
> -
> 
>   distro/src/main/assembly/admin-web.xml 3fc054d7c 
>   distro/src/main/assembly/plugin-ozone.xml 734bf1970 
>   
> plugin-ozone/src/main/java/org/apache/ranger/services/ozone/client/OzoneClient.java
>  695510f47 
>   
> plugin-ozone/src/main/java/org/apache/ranger/services/ozone/client/OzoneConnectionMgr.java
>  5b18ee356 
>   pom.xml 626591d60 
> 
> 
> Diff: https://reviews.apache.org/r/73629/diff/2/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Abhishek  Kumar
> 
>

Re: Review Request 73612: RANGER-3452: Other attributes and sync source are not shown in audits during updates to users/groups.

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73612/#review223639
---


Ship it!




Ship It!

- Sailaja Polavarapu


On Sept. 27, 2021, 7:19 p.m., Abhishek  Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73612/
> ---
> 
> (Updated Sept. 27, 2021, 7:19 p.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, 
> Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3452
> https://issues.apache.org/jira/browse/RANGER-3452
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> The two fields will now be shown in audits whenever updates are made to users 
> and groups. Previously updates to users/groups would not show any diff in the 
> 2 fields.
> 
> 
> Diffs
> -
> 
>   security-admin/src/main/java/org/apache/ranger/service/XGroupService.java 
> c39796806 
>   security-admin/src/main/java/org/apache/ranger/service/XUserService.java 
> eff1718b3 
> 
> 
> Diff: https://reviews.apache.org/r/73612/diff/1/
> 
> 
> Testing
> ---
> 
> New fields are now shown on the UI after updates to users/groups in the audit 
> tab.
> 
> 
> Thanks,
> 
> Abhishek  Kumar
> 
>

Re: Review Request 73629: RANGER-2846 RANGER-2847: Fix TestConnection and Resource Lookup(Bucket) for Ozone Plugin

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73629/#review223638
---




plugin-ozone/src/main/java/org/apache/ranger/services/ozone/client/OzoneConnectionMgr.java
Line 53 (original), 53 (patched)
<https://reviews.apache.org/r/73629/#comment312754>

Minor one - Please rename the variable "connectHive" to "connectOzone"


- Sailaja Polavarapu


On Oct. 5, 2021, 7:52 p.m., Abhishek  Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73629/
> ---
> 
> (Updated Oct. 5, 2021, 7:52 p.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, 
> Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-2846 and RANGER-2847
> https://issues.apache.org/jira/browse/RANGER-2846
> https://issues.apache.org/jira/browse/RANGER-2847
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Test connection for ozone service now works on the Ranger UI.
> Also, fixes ozone bucket lookup while trying to create a new ozone policy in 
> Ranger.
> 
> 
> Diffs
> -
> 
>   distro/src/main/assembly/admin-web.xml 3fc054d7c 
>   distro/src/main/assembly/plugin-ozone.xml 734bf1970 
>   
> plugin-ozone/src/main/java/org/apache/ranger/services/ozone/client/OzoneClient.java
>  695510f47 
>   
> plugin-ozone/src/main/java/org/apache/ranger/services/ozone/client/OzoneConnectionMgr.java
>  5b18ee356 
>   pom.xml 626591d60 
> 
> 
> Diff: https://reviews.apache.org/r/73629/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Abhishek  Kumar
> 
>

[jira] [Assigned] (RANGER-3468) When multiple Ranger tabs are opened, Some tabs are not redirecting to Knox Logout page

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu reassigned RANGER-3468:
--

Assignee: Sailaja Polavarapu

> When multiple Ranger tabs are opened, Some tabs are not redirecting to Knox 
> Logout page 
> 
>
> Key: RANGER-3468
> URL: https://issues.apache.org/jira/browse/RANGER-3468
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
>
> When multiple Ranger tabs are open and session timeout happens some tabs are 
> not getting redirected to Knox logout page but we see new session has been 
> created
> Steps
> 1.Open 3 tabs of Ranger
> 2.Wait for idle session timeout
> 3.When checked all the tabs some tabs are not redirected to knox logout page.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (RANGER-3468) When multiple Ranger tabs are opened, Some tabs are not redirecting to Knox Logout page

[Sailaja Polavarapu (Jira)]

Sailaja Polavarapu created RANGER-3468:
--

 Summary: When multiple Ranger tabs are opened, Some tabs are not 
redirecting to Knox Logout page 
 Key: RANGER-3468
 URL: https://issues.apache.org/jira/browse/RANGER-3468
 Project: Ranger
  Issue Type: Bug
  Components: Ranger
Reporter: Sailaja Polavarapu


When multiple Ranger tabs are open and session timeout happens some tabs are 
not getting redirected to Knox logout page but we see new session has been 
created

Steps
1.Open 3 tabs of Ranger
2.Wait for idle session timeout
3.When checked all the tabs some tabs are not redirected to knox logout page.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 73590: RANGER-3440: Fix missing updates to sync source during upgrades

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73590/#review223538
---


Ship it!




Ship It!

- Sailaja Polavarapu


On Sept. 22, 2021, 6:18 p.m., Abhishek  Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73590/
> ---
> 
> (Updated Sept. 22, 2021, 6:18 p.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Sailaja 
> Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3285
> https://issues.apache.org/jira/browse/RANGER-3285
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> During the ranger upgrades, sync source (newly introduced column) for users 
> and groups is not updated, the review fixes it. Also involves refactoring to 
> simpify the logic.
> 
> 
> Diffs
> -
> 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
>  ff513aa86 
> 
> 
> Diff: https://reviews.apache.org/r/73590/diff/1/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Abhishek  Kumar
> 
>

Re: Review Request 73605: RANGER-3438: In Knox plugin group lookup sometimes takes long time

[Sailaja Polavarapu]

> On Sept. 23, 2021, 9:29 p.m., Abhay Kulkarni wrote:
> > knox-agent/src/main/java/org/apache/ranger/authorization/knox/RangerPDPKnoxFilter.java
> > Line 124 (original), 126 (patched)
> > <https://reviews.apache.org/r/73605/diff/1/?file=2253187#file2253187line127>
> >
> > Is it possible that primaryUser and/or impersonatedUser (therefore 
> > user) are all null? If so, is this error condition?

I don't think this is possible in the regular code flow as knox is supposedly 
set the PrimaryPrincipal in the Subject as part of authentication. Now we are 
actually more expicitly checking if (primaryPrincipals != null && 
primaryPrincipals.size() > 0)


- Sailaja


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73605/#review223529
-------


On Sept. 23, 2021, 9:15 p.m., Sailaja Polavarapu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73605/
> ---
> 
> (Updated Sept. 23, 2021, 9:15 p.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Mehul Parikh, Ramesh Mani, and 
> Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3438
> https://issues.apache.org/jira/browse/RANGER-3438
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> Optimized code to extract GroupPrincipals from javax Subject and used similar 
> logic for retrieving primaryUser & impersonatedUser from Subject.
> 
> 
> Diffs
> -
> 
>   
> knox-agent/src/main/java/org/apache/ranger/authorization/knox/RangerPDPKnoxFilter.java
>  62363ab06 
> 
> 
> Diff: https://reviews.apache.org/r/73605/diff/1/
> 
> 
> Testing
> ---
> 
> 1. Patched cluster and verified functionality by accessing UI services like 
> ranger, atlas, etc... through knox proxy.
> 2. Also verified few regression tests both with group based policies and user 
> based policies for knox
> 
> 
> Thanks,
> 
> Sailaja Polavarapu
> 
>

[jira] [Updated] (RANGER-3438) In Knox plugin group lookup sometimes takes long time

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu updated RANGER-3438:
---
Attachment: 0001-RANGER-3438-Optimized-code-to-extract-GroupPrincipal.patch

> In Knox plugin group lookup sometimes takes long time
> -
>
> Key: RANGER-3438
> URL: https://issues.apache.org/jira/browse/RANGER-3438
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Attachments: 
> 0001-RANGER-3438-Optimized-code-to-extract-GroupPrincipal.patch
>
>
> When trusted proxy is enabled for various UI services, all the request go 
> through knox. plugin authorization. In some cases group lookup is taking long 
> time and the UI is very sluggish.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Review Request 73605: RANGER-3438: In Knox plugin group lookup sometimes takes long time

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73605/
---

Review request for ranger, Abhay Kulkarni, Mehul Parikh, Ramesh Mani, and 
Velmurugan Periasamy.


Bugs: RANGER-3438
https://issues.apache.org/jira/browse/RANGER-3438


Repository: ranger


Description
---

Optimized code to extract GroupPrincipals from javax Subject and used similar 
logic for retrieving primaryUser & impersonatedUser from Subject.


Diffs
-

  
knox-agent/src/main/java/org/apache/ranger/authorization/knox/RangerPDPKnoxFilter.java
 62363ab06 


Diff: https://reviews.apache.org/r/73605/diff/1/


Testing
---

1. Patched cluster and verified functionality by accessing UI services like 
ranger, atlas, etc... through knox proxy.
2. Also verified few regression tests both with group based policies and user 
based policies for knox


Thanks,

Sailaja Polavarapu

[jira] [Assigned] (RANGER-3438) In Knox plugin group lookup sometimes takes long time

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu reassigned RANGER-3438:
--

Assignee: Sailaja Polavarapu

> In Knox plugin group lookup sometimes takes long time
> -
>
> Key: RANGER-3438
> URL: https://issues.apache.org/jira/browse/RANGER-3438
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
>
> When trusted proxy is enabled for various UI services, all the request go 
> through knox. plugin authorization. In some cases group lookup is taking long 
> time and the UI is very sluggish.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (RANGER-3438) In Knox plugin group lookup sometimes takes long time

[Sailaja Polavarapu (Jira)]

Sailaja Polavarapu created RANGER-3438:
--

 Summary: In Knox plugin group lookup sometimes takes long time
 Key: RANGER-3438
 URL: https://issues.apache.org/jira/browse/RANGER-3438
 Project: Ranger
  Issue Type: Improvement
  Components: Ranger
Reporter: Sailaja Polavarapu


When trusted proxy is enabled for various UI services, all the request go 
through knox. plugin authorization. In some cases group lookup is taking long 
time and the UI is very sluggish.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Planning for Apache Ranger 2.2.0 release

[Sailaja Polavarapu]

+1 Thanks Ramesh for putting this together.
- Sailaja.

On Mon, Sep 20, 2021 at 12:46 PM Abhay Kulkarni  wrote:

> +1.
>
> Thanks, Ramesh.
>
> On Mon, Sep 20, 2021 at 8:48 AM Ramesh Mani  wrote:
> >
> > Dear Ranger Community members,
> >
> > This is the reminder to give your opinion on Apache Ranger Release 2.2.0.
> >
> > Thanks,
> > Ramesh
> >
> > On Tue, Sep 14, 2021 at 2:48 PM Ramesh Mani  wrote:
> >>
> >> Dear Ranger Community members,
> >>
> >>
> >>
> >> There are many features and fixes done in Apache Ranger Project since
> the release of Apache Ranger 2.1.0. These features enhance the quality and
> improve the user experience of Apache Ranger overall.
> >>
> >>
> >>
> >> Some of the key enhancements/features in this release are
> >>
> >>
> >>
> >> Schema changes to improve performance of chained plugin features.
> RANGER-3067
> >>
> >> Support delegation-admin for specific permissions.RANGER-3122
> >>
> >> Kafka Client improvement to use Kafka AdminClient API instead of
> Zookeeper. RANGER-3001
> >>
> >> GET API service/xusers/users response time improvement. RANGER-3027/
> RANGER-3024
> >>
> >> Improvement in Ranger Latest UI's Edit Policy Page. RANGER-3130
> >>
> >> Ranger UI Search by object name in page /reports/audit/admin.
> RANGER-3052
> >>
> >> Enhancement to trace additional information on resources. RANGER-3065
> >>
> >> Improve audit log for Role operations in Ranger Hive authorizer.
> RANGER-3170
> >>
> >> Audit-filter feature implementation to help reduce volume of audit logs
> generated. RANGER-3000
> >>
> >> Need feature to make the access log file name configurable for user.
> RANGER-3242/RANGER-3241
> >>
> >> Upgrade the solr version in Ranger to Solr 8.6.3 for better
> performance. RANGER-3091
> >>
> >> Enhance Ranger admin REST Client to use cookies for policy, tag and
> role download. RANGER-3283
> >>
> >> Audit Filter default policies for reducing verbosity in auditing.
> RANGER-3260/RANGER-3283
> >>
> >> Auditing for HDFS chmod and chown operations. RANGER-3148
> >>
> >> Ranger HiveAuthorizer improvements to handle uncharted hive commands.
> RANGER-3368
> >>
> >> Ranger Access audit page improvement. RANGER-3109
> >>
> >> Dockerfile to support building from local repository. RANGER-3012
> >>
> >> Performance improvement for Ranger usersync. RANGER-2986
> >>
> >>
> >>
> >> Bug Fixes:
> >>
> >> In this Apache Ranger release there are around 119 bug fixes done.
> >>
> >>
> >>
> >> There are 324 commits with 219 resolved JIRAs in the release branch
> ranger-2.2.0 and with these improvements it is time to do the next Apache
> Ranger release.
> >>
> >>
> >> Planned timeline to release Apache Ranger 2.2.0 is end of September
> 2021.
> >>
> >>
> >>
> >> Please review and give your comments.
> >>
> >>
> >>
> >> Thanks,
> >>
> >> Ramesh
> >>
> >>
>

[jira] [Updated] (RANGER-3403) Ranger usersync role based rules not working as expected

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu updated RANGER-3403:
---
Attachment: 0001-RANGER-3403-Fixed-issue-where-role-assignments-are-n.patch

> Ranger usersync role based rules not working as expected
> 
>
> Key: RANGER-3403
> URL: https://issues.apache.org/jira/browse/RANGER-3403
> Project: Ranger
>  Issue Type: Bug
>  Components: Ranger, usersync
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Attachments: 
> 0001-RANGER-3403-Fixed-issue-where-role-assignments-are-n.patch
>
>
> When add a group in Ranger Usersync Group based role assignment Rules, all 
> members in the group will have the admin role, but if remove the group from 
> Ranger Usersync Group based role assignment Rules, the members will still be 
> admin. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Review Request 73575: RANGER-3403: Ranger usersync role based rules not working as expected

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73575/
---

Review request for ranger, Abhay Kulkarni, Mehul Parikh, Ramesh Mani, Vishal 
Suvagia, and Velmurugan Periasamy.


Bugs: RANGER-3403
https://issues.apache.org/jira/browse/RANGER-3403


Repository: ranger


Description
---

Added code to reset the users' role to "ROLE_USER" when a user/group is removed 
from the role assignment rules.


Diffs
-

  security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 546345ac0 
  security-admin/src/main/java/org/apache/ranger/db/XXPortalUserDao.java 
d383cf596 
  security-admin/src/main/resources/META-INF/jpa_named_queries.xml 8eff33643 
  
ugsync-util/src/main/java/org/apache/ranger/ugsyncutil/model/UsersGroupRoleAssignments.java
 ce0b9dd8b 
  
ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
 ff513aa86 


Diff: https://reviews.apache.org/r/73575/diff/1/


Testing
---

1. Verified the build is successful and existing test cases ran successfully
2. Patched a cluster and verified the functionality and basic regression tests.


Thanks,

Sailaja Polavarapu

[jira] [Created] (RANGER-3403) Ranger usersync role based rules not working as expected

[Sailaja Polavarapu (Jira)]

Sailaja Polavarapu created RANGER-3403:
--

 Summary: Ranger usersync role based rules not working as expected
 Key: RANGER-3403
 URL: https://issues.apache.org/jira/browse/RANGER-3403
 Project: Ranger
  Issue Type: Bug
  Components: Ranger, usersync
Reporter: Sailaja Polavarapu
Assignee: Sailaja Polavarapu


When add a group in Ranger Usersync Group based role assignment Rules, all 
members in the group will have the admin role, but if remove the group from 
Ranger Usersync Group based role assignment Rules, the members will still be 
admin. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: Review Request 73541: RANGER-3363: Support Session Inactivity Timeout for Ranger

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73541/
---

(Updated Aug. 27, 2021, 12:12 a.m.)


Review request for ranger, Kishor Gollapalliwar, Mehul Parikh, Ramesh Mani, 
Vishal Suvagia, and Velmurugan Periasamy.


Changes
---

Added null check for httpsession before invalidating.


Bugs: RANGER-3363
https://issues.apache.org/jira/browse/RANGER-3363


Repository: ranger


Description
---

Added support in ranger admin for handling session timeout requests with knox 
proxy. When ranger recieves timeout request for an existing session, the 
session will be removed and request will be redirected to logout page.


Diffs (updated)
-

  security-admin/src/main/java/org/apache/ranger/rest/UserREST.java ffdf1019a 
  
security-admin/src/main/java/org/apache/ranger/security/web/authentication/RangerAuthenticationEntryPoint.java
 3e56d9fdf 
  
security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKRBAuthenticationFilter.java
 9877e14c3 
  
security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java
 6d3599110 
  security-admin/src/main/java/org/apache/ranger/util/RestUtil.java 4d7388fe1 
  security-admin/src/main/java/org/apache/ranger/view/VXPortalUser.java 
5a1b203ff 


Diff: https://reviews.apache.org/r/73541/diff/2/

Changes: https://reviews.apache.org/r/73541/diff/1-2/


Testing
---

1. Patched a cluster and verfied the timeout requests are handled as required 
with knox proxy case.


Thanks,

Sailaja Polavarapu

[jira] [Updated] (RANGER-3363) Support Session Inactivity Timeout for Ranger

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu updated RANGER-3363:
---
Attachment: 0001-RANGER-3363-Added-support-for-handling-session-timeo.patch

> Support Session Inactivity Timeout for Ranger
> -
>
> Key: RANGER-3363
> URL: https://issues.apache.org/jira/browse/RANGER-3363
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Attachments: 
> 0001-RANGER-3363-Added-support-for-handling-session-timeo.patch
>
>
> Ranger users who login via one of the supported authentication flavors stay 
> logged in until they choose to log out.
> In the scenario where user has logged in and has left their desk, the 
> inactivity does not result in early log out.
> This implementation addresses that problem.
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Review Request 73541: RANGER-3363: Support Session Inactivity Timeout for Ranger

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73541/
---

Review request for ranger, Kishor Gollapalliwar, Mehul Parikh, Ramesh Mani, 
Vishal Suvagia, and Velmurugan Periasamy.


Bugs: RANGER-3363
https://issues.apache.org/jira/browse/RANGER-3363


Repository: ranger


Description
---

Added support in ranger admin for handling session timeout requests with knox 
proxy. When ranger recieves timeout request for an existing session, the 
session will be removed and request will be redirected to logout page.


Diffs
-

  security-admin/src/main/java/org/apache/ranger/rest/UserREST.java ffdf1019a 
  
security-admin/src/main/java/org/apache/ranger/security/web/authentication/RangerAuthenticationEntryPoint.java
 3e56d9fdf 
  
security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerKRBAuthenticationFilter.java
 9877e14c3 
  
security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java
 6d3599110 
  security-admin/src/main/java/org/apache/ranger/util/RestUtil.java 4d7388fe1 
  security-admin/src/main/java/org/apache/ranger/view/VXPortalUser.java 
5a1b203ff 


Diff: https://reviews.apache.org/r/73541/diff/1/


Testing
---

1. Patched a cluster and verfied the timeout requests are handled as required 
with knox proxy case.


Thanks,

Sailaja Polavarapu

Re: Review Request 73354: RANGER-3285: Introduce new column Sync Source

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73354/#review223374
---


Ship it!




Ship It!

- Sailaja Polavarapu


On Aug. 19, 2021, 4:24 a.m., Abhishek  Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/73354/
> ---
> 
> (Updated Aug. 19, 2021, 4:24 a.m.)
> 
> 
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, 
> Pradeep Agrawal, Sailaja Polavarapu, and Velmurugan Periasamy.
> 
> 
> Bugs: RANGER-3285
> https://issues.apache.org/jira/browse/RANGER-3285
> 
> 
> Repository: ranger
> 
> 
> Description
> ---
> 
> New features implemented in this patch:
> 1. A new coulmn 'sync_source' is available for db tables x_user, 
> x_portal_user and x_group. This column is also a search field for REST API 
> queries.
> 2. Tested the functionality on containers for proper functioning.
> 3. Front-End side changes to follow soon.
> 
> 
> Diffs
> -
> 
>   security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 
> 1ba3e69e8 
>   
> security-admin/db/mysql/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
>  PRE-CREATION 
>   security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 
> 021c34c77 
>   
> security-admin/db/oracle/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
>  PRE-CREATION 
>   security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 
> 89e9e77a0 
>   
> security-admin/db/postgres/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
>  PRE-CREATION 
>   
> security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql
>  b90a48e0a 
>   
> security-admin/db/sqlanywhere/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
>  PRE-CREATION 
>   security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 
> 7e03289dd 
>   
> security-admin/db/sqlserver/patches/055-add-syncSource-col-in-x_user-x_portal_user-x_group.sql
>  PRE-CREATION 
>   security-admin/src/main/java/org/apache/ranger/biz/UserMgr.java e72a7a733 
>   security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 38b06d1b2 
>   security-admin/src/main/java/org/apache/ranger/entity/XXGroup.java 
> 996a9e1d4 
>   security-admin/src/main/java/org/apache/ranger/entity/XXPortalUser.java 
> ed69f385e 
>   security-admin/src/main/java/org/apache/ranger/entity/XXUser.java 0464e7b6e 
>   security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java 
> 4c648aeb1 
>   security-admin/src/main/java/org/apache/ranger/service/XGroupService.java 
> 44eebcfb7 
>   
> security-admin/src/main/java/org/apache/ranger/service/XGroupServiceBase.java 
> cde91dc52 
>   
> security-admin/src/main/java/org/apache/ranger/service/XPortalUserServiceBase.java
>  85e4b8412 
>   security-admin/src/main/java/org/apache/ranger/service/XUserService.java 
> 0728eea2a 
>   
> security-admin/src/main/java/org/apache/ranger/service/XUserServiceBase.java 
> 9cdc14ebf 
>   security-admin/src/main/java/org/apache/ranger/view/VXGroup.java ed6470425 
>   security-admin/src/main/java/org/apache/ranger/view/VXPortalUser.java 
> b6035b236 
>   security-admin/src/main/java/org/apache/ranger/view/VXUser.java 96f6468f7 
>   
> ugsync-util/src/main/java/org/apache/ranger/ugsyncutil/model/XGroupInfo.java 
> a80e5dc1b 
>   ugsync-util/src/main/java/org/apache/ranger/ugsyncutil/model/XUserInfo.java 
> bc1a6655c 
>   
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
>  65e26fe06 
> 
> 
> Diff: https://reviews.apache.org/r/73354/diff/4/
> 
> 
> Testing
> ---
> 
> Tested the changes on postgres:
> 1. New column is present on the tables x_user, x_portal_user and x_group.
> 2. REST API calls (POST) made for user and group result in persistence of 
> data for syncSource: verified via db queries.
> 3. The syncSource field appears in the response for a GET at the user and 
> group APIs with correct data.
> 
> 
> Thanks,
> 
> Abhishek  Kumar
> 
>

[jira] [Updated] (RANGER-3374) Syncing 300K+ user group mappings to ranger is causing ranger to go out of memory

[Sailaja Polavarapu (Jira)]

 [ 
https://issues.apache.org/jira/browse/RANGER-3374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sailaja Polavarapu updated RANGER-3374:
---
Attachment: 0001-RANGER-3374-Creating-new-DB-transaction-for-add-upda.patch

> Syncing 300K+ user group mappings to ranger is causing ranger to go out of 
> memory 
> --
>
> Key: RANGER-3374
> URL: https://issues.apache.org/jira/browse/RANGER-3374
> Project: Ranger
>  Issue Type: Improvement
>  Components: Ranger
>    Reporter: Sailaja Polavarapu
>    Assignee: Sailaja Polavarapu
>Priority: Major
> Attachments: 
> 0001-RANGER-3374-Creating-new-DB-transaction-for-add-upda.patch
>
>
> When there are 300k+ user group mappings to be updated to Ranger from 
> usersync, ranger crashes with out of memory errors. Also noticed that when 
> the user group mappings are around 100k, it is taking very long time for the 
> request to complete. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Review Request 73531: RANGER-3374: Syncing 300K+ user group mappings to ranger is causing ranger to go out of memory

[Sailaja Polavarapu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/73531/
---

Review request for ranger, Abhay Kulkarni, Pradeep Agrawal, Ramesh Mani, and 
Velmurugan Periasamy.


Bugs: RANGER-3374
https://issues.apache.org/jira/browse/RANGER-3374


Repository: ranger


Description
---

Creating new DB transaction for add/update of user group mapping from Ranger 
Usersync.


Diffs
-

  security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 6596bac5d 
  security-admin/src/main/java/org/apache/ranger/service/XGroupUserService.java 
6c0289c1c 


Diff: https://reviews.apache.org/r/73531/diff/1/


Testing
---

1. Patched cluster and verified with almost 1million user group mappings.


Thanks,

Sailaja Polavarapu

[jira] [Created] (RANGER-3374) Syncing 300K+ user group mappings to ranger is causing ranger to go out of memory

[Sailaja Polavarapu (Jira)]

Sailaja Polavarapu created RANGER-3374:
--

 Summary: Syncing 300K+ user group mappings to ranger is causing 
ranger to go out of memory 
 Key: RANGER-3374
 URL: https://issues.apache.org/jira/browse/RANGER-3374
 Project: Ranger
  Issue Type: Improvement
  Components: Ranger
Reporter: Sailaja Polavarapu
Assignee: Sailaja Polavarapu


When there are 300k+ user group mappings to be updated to Ranger from usersync, 
ranger crashes with out of memory errors. Also noticed that when the user group 
mappings are around 100k, it is taking very long time for the request to 
complete. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Re: [DRAFT] [REPORT] Apache Ranger - Aug-2021

[Sailaja Polavarapu]

+1

On Wed, Aug 11, 2021 at 9:29 PM Abhay Kulkarni
 wrote:

> +1
>
> On Wed, Aug 11, 2021 at 4:35 PM Madhan Neethiraj 
> wrote:
> >
> > Rangers,
> >
> >
> >
> > Please review the following draft report and let me know your
> comments/feedback.
> >
> >
> >
> > Thanks,
> >
> > Madhan
> >
> >
> >
> > ## Description:
> >
> > Apache Ranger is a framework to enable, monitor and manage comprehensive
> data
> >
> > security - consistently across various data processing services.
> >
> >
> >
> > ## Issues:
> >
> > There are no issues requiring board attention at this time
> >
> >
> >
> > ## Membership Data:
> >
> > Apache Ranger was founded 2017-01-17 (5 years ago)
> >
> > There are currently 30 committers and 20 PMC members in this project.
> >
> > The Committer-to-PMC ratio is 3:2.
> >
> >
> >
> > Community changes, past quarter:
> >
> >  - No new PMC members. Last addition was Sailaja Polavarapu on
> 2019-09-18.
> >
> >  - No new committers. Last addition was Dhaval Shah on 2021-01-20.
> >
> >
> >
> > ## Project Activity:
> >
> >  - Atlas plugin enhancements to handle authorization of classification
> >
> >add/update/remove on entities
> >
> >  - Hive plugin enhancements to implement SHOW ROLE GRANT operation, and
> >
> >support new operations
> >
> >  - HDFS plugin updates to handle access to paths in snapshots
> >
> >  - upgraded Spring framework and security libraries
> >
> >  - delegated-admin model updated to scope delegation to specific
> access-types
> >
> >  - fixes in Python library/API
> >
> >  - script condition evaluator improvements to make it easier to access
> details
> >
> >of request/user/groups/tags
> >
> >  - policy-engine enhancement to include data-making/row-filtering in
> >
> >getResourceACLs() API
> >
> >  - improv Ranger admin startup in handling availability of Solr
> >
> >  - performance improvements in user-lookup API implementation
> >
> >  - performance improvements to processing of tags by batching
> >
> >
> >
> > ## Community Health:
> >
> >  - as the stats below show, the community is active and continue to
> improve
> >
> >Apache Ranger by adding enhancements and fixes
> >
> >  - a large number of enhancements and fixes were added since last
> release of
> >
> >Ranger. Next minor release, 2.2.0, is likely around mid-September
> >
> >  - Ranger is being adopted by major cloud vendors to authorize access to
> their
> >
> >services/data, for example:
> >
> >   - AWS EMR, S3:
> https://aws.amazon.com/blogs/big-data/introducing-amazon-emr-integration-with-apache-ranger/
> >
> >   - Azure:
> https://docs.microsoft.com/en-us/azure/hdinsight/domain-joined/hdinsight-security-overview
> >
> >   - Google:
> https://cloud.google.com/blog/products/data-analytics/running-cloud-managed-spark-and-hadoop-using-ranger
> >
> >  - Stats
> >
> >   - dev@ranger.apache.org had a 8% decrease in traffic in the past
> quarter
> >
> > (728 emails compared to 783)
> >
> >   - u...@ranger.apache.org had a 433% increase in traffic in the past
> quarter
> >
> > (16 emails compared to 3)
> >
> >   - 82 issues opened in JIRA, past quarter (-21% change)
> >
> >   - 61 issues closed in JIRA, past quarter (-25% change)
> >
> >   - 120 commits in the past quarter (-19% change)
> >
> >   - 17 code contributors in the past quarter (6% increase)
> >
> >   - 12 PRs opened on GitHub, past quarter (20% increase)
> >
> >   - 2 PRs closed on GitHub, past quarter (100% increase)
> >
> >
> >
> >
> >
> >
> >
>