Review Request 69062: RANGER-2253: Unable to get dashboard page after login
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69062/ --- Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Bugs: RANGER-2253 https://issues.apache.org/jira/browse/RANGER-2253 Repository: ranger Description --- **Problem Statement:** Unable to get dashboard page after login. The issue is seen after the commit https://github.com/apache/ranger/commit/8257bc94f0d73175da99fed65eec241bf7a4efe2 **Proposed Solution:** I observed that Ranger is pulling two different jars of the asm library and probably due to jar conflict Ranger Dashboard page is not loading. Proposed patch includes changes of excluding the asm-5.0.4.jar from the ranger packaging. Diffs - security-admin/pom.xml 8283e3159 Diff: https://reviews.apache.org/r/69062/diff/1/ Testing --- Steps performed for Ranger-admin(with patch): 1. Created Build with patch and untar the build. 2. Opened install.properties and provided db configuration in install.properties 3. Called setup.sh 4. Started Ranger admin **Expected Behavior:** 1. Ranger-admin installation should finish successfully and asm-5.0.4 jars should not be available at location: ranger-0.7.2-SNAPSHOT-admin/ews/webapp/WEB-INF/lib 2. Ranger admin should start and dashboard page should load. **Actual Behavior:** 1 Ranger-admin Installation finished successfully and asm-5.0.4 jar was not available at location: ranger-0.7.2-SNAPSHOT-admin/ews/webapp/WEB-INF/lib 2. Ranger admin started successfully and dashboard page was loaded. Note: After Login was able to created policies. Thanks, Pradeep Agrawal
Review Request 69111: RANGER-2260: Atlas servicedef version change patch should update atlas access type def for tag def also.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69111/ --- Review request for ranger, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, and Velmurugan Periasamy. Bugs: RANGER-2260 https://issues.apache.org/jira/browse/RANGER-2260 Repository: ranger Description --- **Problem Statement:** Currently atlas ranger service def name change patch J10013 does not rename its access types of the tag service def. **Proposed Solution:** Proposed solution includes changes in java patch J10013 which shall rename the tag access types and implied grants also if atlas service def is renamed. If java patch J10013 is already executed then sql patch 035 execution will make sure that tag service def access types are renamed as per new atlas servicedef name. Diffs - security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 70447fa2d security-admin/db/mysql/patches/035-update-schema-for-x-policy.sql ee82ae3eb security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 8b513073d security-admin/db/oracle/patches/035-update-schema-for-x-policy.sql 11b41720e security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql a12391191 security-admin/db/postgres/patches/035-update-schema-for-x-policy.sql cb87cd0b8 security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 142302a80 security-admin/db/sqlanywhere/patches/035-update-schema-for-x-policy.sql 24d072f83 security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 75c8fafeb security-admin/db/sqlserver/patches/035-update-schema-for-x-policy.sql ebf44ac83 security-admin/src/main/java/org/apache/ranger/patch/PatchForAtlasServiceDefUpdate_J10013.java 1b315cd76 Diff: https://reviews.apache.org/r/69111/diff/1/ Testing --- **Steps Performed (without patch):** 1. After mvn Build; untar the Ranger-0.7 module and updated install.properties for MySQL DB flavor. 2. Called setup.sh to install Ranger-admin. 3. Started Ranger-admin. 4. Created atlas and tag based policies in ranger-admin 5. Stopped ranger-admin 6. Changed branch to ranger-1.0 and started mvn Build; after getting build untar the Ranger-1.0 module and updated install.properties for MySQL DB flavor with the config provided in step-1. 7. Called setup.sh to install Ranger. 8. Started Ranger. 9. atlas service def was renamed to atlasdev.v1 and was able to view atlas and tag based policies in ranger-admin 10. Stopped ranger-admin 11. Changed branch to ranger-2.0 and started mvn Build; after getting build untar the Ranger-2.0 module and updated install.properties for MySQL DB flavor with the config provided in step-1. 12. Removed 'DB_PATCHES' and 'JAVA_PATCHES' entry from x_db_version_h table to enable execution of new SQL patches. 13. Called setup.sh to install Ranger. **Expected Behavior:** 1. Ranger installation should finish successfully and patch 035 should get applied successfully. 2. 'atlas:read','atlas:write','atlas:delete','atlas:update','atlas:all' entries should rename to 'atlas.v1:read','atlas.v1:write','atlas.v1:delete','atlas.v1:update','atlas.v1:all' in x_access_type_def and x_access_type_def_grants table for tag service def entries. 3. Java patch 19 and 20 starting should migrate policies to new schema and policies should appear in policy listing page after starting the ranger-admin. 4. New policies creation should work and policy should get save in new policy schema. ** Actual Behavior: ** 1. Ranger installation finished successfully and patch 035 was applied successfully. all other new sql and java patches was applied successfully. 2. 'atlas:read','atlas:write','atlas:delete','atlas:update','atlas:all' entries were renamed to 'atlas.v1:read','atlas.v1:write','atlas.v1:delete','atlas.v1:update','atlas.v1:all' in x_access_type_def and x_access_type_def_grants table for tag service def entries. 3. Policy migration completed successfully and after starting ranger-admin policies was appearing in policy listing page. 4. Was able to create new policies in the the new atlas service def. **Note:** 1. Patch has been tested on all supported DB Flavors. 2. Have not done load testing. 3. Have tested the patch on ranger-2.0 (installed ranger from pre RANGER-2203 commit and upgraded to latest master code build having change of this patch. This steps is tested for MySQL db flavors only. Thanks, Pradeep Agrawal
Re: Review Request 69083: RANGER-2251 : Need to provide options for making java heap size memory configurable in Ranger services.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69083/#review209861 --- Ship it! - Pradeep Agrawal On Oct. 22, 2018, 9:01 a.m., Vishal Suvagia wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69083/ > --- > > (Updated Oct. 22, 2018, 9:01 a.m.) > > > Review request for ranger, Ankita Sinha, Colm O hEigeartaigh, Gautam Borad, > Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Pradeep Agrawal, Ramesh Mani, > Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2251 > https://issues.apache.org/jira/browse/RANGER-2251 > > > Repository: ranger > > > Description > --- > > Need to make java heap size memory configurable for Ranger services. > > > Diffs > - > > embeddedwebserver/scripts/ranger-admin-services.sh > 990d3c7922351f298277792baa2551efa5e7a1cc > kms/scripts/ranger-kms 604d7014c5584d5feef26975c7bfffd8c2194f1e > security-admin/scripts/db_setup.py 5ac312fba9c3ddfb8c345d2c2551bab9c49fd67b > security-admin/scripts/install.properties > 34c52ebe58b59892ebf5f8fd66d81a73264aa049 > tagsync/scripts/ranger-tagsync-services.sh > 6fcdf1562569f6203da309936e4762395c9036f0 > unixauthservice/scripts/ranger-usersync-services.sh > 0c03c5a18eb9a15740df8398e96fc14104277dd2 > > > Diff: https://reviews.apache.org/r/69083/diff/2/ > > > Testing > --- > > Tested with fresh installation for heapsize to be effective for Ranger: > Admin, Usersycnc, Tagsync and KMS services. > > > Thanks, > > Vishal Suvagia > >
Re: Review Request 69111: RANGER-2260: Atlas servicedef version change patch should update atlas access type def for tag def also.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69111/ --- (Updated Oct. 22, 2018, 2:02 p.m.) Review request for ranger, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, and Velmurugan Periasamy. Changes --- Fixed PMD violation Bugs: RANGER-2260 https://issues.apache.org/jira/browse/RANGER-2260 Repository: ranger Description --- **Problem Statement:** Currently atlas ranger service def name change patch J10013 does not rename its access types of the tag service def. **Proposed Solution:** Proposed solution includes changes in java patch J10013 which shall rename the tag access types and implied grants also if atlas service def is renamed. If java patch J10013 is already executed then sql patch 035 execution will make sure that tag service def access types are renamed as per new atlas servicedef name. Diffs (updated) - security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 70447fa2d security-admin/db/mysql/patches/035-update-schema-for-x-policy.sql ee82ae3eb security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 8b513073d security-admin/db/oracle/patches/035-update-schema-for-x-policy.sql 11b41720e security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql a12391191 security-admin/db/postgres/patches/035-update-schema-for-x-policy.sql cb87cd0b8 security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 142302a80 security-admin/db/sqlanywhere/patches/035-update-schema-for-x-policy.sql 24d072f83 security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 75c8fafeb security-admin/db/sqlserver/patches/035-update-schema-for-x-policy.sql ebf44ac83 security-admin/src/main/java/org/apache/ranger/patch/PatchForAtlasServiceDefUpdate_J10013.java 1b315cd76 Diff: https://reviews.apache.org/r/69111/diff/2/ Changes: https://reviews.apache.org/r/69111/diff/1-2/ Testing --- **Steps Performed (without patch):** 1. After mvn Build; untar the Ranger-0.7 module and updated install.properties for MySQL DB flavor. 2. Called setup.sh to install Ranger-admin. 3. Started Ranger-admin. 4. Created atlas and tag based policies in ranger-admin 5. Stopped ranger-admin 6. Changed branch to ranger-1.0 and started mvn Build; after getting build untar the Ranger-1.0 module and updated install.properties for MySQL DB flavor with the config provided in step-1. 7. Called setup.sh to install Ranger. 8. Started Ranger. 9. atlas service def was renamed to atlasdev.v1 and was able to view atlas and tag based policies in ranger-admin 10. Stopped ranger-admin 11. Changed branch to ranger-2.0 and started mvn Build; after getting build untar the Ranger-2.0 module and updated install.properties for MySQL DB flavor with the config provided in step-1. 12. Removed 'DB_PATCHES' and 'JAVA_PATCHES' entry from x_db_version_h table to enable execution of new SQL patches. 13. Called setup.sh to install Ranger. **Expected Behavior:** 1. Ranger installation should finish successfully and patch 035 should get applied successfully. 2. 'atlas:read','atlas:write','atlas:delete','atlas:update','atlas:all' entries should rename to 'atlas.v1:read','atlas.v1:write','atlas.v1:delete','atlas.v1:update','atlas.v1:all' in x_access_type_def and x_access_type_def_grants table for tag service def entries. 3. Java patch 19 and 20 starting should migrate policies to new schema and policies should appear in policy listing page after starting the ranger-admin. 4. New policies creation should work and policy should get save in new policy schema. ** Actual Behavior: ** 1. Ranger installation finished successfully and patch 035 was applied successfully. all other new sql and java patches was applied successfully. 2. 'atlas:read','atlas:write','atlas:delete','atlas:update','atlas:all' entries were renamed to 'atlas.v1:read','atlas.v1:write','atlas.v1:delete','atlas.v1:update','atlas.v1:all' in x_access_type_def and x_access_type_def_grants table for tag service def entries. 3. Policy migration completed successfully and after starting ranger-admin policies was appearing in policy listing page. 4. Was able to create new policies in the the new atlas service def. **Note:** 1. Patch has been tested on all supported DB Flavors. 2. Have not done load testing. 3. Have tested the patch on ranger-2.0 (installed ranger from pre RANGER-2203 commit and upgraded to latest master code build having change of this patch. This steps is tested for MySQL db flavors only. Thanks, Pradeep Agrawal
Re: Review Request 69117: Improvement of export to excel from report listing page for Oracle database
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69117/#review209919 --- Ship it! Ship It! - Pradeep Agrawal On Oct. 22, 2018, 12:51 p.m., Nikhil P wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69117/ > --- > > (Updated Oct. 22, 2018, 12:51 p.m.) > > > Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2262 > https://issues.apache.org/jira/browse/RANGER-2262 > > > Repository: ranger > > > Description > --- > > Improvement of export to excel from report listing page for Oracle database > > > Diffs > - > > security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java > 5dfe446 > > > Diff: https://reviews.apache.org/r/69117/diff/1/ > > > Testing > --- > > 1.Tested on VM. > > > Thanks, > > Nikhil P > >
Re: Review Request 69189: Implement best coding practices for validating user input
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69189/#review210095 --- Ship it! Ship It! - Pradeep Agrawal On Oct. 26, 2018, 12:46 p.m., Nikhil P wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69189/ > --- > > (Updated Oct. 26, 2018, 12:46 p.m.) > > > Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, > Madhan Neethiraj, Nitin Galave, Pradeep Agrawal, Ramesh Mani, Selvamohan > Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2269 > https://issues.apache.org/jira/browse/RANGER-2269 > > > Repository: ranger > > > Description > --- > > Implement best coding practices for validating user input > > > Diffs > - > > security-admin/src/main/webapp/scripts/views/users/UserTableLayout.js > 60c25fa > > > Diff: https://reviews.apache.org/r/69189/diff/1/ > > > Testing > --- > > 1.Tested on local > > > Thanks, > > Nikhil P > >
Review Request 69198: RANGER-2270: Restrict tag module access to unprivileged users
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69198/ --- Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Bugs: RANGER-2270 https://issues.apache.org/jira/browse/RANGER-2270 Repository: ranger Description --- **Problem Statement** Currently, if Ranger user does not have access permission in tag module then he can see tag service def but can't view policies in it and can't change any information. **Proposed solution** I propose a change here that user should not able to view the tag service def if he don't have permission in Tag module. Diffs - security-admin/src/main/java/org/apache/ranger/biz/RangerBizUtil.java b304e3e99 security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 5a1e519b2 security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java c51354880 security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java e20cf9f89 Diff: https://reviews.apache.org/r/69198/diff/1/ Testing --- **Steps performed for Ranger-admin(with patch):** 1. Created Build with patch and untar the build. 2. Opened install.properties and provided db configuration in install.properties 3. Called setup.sh 4. Started Ranger admin. 5. Logged in from admin user and created a user testuser1 with user role. Logout from admin account. 6. Logged in from testuser1 id and try to access page http://localhost:6080/index.html#!/policymanager/tag in another tab. **Expected Behavior:** Ranger-admin should not show any tag service def as testuser1 is not having permission in the tag based module. **Actual Behavior:** Ranger-admin was not showing tag service def. Thanks, Pradeep Agrawal
Re: Review Request 69209: RANGER-2272: Ensure that Ranger policy contains lowercase access-type and resource-definition names
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69209/#review210167
---
Ship it!
Tested patch for given use case and its working.
- Pradeep Agrawal
On Oct. 29, 2018, 11:43 p.m., Abhay Kulkarni wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69209/
> ---
>
> (Updated Oct. 29, 2018, 11:43 p.m.)
>
>
> Review request for ranger, Madhan Neethiraj, Mehul Parikh, Ramesh Mani, and
> Velmurugan Periasamy.
>
>
> Bugs: RANGER-2272
> https://issues.apache.org/jira/browse/RANGER-2272
>
>
> Repository: ranger
>
>
> Description
> ---
>
> Access type names and resources definition names in Ranger service definition
> are required to be in lowercase. Ranger Policy also ensure that access type
> names and resource-definition names are in lowercase when it is persisted in
> the database.
>
>
> Diffs
> -
>
>
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerPolicyValidator.java
> 9de860d42
>
> agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerValidator.java
> c7062ddda
>
> agents-common/src/test/java/org/apache/ranger/plugin/model/validation/TestRangerPolicyValidator.java
> 140a9edf5
>
> agents-common/src/test/java/org/apache/ranger/plugin/model/validation/TestRangerValidator.java
> 5bdffda92
>
>
> Diff: https://reviews.apache.org/r/69209/diff/1/
>
>
> Testing
> ---
>
> Tested with mysql and postgres database back-ends. Created policy containing
> uppercase access-type name (such as 'Read' for hive policy), and uppercase
> resource-def name (such as 'DATABASE' for hive-policy). Ensured that the
> created policy can be retrieved using GUI. Verified that database contained
> correct, lowercase access-type name ('read') and resource-def name
> ('database')
>
>
> Thanks,
>
> Abhay Kulkarni
>
>
Re: Review Request 69228: Allow service admin and delegated admin user to view list of users and groups though they have 'USER' role
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69228/#review210312 --- Ship it! Ship It! - Pradeep Agrawal On Nov. 2, 2018, 1:31 p.m., Nikhil P wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69228/ > --- > > (Updated Nov. 2, 2018, 1:31 p.m.) > > > Review request for ranger, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, > Nitin Galave, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2273 > https://issues.apache.org/jira/browse/RANGER-2273 > > > Repository: ranger > > > Description > --- > > Due to RANGER-2208 users list access is too restricted to the users who are > having USER role. To create policies delegated admin and service admin need > access to users and groups information. > > > Diffs > - > > security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java > b40d4f027 > security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java febf2218d > security-admin/src/main/java/org/apache/ranger/rest/XUserREST.java > b5c6e9cc5 > > security-admin/src/main/java/org/apache/ranger/security/context/RangerAPIList.java > 4a6a76998 > security-admin/src/main/webapp/scripts/views/policies/PermissionList.js > 2b996b0e1 > security-admin/src/main/webapp/scripts/views/reports/UserAccessLayout.js > d5bad70df > > > Diff: https://reviews.apache.org/r/69228/diff/3/ > > > Testing > --- > > 1.tested on local VM. > > > Thanks, > > Nikhil P > >
Review Request 69253: RANGER-2278 Unable to delete user if he has references in new ref tables
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69253/ --- Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Bugs: RANGER-2278 https://issues.apache.org/jira/browse/RANGER-2278 Repository: ranger Description --- **Problem Statement:** Created a service/policy from a user. While trying to delete the same user, I was unable to delete the user as "added_by_id" and "upd_by_id" field are having references to x_portal_user(id). so due to foreign key constraint ranger doesn't allow user deletion. This is related to RANGER-2203 as policy structure has been changed and new tables are also having "added_by_id" and "upd_by_id" columns. **Proposed Solution:** Proposed solution shall set the added_by_id and upd_by_id field value with the user id of current logged in user. Diffs - security-admin/src/main/java/org/apache/ranger/service/XPortalUserService.java 5f5b5c4c1 Diff: https://reviews.apache.org/r/69253/diff/1/ Testing --- Steps performed for Ranger-admin(with patch): 1. Created Build with patch and untar the build. 2. Opened install.properties and provided db configuration in install.properties 3. Called setup.sh 4. Started Ranger admin. 5. Logged in from admin user and created a user testuser1 with admin role. Loggedout from admin account. 6. Logged in from testuser1 id and created a service. Loggedout from testuser1 account. 7. Logged in from admin user and delete user testuser1. **Expected Behavior:** testuser1 should be deleted and foreign key references should be updated. **Actual Behavior:** testuser1 was deleted and foreign key references was updated. Thanks, Pradeep Agrawal
Review Request 69238: RANGER-2275: Make db_setup retry delay configurable
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69238/ --- Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Bugs: RANGER-2275 https://issues.apache.org/jira/browse/RANGER-2275 Repository: ranger Description --- **Problem Statement**: In an HA setup, Ranger Admin on both master nodes attempts to create/patch the DB and change passwords. If one of them detects contention for an operation, it sleeps and re-checks later. The length of sleep is fixed at 120 seconds It would be better to make this delay configurable (in seconds) so that user can adjust the delay value as per his environment. **Proposed Solution:** The proposed patch introduces a parameter 'PATCH_RETRY_INTERVAL' in install.properties file which has default value 120 seconds. This property is read during the run of db_setup.py and script shall set 'retryPatchAfterSeconds' variable with the given value of 'PATCH_RETRY_INTERVAL'. If PATCH_RETRY_INTERVAL is not a number then default value will be 120. If PATCH_RETRY_INTERVAL is removed then default value will be 120. Diffs - security-admin/scripts/db_setup.py 3450fa690 security-admin/scripts/install.properties 30b946b31 Diff: https://reviews.apache.org/r/69238/diff/1/ Testing --- Tested patch on local env by setting different values in the property PATCH_RETRY_INTERVAL. Thanks, Pradeep Agrawal
Re: Review Request 69187: RANGER-2268: Optimize policy and tags migration to new schema
> On Oct. 26, 2018, 5:03 a.m., Pradeep Agrawal wrote: > > Ship It! Tested patch with 5000 policies and it took 14 minutes only while earlier it was taking several hours. - Pradeep --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69187/#review210078 --- On Oct. 26, 2018, 1:42 a.m., Abhay Kulkarni wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69187/ > --- > > (Updated Oct. 26, 2018, 1:42 a.m.) > > > Review request for ranger, Madhan Neethiraj, Pradeep Agrawal, and Velmurugan > Periasamy. > > > Bugs: RANGER-2268 > https://issues.apache.org/jira/browse/RANGER-2268 > > > Repository: ranger > > > Description > --- > > Policies and tags are migrated to new schema introduced in RANGER-2203 using > java patches. The time for migration can increase very rapidly as number of > migrated policies/tags increase, and may take hours for a few thousand large > policies containing many users and groups. > > The fix involves updating every policy in a new transaction to ensure that > JPA cache is kept to a minimum size. > > > Diffs > - > > > security-admin/src/main/java/org/apache/ranger/patch/PatchForUpdatingPolicyJson_J10019.java > 6a21effe4 > > security-admin/src/main/java/org/apache/ranger/patch/PatchForUpdatingTagsJson_J10020.java > 035e0239a > > > Diff: https://reviews.apache.org/r/69187/diff/1/ > > > Testing > --- > > Tested migration with a few thousand policies. Ensured that the migration > time is reduced considerably. > > > Thanks, > > Abhay Kulkarni > >
Re: Review Request 69187: RANGER-2268: Optimize policy and tags migration to new schema
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69187/#review210078 --- Ship it! Ship It! - Pradeep Agrawal On Oct. 26, 2018, 1:42 a.m., Abhay Kulkarni wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69187/ > --- > > (Updated Oct. 26, 2018, 1:42 a.m.) > > > Review request for ranger, Madhan Neethiraj, Pradeep Agrawal, and Velmurugan > Periasamy. > > > Bugs: RANGER-2268 > https://issues.apache.org/jira/browse/RANGER-2268 > > > Repository: ranger > > > Description > --- > > Policies and tags are migrated to new schema introduced in RANGER-2203 using > java patches. The time for migration can increase very rapidly as number of > migrated policies/tags increase, and may take hours for a few thousand large > policies containing many users and groups. > > The fix involves updating every policy in a new transaction to ensure that > JPA cache is kept to a minimum size. > > > Diffs > - > > > security-admin/src/main/java/org/apache/ranger/patch/PatchForUpdatingPolicyJson_J10019.java > 6a21effe4 > > security-admin/src/main/java/org/apache/ranger/patch/PatchForUpdatingTagsJson_J10020.java > 035e0239a > > > Diff: https://reviews.apache.org/r/69187/diff/1/ > > > Testing > --- > > Tested migration with a few thousand policies. Ensured that the migration > time is reduced considerably. > > > Thanks, > > Abhay Kulkarni > >
Re: Review Request 69259: RANGER-2279: Reduce the time spent changing passwords during Ranger Admin install
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69259/ --- (Updated Nov. 12, 2018, 1:36 p.m.) Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Changes --- Updated review request after testing the backword compatibility flow. Bugs: RANGER-2279 https://issues.apache.org/jira/browse/RANGER-2279 Repository: ranger Description --- **Problem Statement:** ChangePasswordUtil takes 12-14 seconds to change a password. Its first invocation takes 25-30 seconds (additional time spent creating embedded service-defs). During the initial start of Ranger Admin there are 4 passwords being changed (admin, rangerusersync, rangertagsync, keyadmin), total time for password change is ~74 seconds. We need to Reduce the time spent changing passwords during Ranger Admin Install. **Proposed Solution:** Based on ChangePasswordUtil's logs, most of the time is spent on setup, and the actual password change is very quick. So it will be better to change multiple passwords in one request (by sending multiple users passwords config in a single run of ChangePasswordUtil) Diffs (updated) - security-admin/scripts/db_setup.py f84c8ca3a security-admin/scripts/setup.sh 0b10e0fdc security-admin/src/main/java/org/apache/ranger/patch/cliutil/ChangePasswordUtil.java 9d3ce593c Diff: https://reviews.apache.org/r/69259/diff/2/ Changes: https://reviews.apache.org/r/69259/diff/1-2/ Testing --- **Steps performed for Ranger-admin(with patch):** 1. Created Build with patch and untar the build. 2. Opened install.properties and provided db configuration in install.properties 3. Called setup.sh **Expected Behavior:** 1. There should be a single call of change password request and password of all the default admin's should get changed. 2. After starting the ranger-admin, admin users should able to login from the new passwords and request with old password should fail. **Actual Behavior:** 1. From the ranger installation logs it was observed that there was a single call of change password request and password of all the default admins were succesfully changed. 2. After starting the ranger-admin, all default admin users were able to login from the new passwords and request with old password failed. **Note:** Single user password changed request code still exist and I have not removed it as people can use that call flow also if they want to change only one user password anytime after the ranger installation. Thanks, Pradeep Agrawal
Review Request 69347: RANGER-2286: Ranger install may be prevented by leftover DB entry
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69347/ --- Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Bugs: RANGER-2286 https://issues.apache.org/jira/browse/RANGER-2286 Repository: ranger Description --- **Problem Statement:** Before execution of any db/java patch db_setup.py file makes an entry in x_db_version_h table for that patch version and keep status active='N'. after this it steps ahead and try to apply the patch. if patch gets applies then active status is updated to 'Y'. if patch execution fails then it deletes that entry and terminates ranger installation so that user can retry the ranger install process. Sometime due to network or a runtime error Ranger could not change the active status to 'Y' or delete the entry in that case ranger install may be prevented due to leftover DB entry as the next install attempt may wait for the status to be 'Y' while it could be possible that there would not be any process which can apply the patch(this will happen in Non-HA case). This prevents any further attempt to install/start Ranger without manually deleting the entry from the database. **Proposed Solution:** if entries made by a host is older than 10 minute then ranger setup process will try to delete such entry and retry the patch execution. Patch execution process shall have following rules after this patch. 1) Before execution a patch setup process will check whether a patch is applied or not; if patch is already applied then it will move to next patch. 2) If patch is not applied then it will check whether patch status entry exist with active='N' in x_db_version_h table from any of the host. 3) If patch status entry with active='N' exist in x_db_version_h table for a different host then ranger setup process will wait untill the another host applies the patch successfully and update the patch status entry to active='Y'. Please note that patch recheck such entry in every 2 minutes(default recheck interval) 4) If patch status entry with active='N' exist in x_db_version_h table for the same host then ranger setup process will wait for maximum 10 minutes. After that ranger process will delete that entry and check the entry again. if entry does not exist then it will try to apply the patch again else it will wait for forever or unless you delete such entries manually. Please note that ranger will be checking the status entry every 2 minutes untill 10 minute wait time expires. **Note** 1) Patch has been provided for ranger supported all the db flavors. 2) Cureent patch also removes several lines of audit db related dead code which is not required now. Diffs - security-admin/scripts/db_setup.py 73b24aca0 Diff: https://reviews.apache.org/r/69347/diff/1/ Testing --- **Steps performed for Ranger-admin ** 1. To reproduce the case first installed ranger-0.7: 2. Now from the master branch. 3. Created Build with patch and untar the build. 4. Opened install.properties and provided db configuration in install.properties which was provided in ranger-0.7 branch build. 5. Called setup.sh 6. Ranger setup process shall try to apply the new db/java patches. 7. As soon as setup process started executing the patch J10014, terminated the script by using ctrl+c 8. logged into ranger db and ensured that J10014 patch entry has active='N' 9. Called the setup.sh again 10. setup process waited for the patch J10014 status for sometime and then it deleted the entry of J10014 patch from x_db_version_h table. 11. setup process continues and applied the patch J10014 successfully. 12. setup process completed successfully after applying all the patches. 13. started ranger-admin and was able to see the ranger-admin dashboard page. Thanks, Pradeep Agrawal
Re: Review Request 69347: RANGER-2286: Ranger install may be prevented by leftover DB entry
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69347/ --- (Updated Nov. 16, 2018, 5:23 a.m.) Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Changes --- Address review comment's second point and have updated the patch Bugs: RANGER-2286 https://issues.apache.org/jira/browse/RANGER-2286 Repository: ranger Description --- **Problem Statement:** Before execution of any db/java patch db_setup.py file makes an entry in x_db_version_h table for that patch version and keep status active='N'. after this it steps ahead and try to apply the patch. if patch gets applies then active status is updated to 'Y'. if patch execution fails then it deletes that entry and terminates ranger installation so that user can retry the ranger install process. Sometime due to network or a runtime error Ranger could not change the active status to 'Y' or delete the entry in that case ranger install may be prevented due to leftover DB entry as the next install attempt may wait for the status to be 'Y' while it could be possible that there would not be any process which can apply the patch(this will happen in Non-HA case). This prevents any further attempt to install/start Ranger without manually deleting the entry from the database. **Proposed Solution:** if entries made by a host is older than 10 minute then ranger setup process will try to delete such entry and retry the patch execution. Patch execution process shall have following rules after this patch. 1) Before execution a patch setup process will check whether a patch is applied or not; if patch is already applied then it will move to next patch. 2) If patch is not applied then it will check whether patch status entry exist with active='N' in x_db_version_h table from any of the host. 3) If patch status entry with active='N' exist in x_db_version_h table for a different host then ranger setup process will wait untill the another host applies the patch successfully and update the patch status entry to active='Y'. Please note that patch recheck such entry in every 2 minutes(default recheck interval) 4) If patch status entry with active='N' exist in x_db_version_h table for the same host then ranger setup process will wait for maximum 10 minutes. After that ranger process will delete that entry and check the entry again. if entry does not exist then it will try to apply the patch again else it will wait for forever or unless you delete such entries manually. Please note that ranger will be checking the status entry every 2 minutes untill 10 minute wait time expires. **Note** 1) Patch has been provided for ranger supported all the db flavors. 2) Cureent patch also removes several lines of audit db related dead code which is not required now. Diffs (updated) - security-admin/scripts/db_setup.py 73b24aca0 security-admin/scripts/install.properties f52c6b298 Diff: https://reviews.apache.org/r/69347/diff/2/ Changes: https://reviews.apache.org/r/69347/diff/1-2/ Testing --- **Steps performed for Ranger-admin ** 1. To reproduce the case first installed ranger-0.7: 2. Now from the master branch. 3. Created Build with patch and untar the build. 4. Opened install.properties and provided db configuration in install.properties which was provided in ranger-0.7 branch build. 5. Called setup.sh 6. Ranger setup process shall try to apply the new db/java patches. 7. As soon as setup process started executing the patch J10014, terminated the script by using ctrl+c 8. logged into ranger db and ensured that J10014 patch entry has active='N' 9. Called the setup.sh again 10. setup process waited for the patch J10014 status for sometime and then it deleted the entry of J10014 patch from x_db_version_h table. 11. setup process continues and applied the patch J10014 successfully. 12. setup process completed successfully after applying all the patches. 13. started ranger-admin and was able to see the ranger-admin dashboard page. Thanks, Pradeep Agrawal
Review Request 69259: RANGER-2279: Reduce the time spent changing passwords during Ranger Admin install
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69259/ --- Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Bugs: RANGER-2279 https://issues.apache.org/jira/browse/RANGER-2279 Repository: ranger Description --- **Problem Statement:** ChangePasswordUtil takes 12-14 seconds to change a password. Its first invocation takes 25-30 seconds (additional time spent creating embedded service-defs). During the initial start of Ranger Admin there are 4 passwords being changed (admin, rangerusersync, rangertagsync, keyadmin), total time for password change is ~74 seconds. We need to Reduce the time spent changing passwords during Ranger Admin Install. **Proposed Solution:** Based on ChangePasswordUtil's logs, most of the time is spent on setup, and the actual password change is very quick. So it will be better to change multiple passwords in one request (by sending multiple users passwords config in a single run of ChangePasswordUtil) Diffs - security-admin/scripts/db_setup.py f84c8ca3a security-admin/scripts/setup.sh 0b10e0fdc security-admin/src/main/java/org/apache/ranger/patch/cliutil/ChangePasswordUtil.java 9d3ce593c Diff: https://reviews.apache.org/r/69259/diff/1/ Testing --- **Steps performed for Ranger-admin(with patch):** 1. Created Build with patch and untar the build. 2. Opened install.properties and provided db configuration in install.properties 3. Called setup.sh **Expected Behavior:** 1. There should be a single call of change password request and password of all the default admin's should get changed. 2. After starting the ranger-admin, admin users should able to login from the new passwords and request with old password should fail. **Actual Behavior:** 1. From the ranger installation logs it was observed that there was a single call of change password request and password of all the default admins were succesfully changed. 2. After starting the ranger-admin, all default admin users were able to login from the new passwords and request with old password failed. **Note:** Single user password changed request code still exist and I have not removed it as people can use that call flow also if they want to change only one user password anytime after the ranger installation. Thanks, Pradeep Agrawal
Review Request 68974: RANGER-2245: Exclude Jetty libraries
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68974/ --- Review request for ranger, Ankita Sinha, deepak sharma, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, suja s, and Velmurugan Periasamy. Bugs: RANGER-2245 https://issues.apache.org/jira/browse/RANGER-2245 Repository: ranger Description --- **Problem Statement:** Ranger does not use Jetty, it uses tomcat, so jetty libraries might be coming in as transient dependencies. In that case, we can exclude these libraries from the ranger admin packaging. **Proposed solution:** Exclude jetty jars from security-admin/pom.xml Diffs - security-admin/pom.xml 84bbd6650 Diff: https://reviews.apache.org/r/68974/diff/1/ Testing --- Steps performed for Ranger-admin(with patch): 1. Created Build with patch and untar the build. 2. Opened install.properties and provided db configuration in install.properties 3. Called setup.sh **Expected Behavior:** Ranger-admin installation should finish successfully and jetty jars should not be available at location: ranger-2.0.0-SNAPSHOT-admin/ews/webapp/WEB-INF/lib **Actual Behavior:** Ranger-admin Installation finished successfully and jetty jars was not available at location: ranger-2.0.0-SNAPSHOT-admin/ews/webapp/WEB-INF/lib Note: After setup, started Ranger Thanks, Pradeep Agrawal
Re: Review Request 68974: RANGER-2245: Exclude Jetty libraries
> On Oct. 10, 2018, 3:26 p.m., Velmurugan Periasamy wrote: > > security-admin/pom.xml > > Lines 545 (patched) > > <https://reviews.apache.org/r/68974/diff/1/?file=2095947#file2095947line545> > > > > Could you please make sure this is merged in 1.x branches as well? > > Thanks. Yes, will commit in ranger-1, ranger-1.1 and ranger-1.2 branch. - Pradeep --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68974/#review209409 ------- On Oct. 10, 2018, 2:12 p.m., Pradeep Agrawal wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68974/ > --- > > (Updated Oct. 10, 2018, 2:12 p.m.) > > > Review request for ranger, Ankita Sinha, deepak sharma, Gautam Borad, Abhay > Kulkarni, Madhan Neethiraj, Mehul Parikh, suja s, and Velmurugan Periasamy. > > > Bugs: RANGER-2245 > https://issues.apache.org/jira/browse/RANGER-2245 > > > Repository: ranger > > > Description > --- > > **Problem Statement:** Ranger does not use Jetty, it uses tomcat, so jetty > libraries might be coming in as transient dependencies. In that case, we can > exclude these libraries from the ranger admin packaging. > > **Proposed solution:** Exclude jetty jars from security-admin/pom.xml > > > Diffs > - > > security-admin/pom.xml 84bbd6650 > > > Diff: https://reviews.apache.org/r/68974/diff/1/ > > > Testing > --- > > Steps performed for Ranger-admin(with patch): > 1. Created Build with patch and untar the build. > 2. Opened install.properties and provided db configuration in > install.properties > 3. Called setup.sh > > > **Expected Behavior:** > Ranger-admin installation should finish successfully and jetty jars should > not be available at location: > ranger-2.0.0-SNAPSHOT-admin/ews/webapp/WEB-INF/lib > > > **Actual Behavior:** > Ranger-admin Installation finished successfully and jetty jars was not > available at location: > ranger-2.0.0-SNAPSHOT-admin/ews/webapp/WEB-INF/lib > > Note: After setup, started Ranger > > > Thanks, > > Pradeep Agrawal > >
Re: [VOTE] Apache Ranger Release 1.2.0-rc1
+1 for the release of 1.2.0-rc1 Able to download the source and build with unit test cases Able to install, setup and start Ranger Able to do CRUD operation on service, policy, user, and groups. Thanks, Vel for putting Ranger 1.2.0 for release. Regards, Pradeep On Mon, 1 Oct 2018 at 11:27, Nixon Rodrigues < nixon.rodrig...@freestoneinfotech.com> wrote: > +1 for release of 1.2.0-rc1, > > Thanks Vel for putting Ranger 1.2.0 for release > > Nixon Rodrigues > > On Mon, Oct 1, 2018 at 8:01 AM Koji Kawamura > wrote: > > > +1 > > > > verified build & signature > > > > Thanks, > > Koji > > On Mon, Oct 1, 2018 at 8:58 AM Selvamohan Neethiraj > > > wrote: > > > > > > +1 for Apache Ranger 1.2.0-rc1 release …. > > > > > > verified build & signature > > > > > > Thanks, > > > Selva- > > > > > > > On Sep 29, 2018, at 1:27 PM, Ramesh Mani > > wrote: > > > > > > > > +1 for release Apache Ranger 1.2.0-rc1 > > > > > > > > Verified build for the source with rat plugin and unit tests > > > > Verified some source files > > > > Verified PGP signature > > > > Verified MD5/SHA > > > > > > > > Thanks, > > > > Ramesh > > > > > > > > > > > > > > > > On 9/27/18, 4:24 PM, "Velmurugan Periasamy" wrote: > > > > > > > >> Hello Rangers: > > > >> > > > >> Thank you for your contribution to Apache Ranger community. Apache > > Ranger > > > >> 1.2.0 release candidate #1 is now available for a vote within dev > > > >> community. > > > >> > > > >> Links to RC1 release artifacts are given below. Kindly request all > > > >> Rangers (Dev's & PMC members) to review and vote on this release. > > > >> > > > >> > > > >> Git tag for the release: > > > >> https://github.com/apache/ranger/tree/ranger-1.2.0-rc1 (last commit > > id: > > > >> 39ec5a38913e1d852cffecbdb8688b2370b6318f) > > > >> > > > >> > > > >> Sources for the release: > > > >> > > > https://dist.apache.org/repos/dist/dev/ranger/1.2.0-rc1/apache-ranger-1.2. > > > >> 0.tar.gz > > > >> > > > >> > > > >> Source release verification: > > > >> > > > >> PGP Signature: > > > >> > > > https://dist.apache.org/repos/dist/dev/ranger/1.2.0-rc1/apache-ranger-1.2. > > > >> 0.tar.gz.asc > > > >> > > > >> MD5/SHA Hashes: > > > >> > > > https://dist.apache.org/repos/dist/dev/ranger/1.2.0-rc1/apache-ranger-1.2. > > > >> 0.tar.gz.mds > > > >> > > > https://dist.apache.org/repos/dist/dev/ranger/1.2.0-rc1/apache-ranger-1.2. > > > >> 0.tar.gz.sha1 > > > >> > > > https://dist.apache.org/repos/dist/dev/ranger/1.2.0-rc1/apache-ranger-1.2. > > > >> 0.tar.gz.sha256 > > > >> > > > https://dist.apache.org/repos/dist/dev/ranger/1.2.0-rc1/apache-ranger-1.2. > > > >> 0.tar.gz.sha512 > > > >> > > > >> > > > >> Keys to verify the signature of the release artifact are available > at: > > > >> https://dist.apache.org/repos/dist/release/ranger/KEYS > > > >> > > > >> > > > >> Release Notes: > > > >> > > > https://cwiki.apache.org/confluence/display/RANGER/Apache+Ranger+1.2.0+-+R > > > >> elease+Notes > > > >> > > > >> > > > >> Build verification steps can be found at: > > > >> http://ranger.apache.org/quick_start_guide.html > > > >> > > > >> > > > >> The vote will be open for at least 72 hours or until necessary > number > > of > > > >> votes are reached. > > > >> [ ] +1 approve > > > >> [ ] +0 no opinion > > > >> [ ] -1 disapprove (and reason why) > > > >> > > > >> Here is my +1 > > > >> > > > >> Thank you, > > > >> Vel > > > >> > > > > > > > > > >
Re: Review Request 69677: RANGER-2287: Improve and optimize db_setup.py file code
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69677/ --- (Updated Jan. 15, 2019, 12:55 p.m.) Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Changes --- Updated patch as per the feedback. Bugs: RANGER-2287 https://issues.apache.org/jira/browse/RANGER-2287 Repository: ranger Description --- **Problem Statement:** There are lot of repeated code in db_setup.py which can be removed which shall help developers to make any changes in db_setup.py in future. **Proposed Solution:** Proposed patch shall remove the db setup methods of each db flavor and shall use a single method for a specific work for each db flavor. Based on the db flavor, config values shall be populated and handled in the code after this patch. Diffs (updated) - security-admin/scripts/db_setup.py f1223b38c Diff: https://reviews.apache.org/r/69677/diff/2/ Changes: https://reviews.apache.org/r/69677/diff/1-2/ Testing --- **Use Cases covered for all the db flavors:** *1. Fresh installation(Ranger 2.0):* Tested patch with fresh installation of ranger admin. *2. Upgrade(from 0.7 to 2.0):* Installed Ranger from 0.7 branch and used same db config on Ranger 2.0 installation config and run the setup.sh. Ranger was upgraded successfully. Thanks, Pradeep Agrawal
Re: Review Request 69677: RANGER-2287: Improve and optimize db_setup.py file code
> On Jan. 8, 2019, 11:48 a.m., Zsombor Gegesy wrote:
> > It's great news, that you could delete thousands of lines of repetitive
> > code, however you could achieve more, if instead of putting everything into
> > one class, and put
> > '''
> > if self.XA_DB_FLAVOR == DB_MYSQL:
> >...
> > elif self.XA_DB_FLAVOR == DB_POSTGRES:
> >...
> > '''
> >
> > You can write
> >self.do_something(...)
> >
> > and implement do_something differently in the MySQL/PostgreSQL/Oracle
> > specific adapter class
>
> Pradeep Agrawal wrote:
> There shall be too many self.do_something(...) function I have to write
> which shall look like the previous code. Can you review it once again and let
> me know with few examples.
>
> Zsombor Gegesy wrote:
> Maybe you can add:
> '''
> def execute_query(self, query):
> ''' Execute query and return the output as a string '''
> get_cmd = self.get_jisql_cmd(self.db_user, self.db_password,
> self.db_name)
> if is_unix:
> full_command = get_cmd + " -query \"" + query + "\""
> elif os_name == "WINDOWS":
> full_command = get_cmd + " -query \"" + query + "\" -c ;"
> else:
> raise Exception("This OS is not supported!")
> jisql_log(full_command, self.db_password)
> output = check_output(query)
> return output
>
> def execute_update(self, update):
> ''' Execute the update query and return the error code'''
> get_cmd = self.get_jisql_cmd(self.db_user, self.db_password,
> self.db_name)
> if is_unix:
> full_command = get_cmd + " -query \"" + update + "\""
> jisql_log(full_command, self.db_password)
> return subprocess.call(shlex.split(query))
> elif os_name == "WINDOWS":
> full_command = get_cmd + " -query \"" + update + "\" -c ;"
> jisql_log(full_command, self.db_password)
> ret = subprocess.call(query)
> raise Exception("This OS is not supported!")
> '''
>
> So you can get rid of lot's of repeating code around to support Windows.
>
> And for the db changes, I would imagine something like this:
>
> '''
> class BaseDB(object):
>
> @abstractmethod
> def get_stale_patch_query(self, version, client_host,
> stalePatchEntryHoldTimeInMinutes):
> pass
>
>
> class MysqlConf(BaseDB):
>
> def get_stale_patch_query(self, version, client_host,
> stalePatchEntryHoldTimeInMinutes):
> return "select version from x_db_version_h where version = '%s'
> and active = 'N' and updated_by='%s' and
> TIMESTAMPDIFF(MINUTE,inst_at,CURRENT_TIMESTAMP)>=%s;" % (version,
> client_host, stalePatchEntryHoldTimeInMinutes)
>
> '''
>
>
> So you can write:
>
> '''
> output =
> self.execute_query(self.get_stale_patch_query(version,client_host,stalePatchEntryHoldTimeInMinutes))
> ...
> '''
>
> What do you think, does it makes sense?
Can you please review the updated patch again.
- Pradeep
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69677/#review211760
---
On Jan. 15, 2019, 12:55 p.m., Pradeep Agrawal wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69677/
> ---
>
> (Updated Jan. 15, 2019, 12:55 p.m.)
>
>
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh,
> Nikhil P, Ramesh Mani, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-2287
> https://issues.apache.org/jira/browse/RANGER-2287
>
>
> Repository: ranger
>
>
> Description
> ---
>
> **Problem Statement:** There are lot of repeated code in db_setup.py which
> can be removed which shall help developers to make any changes in db_setup.py
> in future.
>
> **Proposed Solution:** Proposed patch shall remove the db setup methods of
> each db flavor and shall use a single method for a specific work for each db
> flavor. Based on the db flavor, config values shall be populated and handled
> in the code after this patch.
>
>
> Diffs
> -
>
> security-admin/scripts/db_setup.py f1223b38c
>
>
> Diff: https://reviews.apache.org/r/69677/diff/2/
>
>
> Testing
> ---
>
> **Use Cases covered for all the db flavors:**
> *1. Fresh installation(Ranger 2.0):* Tested patch with fresh installation of
> ranger admin.
> *2. Upgrade(from 0.7 to 2.0):* Installed Ranger from 0.7 branch and used same
> db config on Ranger 2.0 installation config and run the setup.sh. Ranger was
> upgraded successfully.
>
>
> Thanks,
>
> Pradeep Agrawal
>
>
Re: Review Request 69677: RANGER-2287: Improve and optimize db_setup.py file code
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69677/ --- (Updated Jan. 22, 2019, 1:47 p.m.) Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Changes --- Updated patch after one more round of testing Bugs: RANGER-2287 https://issues.apache.org/jira/browse/RANGER-2287 Repository: ranger Description --- **Problem Statement:** There are lot of repeated code in db_setup.py which can be removed which shall help developers to make any changes in db_setup.py in future. **Proposed Solution:** Proposed patch shall remove the db setup methods of each db flavor and shall use a single method for a specific work for each db flavor. Based on the db flavor, config values shall be populated and handled in the code after this patch. Diffs (updated) - security-admin/scripts/db_setup.py f1223b38c Diff: https://reviews.apache.org/r/69677/diff/3/ Changes: https://reviews.apache.org/r/69677/diff/2-3/ Testing --- **Use Cases covered for all the db flavors:** *1. Fresh installation(Ranger 2.0):* Tested patch with fresh installation of ranger admin. *2. Upgrade(from 0.7 to 2.0):* Installed Ranger from 0.7 branch and used same db config on Ranger 2.0 installation config and run the setup.sh. Ranger was upgraded successfully. Thanks, Pradeep Agrawal
Re: Review Request 69730: RANGER-2320: Make db schema patches script idempotent for all DB Flavors
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69730/ --- (Updated Jan. 23, 2019, 8:54 a.m.) Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Changes --- updated patch after syncing the local ranger repo with the latest ranger master. Bugs: RANGER-2320 https://issues.apache.org/jira/browse/RANGER-2320 Repository: ranger Description --- RANGER-2291 covers changes only in optimized DB schema script, existing DB patches script should be reviewed and if required these scripts also should be made idempotent. Diffs (updated) - security-admin/db/mysql/patches/009-updated_schema.sql 14ec1d872 security-admin/db/mysql/patches/013-permissionmodel.sql 1c6bcfd13 security-admin/db/mysql/patches/020-datamask-policy.sql 87c0a183a security-admin/db/mysql/patches/030-policy-labels-schema.sql 33af2a858 security-admin/db/mysql/patches/035-update-schema-for-x-policy.sql 84db52604 security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 98c45b05d security-admin/db/postgres/patches/020-datamask-policy.sql 795721c99 security-admin/db/postgres/patches/035-update-schema-for-x-policy.sql 61c7986e4 security-admin/db/sqlserver/patches/020-datamask-policy.sql c50d2b83d security-admin/db/sqlserver/patches/035-update-schema-for-x-policy.sql bd646d695 Diff: https://reviews.apache.org/r/69730/diff/2/ Changes: https://reviews.apache.org/r/69730/diff/1-2/ Testing --- Tested the upgrade case from ranger-0.7 to ranger-2.0 for mysql, postgres and sqlserver db. Thanks, Pradeep Agrawal
Re: Review Request 69468: RANGER-2295: Set specific Ranger version in patches status entry table
> On Dec. 12, 2018, 5:36 p.m., Velmurugan Periasamy wrote:
> > security-admin/scripts/db_setup.py
> > Lines 1024 (patched)
> > <https://reviews.apache.org/r/69468/diff/1/?file=2110824#file2110824line1024>
> >
> > Is it enough to consider only localhost?
Yes, because the code suppose to update only those entries which are made by
ranger core db schema. Ranger core db schema has updated_by='localhost' as hard
coded value in the script.
Ref:
https://github.com/apache/ranger/blob/master/security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql#L1359
- Pradeep
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69468/#review211246
---
On Nov. 28, 2018, 10:19 a.m., Pradeep Agrawal wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69468/
> ---
>
> (Updated Nov. 28, 2018, 10:19 a.m.)
>
>
> Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh,
> Nikhil P, Ramesh Mani, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-2295
> https://issues.apache.org/jira/browse/RANGER-2295
>
>
> Repository: ranger
>
>
> Description
> ---
>
> **Problem Statement:** DB setup script(db_setup.py) looks for a specific
> version (For example: "Ranger 2.0.0-SNAPSHOT") to decide if patches need to
> be applied or not.
>
> For example:
> select version from x_db_version_h where version = 'DB_PATCHES' and inst_by =
> 'Ranger 2.0.0-SNAPSHOT' and active = 'Y';
> select version from x_db_version_h where version = 'JAVA_PATCHES' and inst_by
> = 'Ranger 2.0.0-SNAPSHOT' and active = 'Y';
>
>
> However, the optimized schema creation script comes with a generic version
> (For example: "Ranger 1.0.0"):
>
>
> INSERT INTO x_db_version_h
> (version,inst_at,inst_by,updated_at,updated_by,active) VALUES
> ('DB_PATCHES',CURRENT_TIMESTAMP,'Ranger
> 1.0.0',CURRENT_TIMESTAMP,'localhost','Y');
> INSERT INTO x_db_version_h
> (version,inst_at,inst_by,updated_at,updated_by,active) VALUES
> ('JAVA_PATCHES',CURRENT_TIMESTAMP,'Ranger
> 1.0.0',CURRENT_TIMESTAMP,'localhost','Y');
>
> The result is that a separate check is executed for each patch, which takes
> time. It will be good if the status entries have the exact ranger version
> rather a base version.
>
> **Proposed Solution:** The propsed solution includes following changes:
> After core db schema file(ranger_core_db_*.sql) is imported patch shall
> update the sql/java patches entry version with the exact version+build being
> used. Once the exact version is updated then when DB setup script will look
> for a specific version then it will find a matching entry and skip the all
> patches check.
>
>
> Diffs
> -
>
> security-admin/scripts/db_setup.py 2bda1a8e7
>
>
> Diff: https://reviews.apache.org/r/69468/diff/1/
>
>
> Testing
> ---
>
> **Steps performed for Ranger-admin(with patch):**
>
> 1. Created Build with patch and untar the build.
> 2. Opened install.properties and provided db configuration in
> install.properties
> 3. Called setup.sh
>
> **Expected Behavior:**
> 1. There should be a single call of db schema setup and it should not try to
> apply/check all the db patches entries.
>
> **Actual Behavior:**
> 2. After importing the db schema file, ranger checked for entries of
> 'DB_PATCHES' and 'JAVA_PATCHES' for the current ranger version and skipped
> checking entries of every db and java patches.
>
>
> **Note:**
> Patch has been tested for all the db flavor.
>
>
> Thanks,
>
> Pradeep Agrawal
>
>
Review Request 69453: RANGER-2291: Make optimized db schema script idempotent for all DB Flavors
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69453/ --- Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Bugs: RANGER-2291 https://issues.apache.org/jira/browse/RANGER-2291 Repository: ranger Description --- **Problem Statement:** Currently Ranger core db schema is not idempotent for all db flavors. Ranger core DB schema for Oracle and SQL anywhere flavor may fail to execute if we execute them again for the same DB flavor. **Proposed Solution:** I have added drop statements before the create statements for the various objects(table/constraints etc) Diffs - security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql a4fa1305e security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 0949cbd1d security-admin/db/oracle/patches/009-updated_schema.sql 7e21f69e1 security-admin/db/oracle/patches/013-permissionmodel.sql 4ac7901ba security-admin/db/oracle/patches/016-updated-schema-for-tag-based-policy.sql 12627f589 security-admin/db/oracle/patches/020-datamask-policy.sql 8448a8568 security-admin/db/oracle/patches/022-split-service-table.sql 9b4f69c4c security-admin/db/oracle/patches/025-create-schema-for-plugin-info.sql bedd0a2ef security-admin/db/oracle/patches/030-policy-labels-schema.sql 894b9346f security-admin/db/oracle/patches/031-create-schema-for-usersync-audit-info.sql cb52065c6 security-admin/db/oracle/patches/035-update-schema-for-x-policy.sql c75e62089 security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql a0e02e0e0 security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql db8ebc343 security-admin/db/sqlanywhere/patches/016-updated-schema-for-tag-based-policy.sql f3b64d003 security-admin/db/sqlanywhere/patches/020-datamask-policy.sql fe6fa9f61 security-admin/db/sqlanywhere/patches/022-split-service-table.sql d32966d8c security-admin/db/sqlanywhere/patches/025-create-schema-for-plugin-info.sql 6e9477984 security-admin/db/sqlanywhere/patches/030-policy-labels-schema.sql b2ed2386d security-admin/db/sqlanywhere/patches/031-create-schema-for-usersync-audit-info.sql 8ed84e302 security-admin/db/sqlanywhere/patches/035-update-schema-for-x-policy.sql c079014df security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql 522b57b03 Diff: https://reviews.apache.org/r/69453/diff/1/ Testing --- **Steps Performed (with patch) :** 1. After Build untar the Ranger module and updated install.properties for Oracle DB flavor. 2. Called setup.sh to install Ranger. 3. Started Ranger admin and logged in to check the installation is working or not. 4. create a user 'testuser1'. 5. Stopped Ranger admin. 6. Executed given JISQL command again to import core db schema with the same config (for the same ranger db and user): /usr/jdk64/jdk1.8.0_112/bin/java -Djava.security.egd=file:///dev/urandom -cp /usr/share/java/ojdbc6.jar:/tmp/ranger-2.0.0-SNAPSHOT-admin/jisql/lib/* org.apache.util.sql.Jisql -driver oraclethin -cstring jdbc:oracle:thin:@localhost -u 'ranger112701' -p '' -noheader -trim -input /tmp/ranger-2.0.0-SNAPSHOT-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql -c \; **Expected behavior:** Command should able to execute core db schema file again and should not fail. user testuser1 should not appear in user/groups page of ranger admin **Actual behavior:** Command executed successfully and recreated all the tables again, was able to see new entries and able to login to ranger admin. 'testuser1' was not seen in the ranger admin. Thanks, Pradeep Agrawal
Review Request 69468: RANGER-2295: Set specific Ranger version in patches status entry table
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69468/
---
Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh,
Nikhil P, Ramesh Mani, and Velmurugan Periasamy.
Bugs: RANGER-2295
https://issues.apache.org/jira/browse/RANGER-2295
Repository: ranger
Description
---
**Problem Statement:** DB setup script(db_setup.py) looks for a specific
version (For example: "Ranger 2.0.0-SNAPSHOT") to decide if patches need to be
applied or not.
For example:
select version from x_db_version_h where version = 'DB_PATCHES' and inst_by =
'Ranger 2.0.0-SNAPSHOT' and active = 'Y';
select version from x_db_version_h where version = 'JAVA_PATCHES' and inst_by =
'Ranger 2.0.0-SNAPSHOT' and active = 'Y';
However, the optimized schema creation script comes with a generic version (For
example: "Ranger 1.0.0"):
INSERT INTO x_db_version_h
(version,inst_at,inst_by,updated_at,updated_by,active) VALUES
('DB_PATCHES',CURRENT_TIMESTAMP,'Ranger
1.0.0',CURRENT_TIMESTAMP,'localhost','Y');
INSERT INTO x_db_version_h
(version,inst_at,inst_by,updated_at,updated_by,active) VALUES
('JAVA_PATCHES',CURRENT_TIMESTAMP,'Ranger
1.0.0',CURRENT_TIMESTAMP,'localhost','Y');
The result is that a separate check is executed for each patch, which takes
time. It will be good if the status entries have the exact ranger version
rather a base version.
**Proposed Solution:** The propsed solution includes following changes:
After core db schema file(ranger_core_db_*.sql) is imported patch shall update
the sql/java patches entry version with the exact version+build being used.
Once the exact version is updated then when DB setup script will look for a
specific version then it will find a matching entry and skip the all patches
check.
Diffs
-
security-admin/scripts/db_setup.py 2bda1a8e7
Diff: https://reviews.apache.org/r/69468/diff/1/
Testing
---
**Steps performed for Ranger-admin(with patch):**
1. Created Build with patch and untar the build.
2. Opened install.properties and provided db configuration in install.properties
3. Called setup.sh
**Expected Behavior:**
1. There should be a single call of db schema setup and it should not try to
apply/check all the db patches entries.
**Actual Behavior:**
2. After importing the db schema file, ranger checked for entries of
'DB_PATCHES' and 'JAVA_PATCHES' for the current ranger version and skipped
checking entries of every db and java patches.
**Note:**
Patch has been tested for all the db flavor.
Thanks,
Pradeep Agrawal
Review Request 69526: RANGER-2308: User role user should not able to access usersync audit report if it does not have permissions on the audit module.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69526/ --- Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Bugs: RANGER-2308 https://issues.apache.org/jira/browse/RANGER-2308 Repository: ranger Description --- Currently user is having default access to usersync audit report but it be should able to access the report only when he is having access in audit module. without that user can't see details in the UI which is not as per the default behaviour of dashboard for user role users. Diffs - security-admin/src/main/java/org/apache/ranger/biz/AssetMgr.java 941691aaa security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java e1a6b5859 security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java 471052f62 Diff: https://reviews.apache.org/r/69526/diff/1/ Testing --- Tested at local with patch. Thanks, Pradeep Agrawal
Re: Review Request 69526: RANGER-2308: User role user should not able to access usersync audit report if it does not have permissions on the audit module.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69526/ --- (Updated Dec. 10, 2018, 6 a.m.) Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Changes --- Updated patch. Bugs: RANGER-2308 https://issues.apache.org/jira/browse/RANGER-2308 Repository: ranger Description --- Currently user is having default access to usersync audit report but it be should able to access the report only when he is having access in audit module. without that user can't see details in the UI which is not as per the default behaviour of dashboard for user role users. Diffs (updated) - security-admin/src/main/java/org/apache/ranger/biz/AssetMgr.java 941691aaa security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java e1a6b5859 security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 865e115d3 security-admin/src/test/java/org/apache/ranger/biz/TestXUserMgr.java 471052f62 security-admin/src/test/java/org/apache/ranger/rest/TestServiceREST.java a8e6e61a0 Diff: https://reviews.apache.org/r/69526/diff/2/ Changes: https://reviews.apache.org/r/69526/diff/1-2/ Testing --- Tested at local with patch. Thanks, Pradeep Agrawal
Re: Review Request 69347: RANGER-2286: Ranger install may be prevented by leftover DB entry
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69347/ --- (Updated Nov. 20, 2018, 2:04 p.m.) Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Changes --- updated patch with a minor change Bugs: RANGER-2286 https://issues.apache.org/jira/browse/RANGER-2286 Repository: ranger Description --- **Problem Statement:** Before execution of any db/java patch db_setup.py file makes an entry in x_db_version_h table for that patch version and keep status active='N'. after this it steps ahead and try to apply the patch. if patch gets applies then active status is updated to 'Y'. if patch execution fails then it deletes that entry and terminates ranger installation so that user can retry the ranger install process. Sometime due to network or a runtime error Ranger could not change the active status to 'Y' or delete the entry in that case ranger install may be prevented due to leftover DB entry as the next install attempt may wait for the status to be 'Y' while it could be possible that there would not be any process which can apply the patch(this will happen in Non-HA case). This prevents any further attempt to install/start Ranger without manually deleting the entry from the database. **Proposed Solution:** if entries made by a host is older than 10 minute then ranger setup process will try to delete such entry and retry the patch execution. Patch execution process shall have following rules after this patch. 1) Before execution a patch setup process will check whether a patch is applied or not; if patch is already applied then it will move to next patch. 2) If patch is not applied then it will check whether patch status entry exist with active='N' in x_db_version_h table from any of the host. 3) If patch status entry with active='N' exist in x_db_version_h table for a different host then ranger setup process will wait untill the another host applies the patch successfully and update the patch status entry to active='Y'. Please note that patch recheck such entry in every 2 minutes(default recheck interval) 4) If patch status entry with active='N' exist in x_db_version_h table for the same host then ranger setup process will wait for maximum 10 minutes. After that ranger process will delete that entry and check the entry again. if entry does not exist then it will try to apply the patch again else it will wait for forever or unless you delete such entries manually. Please note that ranger will be checking the status entry every 2 minutes untill 10 minute wait time expires. **Note** 1) Patch has been provided for ranger supported all the db flavors. 2) Cureent patch also removes several lines of audit db related dead code which is not required now. Diffs (updated) - security-admin/scripts/db_setup.py 73b24aca0 security-admin/scripts/install.properties f52c6b298 Diff: https://reviews.apache.org/r/69347/diff/4/ Changes: https://reviews.apache.org/r/69347/diff/3-4/ Testing --- **Steps performed for Ranger-admin ** 1. To reproduce the case first installed ranger-0.7: 2. Now from the master branch. 3. Created Build with patch and untar the build. 4. Opened install.properties and provided db configuration in install.properties which was provided in ranger-0.7 branch build. 5. Called setup.sh 6. Ranger setup process shall try to apply the new db/java patches. 7. As soon as setup process started executing the patch J10014, terminated the script by using ctrl+c 8. logged into ranger db and ensured that J10014 patch entry has active='N' 9. Called the setup.sh again 10. setup process waited for the patch J10014 status for sometime and then it deleted the entry of J10014 patch from x_db_version_h table. 11. setup process continues and applied the patch J10014 successfully. 12. setup process completed successfully after applying all the patches. 13. started ranger-admin and was able to see the ranger-admin dashboard page. Thanks, Pradeep Agrawal
Re: Review Request 69347: RANGER-2286: Ranger install may be prevented by leftover DB entry
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69347/ --- (Updated Nov. 19, 2018, 11:07 a.m.) Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Changes --- Updated patch after few more testing Bugs: RANGER-2286 https://issues.apache.org/jira/browse/RANGER-2286 Repository: ranger Description --- **Problem Statement:** Before execution of any db/java patch db_setup.py file makes an entry in x_db_version_h table for that patch version and keep status active='N'. after this it steps ahead and try to apply the patch. if patch gets applies then active status is updated to 'Y'. if patch execution fails then it deletes that entry and terminates ranger installation so that user can retry the ranger install process. Sometime due to network or a runtime error Ranger could not change the active status to 'Y' or delete the entry in that case ranger install may be prevented due to leftover DB entry as the next install attempt may wait for the status to be 'Y' while it could be possible that there would not be any process which can apply the patch(this will happen in Non-HA case). This prevents any further attempt to install/start Ranger without manually deleting the entry from the database. **Proposed Solution:** if entries made by a host is older than 10 minute then ranger setup process will try to delete such entry and retry the patch execution. Patch execution process shall have following rules after this patch. 1) Before execution a patch setup process will check whether a patch is applied or not; if patch is already applied then it will move to next patch. 2) If patch is not applied then it will check whether patch status entry exist with active='N' in x_db_version_h table from any of the host. 3) If patch status entry with active='N' exist in x_db_version_h table for a different host then ranger setup process will wait untill the another host applies the patch successfully and update the patch status entry to active='Y'. Please note that patch recheck such entry in every 2 minutes(default recheck interval) 4) If patch status entry with active='N' exist in x_db_version_h table for the same host then ranger setup process will wait for maximum 10 minutes. After that ranger process will delete that entry and check the entry again. if entry does not exist then it will try to apply the patch again else it will wait for forever or unless you delete such entries manually. Please note that ranger will be checking the status entry every 2 minutes untill 10 minute wait time expires. **Note** 1) Patch has been provided for ranger supported all the db flavors. 2) Cureent patch also removes several lines of audit db related dead code which is not required now. Diffs (updated) - security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql a4fa1305e security-admin/db/mysql/patches/031-create-schema-for-usersync-audit-info.sql c17f19fe6 security-admin/scripts/db_setup.py 73b24aca0 security-admin/scripts/install.properties f52c6b298 Diff: https://reviews.apache.org/r/69347/diff/3/ Changes: https://reviews.apache.org/r/69347/diff/2-3/ Testing --- **Steps performed for Ranger-admin ** 1. To reproduce the case first installed ranger-0.7: 2. Now from the master branch. 3. Created Build with patch and untar the build. 4. Opened install.properties and provided db configuration in install.properties which was provided in ranger-0.7 branch build. 5. Called setup.sh 6. Ranger setup process shall try to apply the new db/java patches. 7. As soon as setup process started executing the patch J10014, terminated the script by using ctrl+c 8. logged into ranger db and ensured that J10014 patch entry has active='N' 9. Called the setup.sh again 10. setup process waited for the patch J10014 status for sometime and then it deleted the entry of J10014 patch from x_db_version_h table. 11. setup process continues and applied the patch J10014 successfully. 12. setup process completed successfully after applying all the patches. 13. started ranger-admin and was able to see the ranger-admin dashboard page. Thanks, Pradeep Agrawal
Re: Review Request 69677: RANGER-2287: Improve and optimize db_setup.py file code
> On Jan. 8, 2019, 11:48 a.m., Zsombor Gegesy wrote: > > It's great news, that you could delete thousands of lines of repetitive > > code, however you could achieve more, if instead of putting everything into > > one class, and put > > ''' > > if self.XA_DB_FLAVOR == DB_MYSQL: > >... > > elif self.XA_DB_FLAVOR == DB_POSTGRES: > >... > > ''' > > > > You can write > >self.do_something(...) > > > > and implement do_something differently in the MySQL/PostgreSQL/Oracle > > specific adapter class There shall be too many self.do_something(...) function I have to write which shall look like the previous code. Can you review it once again and let me know with few examples. - Pradeep --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69677/#review211760 ------- On Jan. 7, 2019, 6:37 a.m., Pradeep Agrawal wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69677/ > --- > > (Updated Jan. 7, 2019, 6:37 a.m.) > > > Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, > Nikhil P, Ramesh Mani, and Velmurugan Periasamy. > > > Bugs: RANGER-2287 > https://issues.apache.org/jira/browse/RANGER-2287 > > > Repository: ranger > > > Description > --- > > **Problem Statement:** There are lot of repeated code in db_setup.py which > can be removed which shall help developers to make any changes in db_setup.py > in future. > > **Proposed Solution:** Proposed patch shall remove the db setup methods of > each db flavor and shall use a single method for a specific work for each db > flavor. Based on the db flavor, config values shall be populated and handled > in the code after this patch. > > > Diffs > - > > security-admin/scripts/db_setup.py f1223b38c > > > Diff: https://reviews.apache.org/r/69677/diff/1/ > > > Testing > --- > > **Use Cases covered for all the db flavors:** > *1. Fresh installation(Ranger 2.0):* Tested patch with fresh installation of > ranger admin. > *2. Upgrade(from 0.7 to 2.0):* Installed Ranger from 0.7 branch and used same > db config on Ranger 2.0 installation config and run the setup.sh. Ranger was > upgraded successfully. > > > Thanks, > > Pradeep Agrawal > >
Review Request 69730: RANGER-2320: Make db schema patches script idempotent for all DB Flavors
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69730/ --- Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Bugs: RANGER-2320 https://issues.apache.org/jira/browse/RANGER-2320 Repository: ranger Description --- RANGER-2291 covers changes only in optimized DB schema script, existing DB patches script should be reviewed and if required these scripts also should be made idempotent. Diffs - security-admin/db/mysql/patches/009-updated_schema.sql 14ec1d872 security-admin/db/mysql/patches/013-permissionmodel.sql 381bb6f84 security-admin/db/mysql/patches/020-datamask-policy.sql 87c0a183a security-admin/db/mysql/patches/030-policy-labels-schema.sql 33af2a858 security-admin/db/mysql/patches/035-update-schema-for-x-policy.sql 84db52604 security-admin/db/postgres/patches/020-datamask-policy.sql 795721c99 security-admin/db/postgres/patches/035-update-schema-for-x-policy.sql 61c7986e4 security-admin/db/sqlserver/patches/020-datamask-policy.sql c50d2b83d security-admin/db/sqlserver/patches/035-update-schema-for-x-policy.sql bd646d695 Diff: https://reviews.apache.org/r/69730/diff/1/ Testing --- Tested the upgrade case from ranger-0.7 to ranger-2.0 for mysql, postgres and sqlserver db. Thanks, Pradeep Agrawal
Review Request 69677: RANGER-2287: Improve and optimize db_setup.py file code
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69677/ --- Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Bugs: RANGER-2287 https://issues.apache.org/jira/browse/RANGER-2287 Repository: ranger Description --- **Problem Statement:** There are lot of repeated code in db_setup.py which can be removed which shall help developers to make any changes in db_setup.py in future. **Proposed Solution:** Proposed patch shall remove the db setup methods of each db flavor and shall use a single method for a specific work for each db flavor. Based on the db flavor, config values shall be populated and handled in the code after this patch. Diffs - security-admin/scripts/db_setup.py f1223b38c Diff: https://reviews.apache.org/r/69677/diff/1/ Testing --- **Use Cases covered for all the db flavors:** *1. Fresh installation(Ranger 2.0):* Tested patch with fresh installation of ranger admin. *2. Upgrade(from 0.7 to 2.0):* Installed Ranger from 0.7 branch and used same db config on Ranger 2.0 installation config and run the setup.sh. Ranger was upgraded successfully. Thanks, Pradeep Agrawal
Re: Review Request 70058: RANGER-2339: UI changes for User role users should also have access to Security Zone
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70058/#review213382 --- Ship it! Ship It! - Pradeep Agrawal On Feb. 26, 2019, 11:39 a.m., Nitin Galave wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70058/ > --- > > (Updated Feb. 26, 2019, 11:39 a.m.) > > > Review request for ranger, Gautam Borad, Abhay Kulkarni, Mehul Parikh, > Pradeep Agrawal, and Velmurugan Periasamy. > > > Bugs: RANGER-2339 > https://issues.apache.org/jira/browse/RANGER-2339 > > > Repository: ranger > > > Description > --- > > User role user should be able to view the "Security Zone" tab and should get > access to security zones. > Expected Result: "Security Zone" tab should be visible to user role user > also. Operations on zones (create/update/delete) should be restricted to > admin users only (system-wide admin) > > > Diffs > - > > security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java db3d3d6 > security-admin/src/main/java/org/apache/ranger/rest/SecurityZoneREST.java > baded45 > security-admin/src/main/webapp/scripts/models/RangerZone.js 4b667d5 > security-admin/src/main/webapp/scripts/views/common/TopNav.js e08d4a2 > security-admin/src/main/webapp/scripts/views/policies/RangerPolicyForm.js > 50d2237 > > security-admin/src/main/webapp/scripts/views/policies/RangerPolicyTableLayout.js > 2f5b8bb > security-admin/src/main/webapp/scripts/views/policymanager/ServiceLayout.js > f9818d6 > security-admin/src/main/webapp/scripts/views/security_zone/SecurityZone.js > 9630db8 > security-admin/src/main/webapp/styles/xa.css 7fb868d > security-admin/src/main/webapp/templates/common/TopNav_tmpl.html b63ae90 > > security-admin/src/main/webapp/templates/security_zone/SecurityZone_tmpl.html > 75c0b90 > > > Diff: https://reviews.apache.org/r/70058/diff/1/ > > > Testing > --- > > Verified: > 1. All roles user is able to view Security Zone tab except 'Keyadmin' & > 'KMSAuditor' role users > 2. Only Admin role users is able to perform operations on zones. > > > Thanks, > > Nitin Galave > >
Re: Review Request 70059: RANGER-2334: Audits: filter out service audit logs and additional users logs from user audit logs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70059/#review213383 --- Ship it! Ship It! - Pradeep Agrawal On Feb. 26, 2019, 12:21 p.m., Nitin Galave wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70059/ > --- > > (Updated Feb. 26, 2019, 12:21 p.m.) > > > Review request for ranger, Abhay Kulkarni, Mehul Parikh, Nikhil P, Pradeep > Agrawal, and Velmurugan Periasamy. > > > Bugs: RANGER-2334 > https://issues.apache.org/jira/browse/RANGER-2334 > > > Repository: ranger > > > Description > --- > > Allow ranger users to filter out access audit logs for service users and > other specific users. > Provided checkbox on audit log page to filter service users/non-service users. > > > Diffs > - > > security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js 0b47ba7 > security-admin/src/main/webapp/templates/reports/AuditLayout_tmpl.html > 7d9a34a > > > Diff: https://reviews.apache.org/r/70059/diff/1/ > > > Testing > --- > > Verified: > 1.Able to filter out service/non-service users using service users checkbox > option. > > > Thanks, > > Nitin Galave > >
Re: Review Request 69655: RANGER-2317 : make Ranger buildable on newer JDKs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69655/#review213014 --- Tested Ranger installation in OpenJDK 10, 11 and Oracle JDK-11. Observations: Below mentioned java arguments needs to be removed from the ranger-admin-services.sh file for the successful start of ranger admin service as they are deprecated. UseGCLogFileRotation NumberOfGCLogFiles=10 GCLogFileSize=1m PrintHeapAtGC PrintGCDateStamps - Pradeep Agrawal On Jan. 7, 2019, 8:30 p.m., Zsombor Gegesy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69655/ > --- > > (Updated Jan. 7, 2019, 8:30 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-2317 > https://issues.apache.org/jira/browse/RANGER-2317 > > > Repository: ranger > > > Description > --- > > Upgrade librarias to become compatible with JDK11, and JAXB and other > librarias to the modules, and exclude jdk.tools. > (These modules was previously part of the JDK, now they are independent). > Add JDK10/11 to the travis build - with excluding the Hive tests, as we can't > even start the Hive tests on JDK 9 - because internal Hive code cast the > system classloader to URLClassLoader. > Removed one empty test class file, which triggered test failures with newer > mockito version. > > > Diffs > - > > .travis.yml b38b892bd > enunciate.xml f64af1a56 > hbase-agent/pom.xml c044ef0c3 > kms/pom.xml 043c67fde > knox-agent/pom.xml 0c45c84ba > plugin-solr/pom.xml 619cc9ebd > pom.xml 2a2540a0d > ranger-hbase-plugin-shim/pom.xml 9ed0aebe8 > ranger-solr-plugin-shim/pom.xml 46b78218a > security-admin/pom.xml 14cafb4b2 > > security-admin/src/test/java/org/apache/ranger/plugin/store/RangerDBStore.java > cd5bb384e > > ugsync/src/test/java/org/apache/ranger/unixusersync/process/TestUnixUserGroupBuilder.java > 2118c8430 > > > Diff: https://reviews.apache.org/r/69655/diff/2/ > > > Testing > --- > > Tested resulting binaries locally on JDK 8, to stay the same as previously. > All the dependency change was either in test code, or in Maven plugin config, > or has 'provided' scope. > > Tested on Travis that build is succesfull in 4 JVMs - > oraclejdk8/oraclejdk11/openjdk10/openjdk11: > https://travis-ci.org/gzsombor/ranger/builds/474559352 > > > Thanks, > > Zsombor Gegesy > >
Re: Review Request 69985: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration
> On Feb. 14, 2019, 12:02 p.m., Zsombor Gegesy wrote:
> > kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
> > Lines 50 (patched)
> > <https://reviews.apache.org/r/69985/diff/1/?file=2125358#file2125358line50>
> >
> > You can mark all variable as final
>
> Gautam Borad wrote:
> I am initializing non final variables in constructor.
>
> Zsombor Gegesy wrote:
> Yes, that's the way to use final variables: you need to initialize them
> in the constructor.
>
> Pradeep Agrawal wrote:
> @Zsombor Gegesy : To me its seems okay as he is reinitializing the mkSize
> variable at line 60. I don't think it need to be final, however we can make
> it static.
>
> Zsombor Gegesy wrote:
> Initializing these variables with their default values just adds noise to
> the code, the code behaves the same:
>
> class X {
> int x;
>
> void checkX() {
> if (x==0) {
> System.out.println("x is 0!");
> }
> }
> }
>
> new X().checkX() // this will print 'x is 0!'
>
> I dont think making an instance variable 'static' would be a good idea.
> In my opinion, you can safely remove the ' = null' and '= 0'
> initializations, and mark everything final, to make it clear, that these are
> constant values through the lifetime of this class.
>
> Pradeep Agrawal wrote:
> I agree that it don't need to be static and it don't need to be
> initialized with 0 at line 50 but it can't be final as per the code of line
> 60.
>
> Zsombor Gegesy wrote:
> mkSize is not written anywhere else, only be set on line 60, inside the
> constructor, so it can be set to final.
Yes, you are right.
- Pradeep
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69985/#review212827
---
On Feb. 19, 2019, 1:58 p.m., Gautam Borad wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69985/
> -----------
>
> (Updated Feb. 19, 2019, 1:58 p.m.)
>
>
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni,
> Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja
> Polavarapu, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-2331
> https://issues.apache.org/jira/browse/RANGER-2331
>
>
> Repository: ranger
>
>
> Description
> ---
>
> User story: As a security admin, I want to manage encryption keys for
> securing my Hadoop cluster files in Ranger KMS service with Safenet KeySecure
> crypto platform.
>
>
> For Safenet KeySecure overview refer to:
> https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/
>
>
> Acceptance Criteria:
>
>
> 1) Ranger KMS has ability to configure Safenet KeySecure platform to be used
> for key offload
>
>
> 2) Ranger KMS provides ability to provide key management functions (create
> keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform
>
>
> 3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure
> platform
>
>
> Diffs
> -
>
> kms/config/kms-webapp/dbks-site.xml 0e0f2ec
> kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION
> kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION
> kms/scripts/install.properties ddc779d
> kms/scripts/setup.sh 2db05b8
> kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java
> PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 22dce0f
>
> kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java
> PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 1abbf8e
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java
> 267fcf0
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 5614c16
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
> PRE-CREATION
> src/main/assembly/kms.xml fca6a32
>
>
> Diff: https://reviews.apache.org/r/69985/diff/2/
>
>
> Testing
> ---
>
> Verified below scenario:
>
>
> 1) Fresh Installation Of Ranger KMS with Safenet Key Secure (NAE-XML Protocol)
> 2) DB to Key Secure (NAE-XML) master key Migration utility
> 3) Key Secure (NAE-XML) to DB master key Migration utility
>
>
> Thanks,
>
> Gautam Borad
>
>
Re: Review Request 69985: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration
> On Feb. 14, 2019, 12:02 p.m., Zsombor Gegesy wrote:
> > kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
> > Lines 50 (patched)
> > <https://reviews.apache.org/r/69985/diff/1/?file=2125358#file2125358line50>
> >
> > You can mark all variable as final
>
> Gautam Borad wrote:
> I am initializing non final variables in constructor.
>
> Zsombor Gegesy wrote:
> Yes, that's the way to use final variables: you need to initialize them
> in the constructor.
>
> Pradeep Agrawal wrote:
> @Zsombor Gegesy : To me its seems okay as he is reinitializing the mkSize
> variable at line 60. I don't think it need to be final, however we can make
> it static.
>
> Zsombor Gegesy wrote:
> Initializing these variables with their default values just adds noise to
> the code, the code behaves the same:
>
> class X {
> int x;
>
> void checkX() {
> if (x==0) {
> System.out.println("x is 0!");
> }
> }
> }
>
> new X().checkX() // this will print 'x is 0!'
>
> I dont think making an instance variable 'static' would be a good idea.
> In my opinion, you can safely remove the ' = null' and '= 0'
> initializations, and mark everything final, to make it clear, that these are
> constant values through the lifetime of this class.
I agree that it don't need to be static and it don't need to be initialized
with 0 at line 50 but it can't be final as per the code of line 60.
- Pradeep
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69985/#review212827
---
On Feb. 19, 2019, 1:58 p.m., Gautam Borad wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69985/
> -------
>
> (Updated Feb. 19, 2019, 1:58 p.m.)
>
>
> Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni,
> Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja
> Polavarapu, and Velmurugan Periasamy.
>
>
> Bugs: RANGER-2331
> https://issues.apache.org/jira/browse/RANGER-2331
>
>
> Repository: ranger
>
>
> Description
> ---
>
> User story: As a security admin, I want to manage encryption keys for
> securing my Hadoop cluster files in Ranger KMS service with Safenet KeySecure
> crypto platform.
>
>
> For Safenet KeySecure overview refer to:
> https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/
>
>
> Acceptance Criteria:
>
>
> 1) Ranger KMS has ability to configure Safenet KeySecure platform to be used
> for key offload
>
>
> 2) Ranger KMS provides ability to provide key management functions (create
> keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform
>
>
> 3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure
> platform
>
>
> Diffs
> -
>
> kms/config/kms-webapp/dbks-site.xml 0e0f2ec
> kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION
> kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION
> kms/scripts/install.properties ddc779d
> kms/scripts/setup.sh 2db05b8
> kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java
> PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 22dce0f
>
> kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java
> PRE-CREATION
> kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 1abbf8e
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java
> 267fcf0
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 5614c16
> kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java
> PRE-CREATION
> src/main/assembly/kms.xml fca6a32
>
>
> Diff: https://reviews.apache.org/r/69985/diff/2/
>
>
> Testing
> ---
>
> Verified below scenario:
>
>
> 1) Fresh Installation Of Ranger KMS with Safenet Key Secure (NAE-XML Protocol)
> 2) DB to Key Secure (NAE-XML) master key Migration utility
> 3) Key Secure (NAE-XML) to DB master key Migration utility
>
>
> Thanks,
>
> Gautam Borad
>
>
Re: Review Request 70006: RANGER-2335: Overlapping of 'include' toggle button on policy create/edit page.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70006/#review213147 --- Ship it! Ship It! - Pradeep Agrawal On Feb. 19, 2019, 12:59 p.m., Nitin Galave wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70006/ > --- > > (Updated Feb. 19, 2019, 12:59 p.m.) > > > Review request for ranger, Gautam Borad, Mehul Parikh, Pradeep Agrawal, and > Velmurugan Periasamy. > > > Bugs: RANGER-2335 > https://issues.apache.org/jira/browse/RANGER-2335 > > > Repository: ranger > > > Description > --- > > Steps to reproduce: > 1) In HDFS service definition, for path resource field set exclude and > recursive support to true. > 2) Go to HDFS-Policy creation/edit page. > > > Diffs > - > > security-admin/src/main/webapp/scripts/modules/XAOverrides.js c45f74d > security-admin/src/main/webapp/scripts/views/policies/RangerPolicyRO.js > e494bcc > security-admin/src/main/webapp/styles/xa.css 7fb868d > security-admin/src/main/webapp/templates/policies/RangerPolicyRO_tmpl.html > 856e92a > > > Diff: https://reviews.apache.org/r/70006/diff/1/ > > > Testing > --- > > 1. Verified that overlap of "include" toggle button does not occur in case of > exclude & recursive support sets to true. > 2. Covered following test scenario's for resource `path`: >exclude:true, recursive:false : >exclude:false, recursive:true: >exclude:true, recursive:true : >exclude:false, recursive:false: > 3. Tested by adding new resources as well. > > > Thanks, > > Nitin Galave > >
Review Request 70126: RANGER-2342: Exclude jackson jaxrs library from ranger-admin packaging
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70126/ --- Review request for ranger, Don Bosco Durai, Colm O hEigeartaigh, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2342 https://issues.apache.org/jira/browse/RANGER-2342 Repository: ranger Description --- Unable to see user/group page listing as the jackson-jaxrs-json-provider-2.7.8.jar is being pulled by MapReduce client core jar. This issue is seen after changing the hadoop version to 3.0 (RANGER-1935 commit) Diffs - pom.xml 87bea2340 security-admin/pom.xml c9652ccea Diff: https://reviews.apache.org/r/70126/diff/1/ Testing --- After patch build the ranger admin and extracted the tar. extracted ranger admin package is not having jackson-jaxrs-json-provider-2.7.8.jar after ranger start able to see the users/groups list which was not coming earlier(without patch) Thanks, Pradeep Agrawal
Re: Review Request 70204: RANGER-2358: Upgrade Jackson Databind to 2.9.8
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70204/ --- (Updated March 14, 2019, 7:11 a.m.) Review request for ranger, Don Bosco Durai, Colm O hEigeartaigh, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy. Changes --- patch rebase Bugs: RANGER-2358 https://issues.apache.org/jira/browse/RANGER-2358 Repository: ranger Description --- It seems there are different versions of same jar is being pulled by different parent libraries during the ranger build. This issue shall address following libraries versions. 1) jackson-core 2) jackson-annotations 3) jackson-databind 4) commons.codec 5) commons.io.version 6) commons.net.version 7) netty-all.version 8) zookeeper.version Diffs (updated) - kms/pom.xml 8c5358486 pom.xml 553a2f6ff ranger-examples/src/main/assembly/plugin-sampleapp.xml 42d2e2d62 security-admin/pom.xml 1414745c5 src/main/assembly/kms.xml 8a7c6a7c4 src/main/assembly/tagsync.xml dd7580b01 Diff: https://reviews.apache.org/r/70204/diff/2/ Changes: https://reviews.apache.org/r/70204/diff/1-2/ Testing --- Tested Ranger installation and user/policy CRUD operations. Thanks, Pradeep Agrawal
Re: Review Request 69655: RANGER-2317 : make Ranger buildable on newer JDKs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69655/#review213616 --- Ship it! Ship It! - Pradeep Agrawal On March 11, 2019, 7:10 p.m., Zsombor Gegesy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69655/ > --- > > (Updated March 11, 2019, 7:10 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-2317 > https://issues.apache.org/jira/browse/RANGER-2317 > > > Repository: ranger > > > Description > --- > > Upgrade librarias to become compatible with JDK11, and JAXB and other > librarias to the modules, and exclude jdk.tools. > (These modules was previously part of the JDK, now they are independent). > Add JDK10/11 to the travis build - with excluding the Hive tests, as we can't > even start the Hive tests on JDK 9 - because internal Hive code cast the > system classloader to URLClassLoader. > Removed one empty test class file, which triggered test failures with newer > mockito version. > > > Diffs > - > > .travis.yml b38b892bd > embeddedwebserver/scripts/ranger-admin-services.sh c04e1fc72 > enunciate.xml f64af1a56 > hbase-agent/pom.xml 6b3763f22 > kms/pom.xml 59dd2761f > knox-agent/pom.xml bd68221cc > plugin-solr/pom.xml 619cc9ebd > pom.xml d5c4e924a > ranger-hbase-plugin-shim/pom.xml 9ed0aebe8 > ranger-solr-plugin-shim/pom.xml 46b78218a > security-admin/pom.xml 2c587605b > > security-admin/src/test/java/org/apache/ranger/plugin/store/RangerDBStore.java > cd5bb384e > > ugsync/src/test/java/org/apache/ranger/unixusersync/process/TestUnixUserGroupBuilder.java > 2118c8430 > > > Diff: https://reviews.apache.org/r/69655/diff/4/ > > > Testing > --- > > Tested resulting binaries locally on JDK 8, to stay the same as previously. > All the dependency change was either in test code, or in Maven plugin config, > or has 'provided' scope. > > Tested on Travis that build is succesfull in 4 JVMs - > oraclejdk8/oraclejdk11/openjdk10/openjdk11: > https://travis-ci.org/gzsombor/ranger/builds/474559352 > > > Thanks, > > Zsombor Gegesy > >
Review Request 70204: RANGER-2358: Upgrade Jackson Databind to 2.9.8
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70204/ --- Review request for ranger, Don Bosco Durai, Colm O hEigeartaigh, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2358 https://issues.apache.org/jira/browse/RANGER-2358 Repository: ranger Description --- It seems there are different versions of same jar is being pulled by different parent libraries during the ranger build. This issue shall address following libraries versions. 1) jackson-core 2) jackson-annotations 3) jackson-databind 4) commons.codec 5) commons.io.version 6) commons.net.version 7) netty-all.version 8) zookeeper.version Diffs - kms/pom.xml 59dd2761f pom.xml d5c4e924a ranger-examples/src/main/assembly/plugin-sampleapp.xml 42d2e2d62 security-admin/pom.xml 2c587605b src/main/assembly/kms.xml 8a7c6a7c4 src/main/assembly/tagsync.xml dd7580b01 Diff: https://reviews.apache.org/r/70204/diff/1/ Testing --- Tested Ranger installation and user/policy CRUD operations. Thanks, Pradeep Agrawal
Re: Review Request 70167: RANGER-2350:Ranger UI: Clicking on zone edit Breadcrumb redirect to 404 page not found
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70167/#review213591 --- Ship it! Ship It! - Pradeep Agrawal On March 8, 2019, 2:38 p.m., Nitin Galave wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70167/ > --- > > (Updated March 8, 2019, 2:38 p.m.) > > > Review request for ranger, Gautam Borad, Mehul Parikh, Pradeep Agrawal, and > Velmurugan Periasamy. > > > Bugs: RANGER-2350 > https://issues.apache.org/jira/browse/RANGER-2350 > > > Repository: ranger > > > Description > --- > > Steps to reproduce: > 1. Ranger UI -> goto -> Security Zone Menu. > 2. Click on edit zone button to open edit zone page > 3. Click on zone edit Breadcrumb redirect to 404 page not found > > > Diffs > - > > security-admin/src/main/java/org/apache/ranger/common/AppConstants.java > 34391a3 > security-admin/src/main/webapp/scripts/modules/XALinks.js a462033 > security-admin/src/main/webapp/scripts/utils/XAEnums.js 4865a4f > security-admin/src/main/webapp/scripts/utils/XAViewUtils.js a9f7df3 > security-admin/src/main/webapp/templates/helpers/XAHelpers.js 181ffc4 > > > Diff: https://reviews.apache.org/r/70167/diff/1/ > > > Testing > --- > > Verified zone edit breadcrumb link, it does not redirect to 404 page on click > on it. > It stays on current page after clicking on it as expected. > > > Thanks, > > Nitin Galave > >
Review Request 70181: RANGER-2252: Ranger installation is failing for Oracle and Postgres DB
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70181/ --- Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2327 https://issues.apache.org/jira/browse/RANGER-2327 Repository: ranger Description --- Update Security zone table schema should use common sequence name for Oracle and Postgres. It seems RANGER-2327 issue pop-up again during the code refactoring of RANGER-2341. Diffs - security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql e1dc7e000 security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql dae8086f5 Diff: https://reviews.apache.org/r/70181/diff/1/ Testing --- tested the patch with oracle and postgres db flavor. Thanks, Pradeep Agrawal
Review Request 70237: RANGER-2372 : Remove non-existing URL entries from spring config file
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70237/ --- Review request for ranger and Velmurugan Periasamy. Bugs: RANGER-2372 https://issues.apache.org/jira/browse/RANGER-2372 Repository: ranger Description --- Remove the outdated and invalid entries from the security-applicationContext.xml file. Restrict default access to anonymous users on ranger admin files and folders. Diffs - security-admin/src/main/resources/conf.dist/security-applicationContext.xml c008a1266 Diff: https://reviews.apache.org/r/70237/diff/1/ Testing --- Thanks, Pradeep Agrawal
Re: Review Request 69917: Audits: filter out service audit logs and additional users logs from user audit logs
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69917/#review212891 --- Ship it! Ship It! - Pradeep Agrawal On Feb. 11, 2019, 11:30 a.m., Nikhil P wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69917/ > --- > > (Updated Feb. 11, 2019, 11:30 a.m.) > > > Review request for ranger, Don Bosco Durai, Gautam Borad, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2334 > https://issues.apache.org/jira/browse/RANGER-2334 > > > Repository: ranger > > > Description > --- > > This feature of excluding service users from ranger access audits is provided > because there are too many system audits generated by service users. > > additional filter is added in ranger UI - access audits page i.e. Exclude > service user : true/false > > setting above filter as true & after setting below properties, ranger admin > will filter out service users from access audit logs. > > additionally,In order to remove additional user logs from access audits, one > can set another property for exclusion of audit logs for those users. > > 1)In order to use this feature, user should set component service users in > ranger-admin-site.xml whose logs are needed to be filtered out. > eg. say ranger user wants to filter out atlas and hbase service user. > > ranger.plugins.atlas.serviceuser > atlas > > > > ranger.plugins.hbase.serviceuser > hbase > > > 2)In addition to service users new property > "ranger.accesslogs.exclude.users.list" is introduced, in which user can > specify list of additional users(other than service users)whose logs needs be > excluded from ranger access audits. > > ranger.accesslogs.exclude.users.list > yarn-ats,testUser > > > after setting above properties, logs from above users will not show up in > ranger access audits. > > > Diffs > - > > > agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java > 110f763 > security-admin/src/main/java/org/apache/ranger/rest/AssetREST.java 8a0ca95 > > security-admin/src/main/java/org/apache/ranger/solr/SolrAccessAuditsService.java > f64c0db > security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js 4894480 > security-admin/src/test/java/org/apache/ranger/rest/TestAssetREST.java > 40e680a > > > Diff: https://reviews.apache.org/r/69917/diff/1/ > > > Testing > --- > > 1.Tested that after setting properties & sending UI flag(Exclude service > user) as true ,ranger access audit logs is not showing audits for > serviceusers(serviceusers are set in property > ranger.plugins..serviceuser). > 2.checked if additional user logs are also getting filtered out after setting > those users list in property(ranger.accesslogs.exclude.users.list) and > sending UI flag(Exclude service user) as true on access audits page. > > > Thanks, > > Nikhil P > >
Re: Review Request 69930: RANGER-2333: Logs does not get generated for Zone Description field available on Security Zone page.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69930/ --- (Updated Feb. 13, 2019, 3:01 a.m.) Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Changes --- little optimization in validation function Bugs: RANGER-2333 https://issues.apache.org/jira/browse/RANGER-2333 Repository: ranger Description --- **Problem Stamtents:** 1) Zone Description field does not exist in x_security_zone table hence zone desciption provided in UI does not get saved. 2) Zone name with blank space can be created which should be restricted. 3) adminUsers or adminUsersGroups and auditUsers or auditUsersGroups and Resource field should be restricted for blank values. 4) All users should have access to Security zone module. 5) If users are not allowed to create service zone then the response code should be 403 rather 400. **Proposed Solutions:** 1) Added a column in x_security_zone table added code to add/update description field values. 2) Trimmed the zone name value before the existing validation check. 3) Removed empty list items from the various list before the validation. 4) Added access on the security zone module to all users. 5) Changed response code and message. Diffs (updated) - agents-common/src/main/java/org/apache/ranger/plugin/model/RangerSecurityZone.java 9eaf10214 agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerSecurityZoneValidator.java 40137072a agents-common/src/test/java/org/apache/ranger/plugin/model/validation/RangerSecurityZoneValidatorTest.java b16ccd6a4 security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 71cfa8f8c security-admin/db/mysql/patches/037-create-security-zone-schema.sql 0df549137 security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 6b569f254 security-admin/db/oracle/patches/037-create-security-zone-schema.sql e71f3db38 security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql a4e93cada security-admin/db/postgres/patches/037-create-security-zone-schema.sql e81da8078 security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 71f0ba410 security-admin/db/sqlanywhere/patches/037-create-security-zone-schema.sql b96b6e5c5 security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql d69c2dd37 security-admin/db/sqlserver/patches/037-create-security-zone-schema.sql 1c687220b security-admin/src/main/java/org/apache/ranger/biz/RangerBizUtil.java d350fd1c0 security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 933e99fff security-admin/src/main/java/org/apache/ranger/entity/XXSecurityZoneBase.java 9c7871041 security-admin/src/main/java/org/apache/ranger/patch/PatchAssignSecurityZonePersmissionToAdmin_J10026.java 64e39e391 security-admin/src/main/java/org/apache/ranger/rest/SecurityZoneREST.java 1145122cf security-admin/src/main/java/org/apache/ranger/service/RangerSecurityZoneService.java cc796d5aa security-admin/src/main/java/org/apache/ranger/service/RangerSecurityZoneServiceBase.java 0620441b9 security-admin/src/test/java/org/apache/ranger/rest/TestSecurityZoneREST.java 456b858a3 Diff: https://reviews.apache.org/r/69930/diff/2/ Changes: https://reviews.apache.org/r/69930/diff/1-2/ Testing --- Tested above mentioned usecases on my local environments. Thanks, Pradeep Agrawal
Re: Review Request 69930: RANGER-2333: Logs does not get generated for Zone Description field available on Security Zone page.
> On Feb. 13, 2019, 10:56 p.m., Abhay Kulkarni wrote: > > security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql > > Line 2690 (original), 2691 (patched) > > <https://reviews.apache.org/r/69930/diff/2/?file=2125037#file2125037line2691> > > > > Was this a bug in the earlier version of this SQL script? Yes - Pradeep --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69930/#review212810 --- On Feb. 14, 2019, 9:11 a.m., Pradeep Agrawal wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69930/ > --- > > (Updated Feb. 14, 2019, 9:11 a.m.) > > > Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, > Nikhil P, Nitin Galave, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and > Velmurugan Periasamy. > > > Bugs: RANGER-2333 > https://issues.apache.org/jira/browse/RANGER-2333 > > > Repository: ranger > > > Description > --- > > **Problem Stamtents:** > > 1) Zone Description field does not exist in x_security_zone table hence zone > desciption provided in UI does not get saved. > 2) Zone name with blank space can be created which should be restricted. > 3) adminUsers or adminUsersGroups and auditUsers or auditUsersGroups and > Resource field should be restricted for blank values. > 4) All users should have access to Security zone module. > 5) If users are not allowed to create service zone then the response code > should be 403 rather 400. > > **Proposed Solutions:** > > 1) Added a column in x_security_zone table added code to add/update > description field values. > 2) Trimmed the zone name value before the existing validation check. > 3) Removed empty list items from the various list before the validation. > 4) Added access on the security zone module to all users. > 5) Changed response code and message. > > > Diffs > - > > > agents-common/src/main/java/org/apache/ranger/plugin/model/RangerSecurityZone.java > 9eaf10214 > > agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerSecurityZoneValidator.java > 40137072a > > agents-common/src/test/java/org/apache/ranger/plugin/model/validation/RangerSecurityZoneValidatorTest.java > b16ccd6a4 > security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql > 71cfa8f8c > security-admin/db/mysql/patches/037-create-security-zone-schema.sql > 0df549137 > security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql > 6b569f254 > security-admin/db/oracle/patches/037-create-security-zone-schema.sql > e71f3db38 > security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql > a4e93cada > security-admin/db/postgres/patches/037-create-security-zone-schema.sql > e81da8078 > > security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql > 71f0ba410 > security-admin/db/sqlanywhere/patches/037-create-security-zone-schema.sql > b96b6e5c5 > security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql > d69c2dd37 > security-admin/db/sqlserver/patches/037-create-security-zone-schema.sql > 1c687220b > security-admin/src/main/java/org/apache/ranger/biz/RangerBizUtil.java > d350fd1c0 > security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 933e99fff > > security-admin/src/main/java/org/apache/ranger/entity/XXSecurityZoneBase.java > 9c7871041 > > security-admin/src/main/java/org/apache/ranger/patch/PatchAssignSecurityZonePersmissionToAdmin_J10026.java > 64e39e391 > security-admin/src/main/java/org/apache/ranger/rest/SecurityZoneREST.java > 1145122cf > > security-admin/src/main/java/org/apache/ranger/service/RangerSecurityZoneService.java > cc796d5aa > > security-admin/src/main/java/org/apache/ranger/service/RangerSecurityZoneServiceBase.java > 0620441b9 > > security-admin/src/test/java/org/apache/ranger/rest/TestSecurityZoneREST.java > 456b858a3 > > > Diff: https://reviews.apache.org/r/69930/diff/3/ > > > Testing > --- > > Tested above mentioned usecases on my local environments. > > > Thanks, > > Pradeep Agrawal > >
Re: Review Request 69930: RANGER-2333: Logs does not get generated for Zone Description field available on Security Zone page.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69930/ --- (Updated Feb. 14, 2019, 9:11 a.m.) Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Changes --- addressed review comments Bugs: RANGER-2333 https://issues.apache.org/jira/browse/RANGER-2333 Repository: ranger Description --- **Problem Stamtents:** 1) Zone Description field does not exist in x_security_zone table hence zone desciption provided in UI does not get saved. 2) Zone name with blank space can be created which should be restricted. 3) adminUsers or adminUsersGroups and auditUsers or auditUsersGroups and Resource field should be restricted for blank values. 4) All users should have access to Security zone module. 5) If users are not allowed to create service zone then the response code should be 403 rather 400. **Proposed Solutions:** 1) Added a column in x_security_zone table added code to add/update description field values. 2) Trimmed the zone name value before the existing validation check. 3) Removed empty list items from the various list before the validation. 4) Added access on the security zone module to all users. 5) Changed response code and message. Diffs (updated) - agents-common/src/main/java/org/apache/ranger/plugin/model/RangerSecurityZone.java 9eaf10214 agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerSecurityZoneValidator.java 40137072a agents-common/src/test/java/org/apache/ranger/plugin/model/validation/RangerSecurityZoneValidatorTest.java b16ccd6a4 security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 71cfa8f8c security-admin/db/mysql/patches/037-create-security-zone-schema.sql 0df549137 security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 6b569f254 security-admin/db/oracle/patches/037-create-security-zone-schema.sql e71f3db38 security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql a4e93cada security-admin/db/postgres/patches/037-create-security-zone-schema.sql e81da8078 security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 71f0ba410 security-admin/db/sqlanywhere/patches/037-create-security-zone-schema.sql b96b6e5c5 security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql d69c2dd37 security-admin/db/sqlserver/patches/037-create-security-zone-schema.sql 1c687220b security-admin/src/main/java/org/apache/ranger/biz/RangerBizUtil.java d350fd1c0 security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 933e99fff security-admin/src/main/java/org/apache/ranger/entity/XXSecurityZoneBase.java 9c7871041 security-admin/src/main/java/org/apache/ranger/patch/PatchAssignSecurityZonePersmissionToAdmin_J10026.java 64e39e391 security-admin/src/main/java/org/apache/ranger/rest/SecurityZoneREST.java 1145122cf security-admin/src/main/java/org/apache/ranger/service/RangerSecurityZoneService.java cc796d5aa security-admin/src/main/java/org/apache/ranger/service/RangerSecurityZoneServiceBase.java 0620441b9 security-admin/src/test/java/org/apache/ranger/rest/TestSecurityZoneREST.java 456b858a3 Diff: https://reviews.apache.org/r/69930/diff/3/ Changes: https://reviews.apache.org/r/69930/diff/2-3/ Testing --- Tested above mentioned usecases on my local environments. Thanks, Pradeep Agrawal
Review Request 69930: RANGER-2333: Logs does not get generated for Zone Description field available on Security Zone page.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69930/ --- Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2333 https://issues.apache.org/jira/browse/RANGER-2333 Repository: ranger Description --- **Problem Stamtents:** 1) Zone Description field does not exist in x_security_zone table hence zone desciption provided in UI does not get saved. 2) Zone name with blank space can be created which should be restricted. 3) adminUsers or adminUsersGroups and auditUsers or auditUsersGroups and Resource field should be restricted for blank values. 4) All users should have access to Security zone module. 5) If users are not allowed to create service zone then the response code should be 403 rather 400. **Proposed Solutions:** 1) Added a column in x_security_zone table added code to add/update description field values. 2) Trimmed the zone name value before the existing validation check. 3) Removed empty list items from the various list before the validation. 4) Added access on the security zone module to all users. 5) Changed response code and message. Diffs - agents-common/src/main/java/org/apache/ranger/plugin/model/RangerSecurityZone.java 9eaf10214 agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerSecurityZoneValidator.java 40137072a agents-common/src/test/java/org/apache/ranger/plugin/model/validation/RangerSecurityZoneValidatorTest.java b16ccd6a4 security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 71cfa8f8c security-admin/db/mysql/patches/037-create-security-zone-schema.sql 0df549137 security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 6b569f254 security-admin/db/oracle/patches/037-create-security-zone-schema.sql e71f3db38 security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql a4e93cada security-admin/db/postgres/patches/037-create-security-zone-schema.sql e81da8078 security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql 71f0ba410 security-admin/db/sqlanywhere/patches/037-create-security-zone-schema.sql b96b6e5c5 security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql d69c2dd37 security-admin/db/sqlserver/patches/037-create-security-zone-schema.sql 1c687220b security-admin/src/main/java/org/apache/ranger/biz/RangerBizUtil.java d350fd1c0 security-admin/src/main/java/org/apache/ranger/biz/XUserMgr.java 933e99fff security-admin/src/main/java/org/apache/ranger/entity/XXSecurityZoneBase.java 9c7871041 security-admin/src/main/java/org/apache/ranger/patch/PatchAssignSecurityZonePersmissionToAdmin_J10026.java 64e39e391 security-admin/src/main/java/org/apache/ranger/rest/SecurityZoneREST.java 1145122cf security-admin/src/main/java/org/apache/ranger/service/RangerSecurityZoneService.java cc796d5aa security-admin/src/main/java/org/apache/ranger/service/RangerSecurityZoneServiceBase.java 0620441b9 security-admin/src/test/java/org/apache/ranger/rest/TestSecurityZoneREST.java 456b858a3 Diff: https://reviews.apache.org/r/69930/diff/1/ Testing --- Tested above mentioned usecases on my local environments. Thanks, Pradeep Agrawal
Re: Review Request 69985: RANGER-2331 : Ranger-KMS - KeySecure HSM Integration
> On Feb. 14, 2019, 12:02 p.m., Zsombor Gegesy wrote: > > kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java > > Lines 50 (patched) > > <https://reviews.apache.org/r/69985/diff/1/?file=2125358#file2125358line50> > > > > You can mark all variable as final > > Gautam Borad wrote: > I am initializing non final variables in constructor. > > Zsombor Gegesy wrote: > Yes, that's the way to use final variables: you need to initialize them > in the constructor. @Zsombor Gegesy : To me its seems okay as he is reinitializing the mkSize variable at line 60. I don't think it need to be final, however we can make it static. - Pradeep --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69985/#review212827 --- On Feb. 19, 2019, 1:58 p.m., Gautam Borad wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/69985/ > --- > > (Updated Feb. 19, 2019, 1:58 p.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Abhay Kulkarni, > Madhan Neethiraj, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2331 > https://issues.apache.org/jira/browse/RANGER-2331 > > > Repository: ranger > > > Description > --- > > User story: As a security admin, I want to manage encryption keys for > securing my Hadoop cluster files in Ranger KMS service with Safenet KeySecure > crypto platform. > > > For Safenet KeySecure overview refer to: > https://safenet.gemalto.com/data-encryption/enterprise-key-management/key-secure/ > > > Acceptance Criteria: > > > 1) Ranger KMS has ability to configure Safenet KeySecure platform to be used > for key offload > > > 2) Ranger KMS provides ability to provide key management functions (create > keys, manage keys, retrieve keys, rollover) using Safenet KeySecure platform > > > 3) Ranger KMS UI panel on Ambari can be used to configure Safenet KeySecure > platform > > > Diffs > - > > kms/config/kms-webapp/dbks-site.xml 0e0f2ec > kms/scripts/DBMKTOKEYSECURE.sh PRE-CREATION > kms/scripts/KEYSECUREMKTOKMSDB.sh PRE-CREATION > kms/scripts/install.properties ddc779d > kms/scripts/setup.sh 2db05b8 > kms/src/main/java/org/apache/hadoop/crypto/key/DBToKeySecure.java > PRE-CREATION > kms/src/main/java/org/apache/hadoop/crypto/key/JKS2RangerUtil.java 22dce0f > > kms/src/main/java/org/apache/hadoop/crypto/key/KeySecureToRangerDBMKUtil.java > PRE-CREATION > kms/src/main/java/org/apache/hadoop/crypto/key/Ranger2JKSUtil.java 1abbf8e > kms/src/main/java/org/apache/hadoop/crypto/key/RangerKeyStoreProvider.java > 267fcf0 > kms/src/main/java/org/apache/hadoop/crypto/key/RangerMasterKey.java 5614c16 > kms/src/main/java/org/apache/hadoop/crypto/key/RangerSafenetKeySecure.java > PRE-CREATION > src/main/assembly/kms.xml fca6a32 > > > Diff: https://reviews.apache.org/r/69985/diff/2/ > > > Testing > --- > > Verified below scenario: > > > 1) Fresh Installation Of Ranger KMS with Safenet Key Secure (NAE-XML Protocol) > 2) DB to Key Secure (NAE-XML) master key Migration utility > 3) Key Secure (NAE-XML) to DB master key Migration utility > > > Thanks, > > Gautam Borad > >
Re: Review Request 69740: RANGER-2322 - Use "TLS" in SSLContext.getInstance
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69740/#review212436
---
Can you please add what all testing has been done with this patch. Thanks.
- Pradeep Agrawal
On Jan. 14, 2019, 11:20 a.m., Colm O hEigeartaigh wrote:
>
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69740/
> ---
>
> (Updated Jan. 14, 2019, 11:20 a.m.)
>
>
> Review request for ranger.
>
>
> Bugs: RANGER-2322
> https://issues.apache.org/jira/browse/RANGER-2322
>
>
> Repository: ranger
>
>
> Description
> ---
>
> There are a few instances in the code of SSLContext.getInstance("SSL").
> Instead we should use "TLS".
>
>
> Diffs
> -
>
>
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java
> f592ed4e7
>
> agents-common/src/main/java/org/apache/ranger/plugin/util/RangerSslHelper.java
> 2036661e8
>
> ugsync/src/main/java/org/apache/ranger/ldapusersync/process/LdapPolicyMgrUserGroupBuilder.java
> f653b81f3
>
> ugsync/src/main/java/org/apache/ranger/unixusersync/process/PolicyMgrUserGroupBuilder.java
> cb9b51c95
>
>
> Diff: https://reviews.apache.org/r/69740/diff/1/
>
>
> Testing
> ---
>
>
> Thanks,
>
> Colm O hEigeartaigh
>
>
Review Request 69853: RANGER-2327: Update Ranger db schema to use common sequence name
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/69853/ --- Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Pradeep Agrawal, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2327 https://issues.apache.org/jira/browse/RANGER-2327 Repository: ranger Description --- Update Security zone table schema should use common sequence name for Oracle and Postgres. It seems during the code refactoring in the oracle schema sequnce name was modified(possibly due to identifier length is restricted to 30 characters in oracle) but not at the postgres schema. we need to mention the sequence names in the entity classes to avoid sequence name mismatch issue. Diffs - security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql f90248a28 security-admin/db/oracle/patches/037-create-security-zone-schema.sql 357e73fb2 security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 79357ed5d security-admin/db/postgres/patches/037-create-security-zone-schema.sql 6baa7b975 security-admin/scripts/db_setup.py cab9e59a1 security-admin/src/main/java/org/apache/ranger/entity/XXSecurityZoneRefGroup.java 91d90703b security-admin/src/main/java/org/apache/ranger/entity/XXSecurityZoneRefResource.java 3d68ba317 security-admin/src/main/java/org/apache/ranger/entity/XXSecurityZoneRefService.java 56ec01862 security-admin/src/main/java/org/apache/ranger/entity/XXSecurityZoneRefUser.java 3e5736621 Diff: https://reviews.apache.org/r/69853/diff/1/ Testing --- tested the patch with oracle and postgres db flavor. Thanks, Pradeep Agrawal
Re: Review Request 70424: RANGER-2399:User's listing page hits users API call twice from UI
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70424/#review214520 --- Ship it! Ship It! - Pradeep Agrawal On April 8, 2019, 10:58 a.m., Nitin Galave wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70424/ > --- > > (Updated April 8, 2019, 10:58 a.m.) > > > Review request for ranger, Mehul Parikh, Nikhil P, Pradeep Agrawal, and > Velmurugan Periasamy. > > > Bugs: RANGER-2399 > https://issues.apache.org/jira/browse/RANGER-2399 > > > Repository: ranger > > > Description > --- > > #issue 1 > Steps: Login and go to users/group tab > 1. Click on username name link and open user update page > 2. And go back to user's listing page. > See user API call hit twice. The unnecessary call needs to be removed from UI > > #issue 2 > 2. Zonename title placement on zone listing page to be checked when zone name > is long > > > Diffs > - > > security-admin/src/main/webapp/scripts/controllers/Controller.js 934a1a2 > security-admin/src/main/webapp/scripts/views/security_zone/SecurityZone.js > a1e9716 > security-admin/src/main/webapp/scripts/views/users/GroupCreate.js a764387 > security-admin/src/main/webapp/scripts/views/users/UserCreate.js e1afff1 > security-admin/src/main/webapp/scripts/views/users/UserTableLayout.js > 94afd75 > security-admin/src/main/webapp/styles/xa.css f90f21b > security-admin/src/main/webapp/templates/common/breadcrumbs.html 8de0ad3 > > security-admin/src/main/webapp/templates/security_zone/SecurityZone_tmpl.html > f0b2572 > > > Diff: https://reviews.apache.org/r/70424/diff/1/ > > > Testing > --- > > 1. Verified that multiple API Call does not get generated for single request. > e.g. On creation/edit/cancel of User/Group. > 2. Zone name title displays properly in case of having long zone name, > Verified on Zone listing page, Zone name header, Zone breadcrumb. > > > Thanks, > > Nitin Galave > >
Re: Review Request 70435: RANGER-2401: Ranger Secuity Zone needs to be added in audit type filter in admin audit
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70435/#review214519 --- Ship it! Ship It! - Pradeep Agrawal On April 9, 2019, 12:53 p.m., Nitin Galave wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70435/ > --- > > (Updated April 9, 2019, 12:53 p.m.) > > > Review request for ranger, deepak sharma, Mehul Parikh, Pradeep Agrawal, and > Velmurugan Periasamy. > > > Bugs: RANGER-2401 > https://issues.apache.org/jira/browse/RANGER-2401 > > > Repository: ranger > > > Description > --- > > 1. Ranger Security Zone is missing in audit type filter in the admin audit > log. > > 2. Also, Include "User Profile" in the audit type filter for admin audits > When a user is deleted, admin audit entries are generated for user deletion > and user profile deletion with audit types as "Ranger User" and "User > Profile". > "Ranger User" option is available in the audit type search filter, "User > Profile" is not. Suggestion is to include "User Profile" option also in audit > type filter > > > Diffs > - > > security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js e7b45d9 > > > Diff: https://reviews.apache.org/r/70435/diff/1/ > > > Testing > --- > > 1. Verified that System display "Ranger Security Zone" filter in Audit>>Admin > tab with Audit Type filter option. > 2. Verified that,System display "User Profile" filter in Audit>>Admin tab > with Audit Type filter option. > > > Thanks, > > Nitin Galave > >
Re: Review Request 70339: RANGER-2386: Code duplication due to RangerCredentialProvider.getCredentialString returns char[]
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70339/#review214521 --- Ship it! Ship It! - Pradeep Agrawal On March 29, 2019, 12:18 p.m., Zsombor Gegesy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70339/ > --- > > (Updated March 29, 2019, 12:18 p.m.) > > > Review request for ranger. > > > Bugs: RANGER-2386 > https://issues.apache.org/jira/browse/RANGER-2386 > > > Repository: ranger > > > Description > --- > > The same code appears in lot's of places, because > RangerCredentialProvider.getCredentialString returns a char array, which > needs to be converted to String - remove this duplication. > > > Diffs > - > > > agents-audit/src/main/java/org/apache/ranger/audit/provider/DbAuditProvider.java > 1490c2782a9d603b8ad1392a1aafef7655e56562 > agents-audit/src/main/java/org/apache/ranger/audit/provider/MiscUtil.java > eff38249e044a910b747aa7e81003028d2b3 > > agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTClient.java > 2a4b9c90186effdcc2d9bc5b8726a5351ea668bd > > agents-common/src/main/java/org/apache/ranger/plugin/util/RangerSslHelper.java > 2036661e81201b15fc296d53f2a831a8a1770b36 > > agents-cred/src/main/java/org/apache/ranger/authorization/hadoop/utils/RangerCredentialProvider.java > 44cadfba2dc2f209a94b11f6d3ad7e7582f3d8cb > > agents-cred/src/test/java/org/apache/ranger/authorization/hadoop/utils/RangerCredentialProviderTest.java > 971ee681a33dfcf5993e4d79e25c648c90646df2 > > > Diff: https://reviews.apache.org/r/70339/diff/1/ > > > Testing > --- > > Tested with plugins on a cluster > > > Thanks, > > Zsombor Gegesy > >
Re: Review Request 70376: RANGER-2392: Create / Update zone to have provision to associate Tag based service with zone
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70376/#review214527 --- security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql Lines 2272 (patched) <https://reviews.apache.org/r/70376/#comment300718> change this to tag_srvc_id security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql Lines 2273 (patched) <https://reviews.apache.org/r/70376/#comment300719> change this to tag_srvc_name - Pradeep Agrawal On April 9, 2019, 12:38 p.m., Nitin Galave wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70376/ > --- > > (Updated April 9, 2019, 12:38 p.m.) > > > Review request for ranger, Abhay Kulkarni, Mehul Parikh, Nikhil P, Pradeep > Agrawal, and Velmurugan Periasamy. > > > Bugs: RANGER-2392 > https://issues.apache.org/jira/browse/RANGER-2392 > > > Repository: ranger > > > Description > --- > > Currently, Resource-based services are associated with the zone as part of > zone creation. > Need to add a feature to only associate Tag based service as part of zone. > No need to add resources with tag-based service association. > > > Diffs > - > > > agents-common/src/main/java/org/apache/ranger/plugin/model/RangerSecurityZone.java > 547e2d2 > security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 2eaffcc > security-admin/db/mysql/patches/037-create-security-zone-schema.sql aff9786 > security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql > 5ca1e36 > security-admin/db/oracle/patches/037-create-security-zone-schema.sql > aae31dc > security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql > 51f4999 > security-admin/db/postgres/patches/037-create-security-zone-schema.sql > 4a94d26 > > security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql > 5f64a2d > security-admin/db/sqlanywhere/patches/037-create-security-zone-schema.sql > 5b7dade > security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql > fc21cec > security-admin/db/sqlserver/patches/037-create-security-zone-schema.sql > e621b43 > > security-admin/src/main/java/org/apache/ranger/biz/SecurityZoneRefUpdater.java > 2b0c462 > security-admin/src/main/java/org/apache/ranger/common/RangerConstants.java > cdb5e11 > security-admin/src/main/java/org/apache/ranger/db/RangerDaoManagerBase.java > 3599cf3 > > security-admin/src/main/java/org/apache/ranger/db/XXSecurityZoneRefTagServiceDao.java > PRE-CREATION > > security-admin/src/main/java/org/apache/ranger/entity/XXSecurityZoneRefTagService.java > PRE-CREATION > security-admin/src/main/java/org/apache/ranger/rest/SecurityZoneREST.java > b8b8c1b > > security-admin/src/main/java/org/apache/ranger/service/RangerSecurityZoneService.java > 04003f4 > security-admin/src/main/resources/META-INF/jpa_named_queries.xml 7e702c1 > security-admin/src/main/webapp/scripts/models/RangerZone.js 4f99c83 > security-admin/src/main/webapp/scripts/utils/XAUtils.js b14f4b9 > security-admin/src/main/webapp/scripts/views/policymanager/ServiceLayout.js > d3eb026 > security-admin/src/main/webapp/scripts/views/security_zone/SecurityZone.js > a1e9716 > > security-admin/src/main/webapp/scripts/views/security_zone/ZoneAdministration.js > 63fafb4 > security-admin/src/main/webapp/scripts/views/security_zone/ZoneCreate.js > dd34605 > security-admin/src/main/webapp/styles/xa.css f90f21b > > security-admin/src/main/webapp/templates/security_zone/SecurityZone_tmpl.html > f0b2572 > > security-admin/src/main/webapp/templates/security_zone/ZoneAdministration_tmpl.html > 89acb10 > > security-admin/src/main/webapp/templates/security_zone/ZoneCreateForm_tmpl.html > fdcb821 > > > Diff: https://reviews.apache.org/r/70376/diff/2/ > > > Testing > --- > > 1. Verified that new field "Zone Tag Services" is available on Zone form > (create/update page). > 2. Verified that multiple Tag services can be mapped to zone using newly > added field i.e "Zone Tag Services". > 3. Verified that Same Tag services can be mapped to multiple zones. > 4. Zone detail page display tag services associated with that zone. > 5. Verified that When tag based services is not mapped to any zone, It > displays message "No tag based services are associated with this zone" on > Zone listing page. > 6. Verified that Audit>>Admin logs gets generated for zone tag services field > in zone form > > > Thanks, > > Nitin Galave > >
Re: Review Request 70340: RANGER-2382 : Improvement to Access Audit page-Add ‘agentHostname’ column to audit log table, which records IP-address/hostname of the plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70340/#review214693 --- Ship it! Ship It! - Pradeep Agrawal On April 12, 2019, 2:11 p.m., Nikhil P wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70340/ > --- > > (Updated April 12, 2019, 2:11 p.m.) > > > Review request for ranger, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, > Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, and > Velmurugan Periasamy. > > > Bugs: RANGER-2382 > https://issues.apache.org/jira/browse/RANGER-2382 > > > Repository: ranger > > > Description > --- > > In Audit Page > > Access tab: > Add ‘agentHostname’ column to audit log table, which records > IP-address/hostname of the plugin. Having this field in audit UI can avoid > having to scan all the hosts. > > > Diffs > - > > LICENSE.txt 7aa4bf4 > > agents-audit/src/main/java/org/apache/ranger/audit/destination/SolrAuditDestination.java > 80a6d9e > > agents-audit/src/main/java/org/apache/ranger/audit/provider/solr/SolrAuditProvider.java > ee95800 > agents-common/pom.xml 4486af3 > > agents-common/src/main/java/org/apache/ranger/plugin/audit/RangerDefaultAuditHandler.java > 4421442 > > agents-common/src/main/java/org/apache/ranger/plugin/util/RangerRESTUtils.java > 65f5c01 > pom.xml 2a45f8b > security-admin/src/main/java/org/apache/ranger/rest/AssetREST.java ce577e0 > > security-admin/src/main/java/org/apache/ranger/service/XAccessAuditService.java > 6687b19 > > security-admin/src/main/java/org/apache/ranger/solr/SolrAccessAuditsService.java > cf2a3b4 > security-admin/src/main/java/org/apache/ranger/view/VXAccessAudit.java > 7b63a63 > security-admin/src/main/webapp/scripts/modules/globalize/message/en.js > d4d87d7 > security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js daf6924 > security-admin/src/test/java/org/apache/ranger/rest/TestAssetREST.java > ef149d5 > src/main/assembly/admin-web.xml 06ba8c4 > src/main/assembly/hbase-agent.xml fa79283 > src/main/assembly/hdfs-agent.xml add00e6 > src/main/assembly/hive-agent.xml 4f80a95 > src/main/assembly/kms.xml 17d6786 > src/main/assembly/knox-agent.xml 9e30c4f > src/main/assembly/plugin-atlas.xml 73b624b > src/main/assembly/plugin-elasticsearch.xml 41f73a3 > src/main/assembly/plugin-kafka.xml ca50e6b > src/main/assembly/plugin-kms.xml f3473b3 > src/main/assembly/plugin-kylin.xml 87a8154 > src/main/assembly/plugin-solr.xml 16a0696 > src/main/assembly/plugin-sqoop.xml 7ce714b > src/main/assembly/plugin-yarn.xml 04d252e > src/main/assembly/ranger-tools.xml 03d58bf > src/main/assembly/storm-agent.xml 0e1e946 > src/main/assembly/tagsync.xml 985b99a > src/main/assembly/usersync.xml 9e6a882 > > > Diff: https://reviews.apache.org/r/70340/diff/4/ > > > Testing > --- > > 1)verified if ‘agentHostname’ is getting added in solr and also getting > displayed on Ranger UI -access audit page. > 2)verified agentHostName shows destination of agents. > > > Thanks, > > Nikhil P > >
Review Request 70500: RANGER-2402: Best Practices: Make db schema script idempotent
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70500/ --- Review request for ranger, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Ramesh Mani, and Velmurugan Periasamy. Bugs: RANGER-2402 https://issues.apache.org/jira/browse/RANGER-2402 Repository: ranger Description --- Problem Statement: Currently Ranger core db schema is not idempotent for all db flavors. Ranger core DB schema for Oracle and SQL anywhere flavor may fail to execute if we execute them again for the same DB flavor. Proposed Solution: I have added if statements before the insert statements. Diffs - security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql 2cdd8fbb6 security-admin/db/mysql/patches/037-create-security-zone-schema.sql b2b69f0dc security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql 88fa98a99 security-admin/db/oracle/patches/037-create-security-zone-schema.sql 9b6931451 security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql 0ae714753 security-admin/db/postgres/patches/037-create-security-zone-schema.sql 812133077 security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql ea0ff2b9c security-admin/db/sqlanywhere/patches/037-create-security-zone-schema.sql e4bfb78e4 security-admin/db/sqlserver/optimized/0.7.0/ranger_core_db_sqlserver.sql 13be7daa9 security-admin/db/sqlserver/optimized/1.0.0/ranger_core_db_sqlserver.sql 68d3a05bf security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql d45170957 security-admin/db/sqlserver/patches/037-create-security-zone-schema.sql be0287cf4 Diff: https://reviews.apache.org/r/70500/diff/1/ Testing --- Tested the upgrade case for all the db flavor after RANGER-2400 commit. Thanks, Pradeep Agrawal
Re: Review Request 70649: RANGER-2429: Ranger KMS is not starting properly
> On May 15, 2019, 6:17 p.m., Abhay Kulkarni wrote: > > agents-common/src/main/java/org/apache/ranger/plugin/util/RangerSslHelper.java > > Line 49 (original), 49 (patched) > > <https://reviews.apache.org/r/70649/diff/1/?file=2145420#file2145420line49> > > > > If MoreObjects package is included only for printing debug messages, > > please consider removing this dependency altogether. I have create RANGER-2434 to consider removing dependecy of MoreObjects class. Please review other dependent classes mentioned there and add the suggestions(if any) - Pradeep --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70649/#review215282 ------- On May 15, 2019, 12:49 p.m., Pradeep Agrawal wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70649/ > --- > > (Updated May 15, 2019, 12:49 p.m.) > > > Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay > Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh > Mani, Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2429 > https://issues.apache.org/jira/browse/RANGER-2429 > > > Repository: ranger > > > Description > --- > > Ranger KMS start is failing with java.lang.NoClassDefFoundError as zookeeper, > common-codec jar is not available. > > > Diffs > - > > kms/pom.xml 856eebd91 > pom.xml c0a480c65 > > > Diff: https://reviews.apache.org/r/70649/diff/2/ > > > Testing > --- > > Build Ranger with proposed changes and can see required jars in ranger-kms > module. > After installation java.lang.NoClassDefFoundError is not coming and > ranger-kms is loading properly. > > > Thanks, > > Pradeep Agrawal > >
Re: Review Request 70677: RANGER-2432: Upgrade Hadoop Version to 3.1.1
FSWALProvider.createWAL(AsyncFSWALProvider.java:73) > at > org.apache.hadoop.hbase.wal.AsyncFSWALProvider.createWAL(AsyncFSWALProvider.java:48) > at > org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:138) > at > org.apache.hadoop.hbase.wal.AbstractFSWALProvider.getWAL(AbstractFSWALProvider.java:57) > at org.apache.hadoop.hbase.wal.WALFactory.getWAL(WALFactory.java:276) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.getWAL(HRegionServer.java:2064) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.buildServerLoad(HRegionServer.java:1290) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.tryRegionServerReport(HRegionServer.java:1172) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:989) > at > org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:183) > at > org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:129) > at > org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:167) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1709) > at > org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:307) > at > org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:164) > at java.lang.Thread.run(Thread.java:748) > 2019-05-23 16:24:08,524 ERROR [RS:0;10.22.9.38:52417] > regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: > [org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor] > > > Also, Hbase JIRA which addresses this is : > https://issues.apache.org/jira/browse/HBASE-22394 Complete stacktrace after setting test cases log level(ranger.root.logger=WARN,console) in the log4j.properties of ranger hbase plugins : https://gist.github.com/pradeepagrawal8184/798bd00df71797ae74fe7aa4d53f3017 - Pradeep --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70677/#review215499 --- On May 19, 2019, 12:56 p.m., Pradeep Agrawal wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70677/ > --- > > (Updated May 19, 2019, 12:56 p.m.) > > > Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay > Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh > Mani, Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2432 > https://issues.apache.org/jira/browse/RANGER-2432 > > > Repository: ranger > > > Description > --- > > **Following libraries can be upgraded:** > > Hadoop version from 3.0.0 to 3.1.1 > HBase version from 2.0.0 to 2.0.2 > Hive version from 3.0.0 to 3.1.0 > Atlas version from 1.1.0 to 2.0.0 > > > **Note:** Disabled Hbase test cases as setup() in > HBaseRangerAuthorizationTest is failing. It could be due to any of the issues > mentioned in https://issues.apache.org/jira/browse/HBASE-21688. If its so, > then probably fix is not available in the released hbase versions as I have > tried all released Hbase > 2.0.0. Also have tried with all the combination of > hadoop > 3.0.0 > It appears this issue is due to changes in hadoop after 3.0.0 release but not > fixed from hbase end in released version. > > I could not find any work around to get them working as earlier. If you have > any approach to fix these testcases please leave your comment here or in the > Jira. > > **Stacktrace after the testcase failure:** > > Thread 1 (main): > State: RUNNABLE > Blocked count: 50 > Waited count: 499 > Stack: > sun.management.ThreadImpl.getThreadInfo1(Native Method) > sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:178) > sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:139) > > org.apache.hadoop.util.ReflectionUtils.printThreadInfo(ReflectionUtils.java:169) > sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > j
Review Request 70807: RANGER-2464: Upgrade spring, zookeeper, c3p0, jackson-databind, tomcat libraries
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70807/ --- Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2464 https://issues.apache.org/jira/browse/RANGER-2464 Repository: ranger Description --- Here I am proposing to change following libraries: Spring Framework - 4.3.18.RELEASE ==> 4.3.20.RELEASE Spring TestContext Framework - 4.3.18.RELEASE ==> 4.3.20.RELEASE Spring Security - 4.2.7 ==> 4.2.12 zookeeper: 3.4.13 ==> 3.4.14 c3p0: 0.9.5.2 ==> 0.9.5.3 jackson-databind: 2.9.8. ==> 2.9.9 tomcat libraries: 7.0.91 ==> 7.0.94 Diffs - pom.xml 74305dcaf security-admin/pom.xml 3ada9a319 src/main/assembly/admin-web.xml a2d4194be Diff: https://reviews.apache.org/r/70807/diff/1/ Testing --- Tested Ranger admin installation, user login, usersync and other crud operations on service, policy, user and group module. Thanks, Pradeep Agrawal
Review Request 70761: RANGER-2397: HiveServer2 fails to start with Hive Plugin for Ranger
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70761/ --- Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2397 https://issues.apache.org/jira/browse/RANGER-2397 Repository: ranger Description --- **Problem Statement:** HIVE-18777 added Authorization interface to support information_schema integration with external authorization, but it causes the bug with Hive Plugin for Ranger. Specifically, it causes the following error while starting HiveServer2 with the Hive Plugin, and blocks HiveServer2 to start. java.lang.AbstractMethodError: org.apache.ranger.authorization.hive.authorizer.RangerHiveAuthorizer.getHivePolicyProvider()Lorg/apache/hadoop/hive/ql/security/authorization/plugin/HivePolicyProvider **Solution:(provided by Junseung Hwang in the Jira)** : Hive already has an abstract class to prevent this kind of error, so the patch just makes RangerHiveAuthorizerBase to extend AbstractHiveAuthorizer instead of implementing HiveAuthorizer directly. Diffs - hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAuthorizerBase.java afc57b2e8 Diff: https://reviews.apache.org/r/70761/diff/1/ Testing --- Tested Ranger build with this patch. Thanks, Pradeep Agrawal
Re: Review Request 70703: RANGER-2445 : Import of Tagservice for zone
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70703/#review215525 --- Ship it! Ship It! - Pradeep Agrawal On May 23, 2019, 1:24 p.m., Nikhil P wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70703/ > --- > > (Updated May 23, 2019, 1:24 p.m.) > > > Review request for ranger, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, > Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, and > Velmurugan Periasamy. > > > Bugs: RANGER-2445 > https://issues.apache.org/jira/browse/RANGER-2445 > > > Repository: ranger > > > Description > --- > > Import of tagservice is not working for zone > > > Diffs > - > > > security-admin/src/main/java/org/apache/ranger/db/XXSecurityZoneRefTagServiceDao.java > 8006272 > security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java > 0e7cd8f > security-admin/src/main/resources/META-INF/jpa_named_queries.xml e4647b1 > security-admin/src/main/webapp/scripts/views/UploadServicePolicy.js 94362bf > security-admin/src/test/java/org/apache/ranger/rest/TestServiceREST.java > 19f162b > > > Diff: https://reviews.apache.org/r/70703/diff/1/ > > > Testing > --- > > Tested If tag based policies are getting exported and imported properly in > the provided zone. > Tested unzoned and zoned policies are getting imported correctly based to > mapping provided. > > > Thanks, > > Nikhil P > >
Review Request 70893: RANGER-2377: Ranger KnoxSSO authentication when x-forwarded-host header is not forwarded
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70893/ --- Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2377 https://issues.apache.org/jira/browse/RANGER-2377 Repository: ranger Description --- Ranger is unable to forward the request to Ranger if LB is SSL and KnoxSSO is enabled and x-forwarded-host header is not forwarded from LB. Usually Ranger expects that x-forwarded-host shall be provided by LB so current implementation forward the request to the same host but does not change the protocol to https if LB is also SSL(x-forwarded-proto) Proposed solution: proposed patch contains changes which shall replace the x-forwarded-proto value in the request URL if request URL contains protocol http while x-forwarded-proto value is https. Diffs - security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java c3fbe9c23 Diff: https://reviews.apache.org/r/70893/diff/1/ Testing --- Tested knoxsso, knox proxy and ranger HA based authentications. Thanks, Pradeep Agrawal
Review Request 70908: RANGER-2479: Change test connection preferred SQL statement for Oracle DB Flavor
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70908/ --- Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2479 https://issues.apache.org/jira/browse/RANGER-2479 Repository: ranger Description --- Currently test connection preferred SQL statement is set to "Select 1" for all the DB flavors which may not work for Oracle as it should be "Select 1 from dual;" Proposed solution shall replace the value of "ranger.jpa.jdbc.preferredtestquery" property in ranger-admin-default-site.xml file during the ranger installation. Diffs - security-admin/scripts/setup.sh 4916828ce Diff: https://reviews.apache.org/r/70908/diff/1/ Testing --- Tested ranger installation and can see ranger.jpa.jdbc.preferredtestquery values updated to "Select 1 from dual;" for oracle db flavor. Thanks, Pradeep Agrawal
Review Request 70614: RANGER-2419: Improve sql script to skip statements when atlas service def is not supported
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70614/ --- Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2419 https://issues.apache.org/jira/browse/RANGER-2419 Repository: ranger Description --- **Problem Description:** If previous version of ranger(<=0.6) is installed without atlas service def then upgrading ranger to higher version may fail because statement https://github.com/hortonworks/ranger/blob/HDF-3.4.0.0/security-admin/db/oracle/patches/035-update-schema-for-x-policy.sql#L208 will return no results and oracle pl/sql engine doesn't handle it internally. **Proposed solution:** First check if any record exist then only load the record in internal variable. Diffs - security-admin/db/oracle/patches/035-update-schema-for-x-policy.sql 3afdf47d8 Diff: https://reviews.apache.org/r/70614/diff/1/ Testing --- **Steps performed to test the patch on Ranger-admin** To reproduce the case first installed ranger-0.6 for oracle db flavor. In install.properties file provided below values for the properties policymgr_supportedcomponents => hdfs,hive,hbase,knox,storm Now from the master branch. Created Build with patch and untar the build. Opened install.properties and provided db configuration in install.properties which was provided in ranger-0.6 branch build. Called setup.sh Ranger setup process shall try to apply the new db patches. **Expected Behaviour:** Ranger installation should proceed without db patch 035 failure **Actual Behaviour:** Ranger installation finished successfully without db patch 035 failure. Thanks, Pradeep Agrawal
Review Request 70632: RANGER-2423: Ranger KnoxSSO authentication in Ranger HA environment
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70632/ --- Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2423 https://issues.apache.org/jira/browse/RANGER-2423 Repository: ranger Description --- **Problem Description: ** If Ranger LB is non ssl and KnoxSSO is enabled then for the Knox request originURL is the LB URL. However If Ranger LB is ssl and KnoxSSO is enabled then for the Knox request originURL changes to either of Ranger host. It is expected that behaviour of originURL should not change irrespective of ranger ssl/non ssl mode. Currently if Ranger LB is SSL enabled then sending X-Forwarded-Proto and X-Forwarded-SSL header doesn't work. if these headers are not sent from LB then forward URL becomes the actual ranger-admin URL than LB URL. **Proposed Solution:** If LB is SSL then proposed patch shall accept the X-Forwarded-Proto and X-Forwarded-SSL headers and will ensure the origin URL is LB URL. Diffs - security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java 8a6c39b8f Diff: https://reviews.apache.org/r/70632/diff/1/ Testing --- Scenario tested when LB is simple and SSL enabled. 1.Tested Ranger HA with knoxproxy 2.Tested Ranger HA with Knoxsso 3.Tested Ranger HA with knoxproxy and knoxSSO 4.Tested Ranger HA with Knoxsso through curl(using hadoop-jwt token) Thanks, Pradeep Agrawal
Review Request 70675: RANGER-2434: Remove dependency from com.google.common.base.Objects
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70675/ --- Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2434 https://issues.apache.org/jira/browse/RANGER-2434 Repository: ranger Description --- **Problem Statement:** After moving to guava-25, class com.google.common.base.Objects is not available in the library and the toStringHelper() method of the same class has been moved to com.google.common.base.MoreObjects. If debug mode is enabled then ranger-kms start tries to call com.google.common.base.Objects.toStringHelper() to print the debug logs but start fails as the class is no more available in guava jar. **Proposed solution:** There are two way to fix the problem: Option-1: Add the guava library dependency and change the com.google.common.base.Objects to com.google.common.base.MoreObjects. Option-2: concate the attribute and valies using string concatenation. The proposed patch is having option-2 implementation. **Note:** There are other 3 classes(given below) which are using com.google.common.base.MoreObjects but changing the implementation using option-2 shall not help removing the dependency as the same classes are using other classes of the guava library. https://github.com/apache/ranger/blob/master/hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/AuthorizationSession.java https://github.com/apache/ranger/blob/master/hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationCoprocessor.java https://github.com/apache/ranger/blob/master/hbase-agent/src/main/java/org/apache/ranger/authorization/hbase/RangerAuthorizationFilter.java Diffs - agents-common/src/main/java/org/apache/ranger/plugin/util/RangerSslHelper.java 5fc3358c6 Diff: https://reviews.apache.org/r/70675/diff/1/ Testing --- Tested the ranger build and ranger-kms installation Thanks, Pradeep Agrawal
Re: Review Request 70658: RANGER-2436 - Custom condition: Access from cluster
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70658/#review215443 --- Ship it! Ship It! - Pradeep Agrawal On May 21, 2019, 1:27 p.m., Nikhil P wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70658/ > --- > > (Updated May 21, 2019, 1:27 p.m.) > > > Review request for ranger, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, > Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, and > Velmurugan Periasamy. > > > Bugs: RANGER-2436 > https://issues.apache.org/jira/browse/RANGER-2436 > > > Repository: ranger > > > Description > --- > > Include a custom-condition that checks if the current cluster-name matches > one of the condition values. This will enable setting up different > authorization policies depending on the cluster from which access was > performed. > > > Diffs > - > > > agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerAccessedFromClusterCondition.java > PRE-CREATION > > agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerAccessedNotFromClusterCondition.java > PRE-CREATION > > > Diff: https://reviews.apache.org/r/70658/diff/3/ > > > Testing > --- > > 1.Tested If cluster name condition is provided in policy/policyItem condition > then access is granted if that cluster name is given in policy condition > while setting up the policy. > 2.If condition is specified with some cluster names and the cluster from > which access request is coming is not present in condition then access is > denied. > 3.Tested for hive plugin > > > Thanks, > > Nikhil P > >
Re: Review Request 70654: RANGER-2430 Zoneadmin User is able to create policy for those services which is not associated to zone
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70654/#review215398 --- Ship it! Ship It! - Pradeep Agrawal On May 20, 2019, 12:08 p.m., bhavik patel wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70654/ > --- > > (Updated May 20, 2019, 12:08 p.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay > Kulkarni, Madhan Neethiraj, Oliver Szabo, Pradeep Agrawal, Ramesh Mani, > Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2430 > https://issues.apache.org/jira/browse/RANGER-2430 > > > Repository: ranger > > > Description > --- > > Steps to Reproduce: > 1. Create a zone for HDFS service > Zone name: zone1 > Zoneadmin user: zoneadmin1 > 2. Create a hbase policy with zone name as "zone1" through curl from > zoneadmin user: zoneadmin1 > 3. It should not be allowed as hbase service is not associated with zone: > zone1 > > > Diffs > - > > > agents-common/src/main/java/org/apache/ranger/plugin/errors/ValidationErrorCode.java > 3111037 > > agents-common/src/main/java/org/apache/ranger/plugin/model/validation/RangerPolicyValidator.java > 990aab0 > > agents-common/src/test/java/org/apache/ranger/plugin/model/validation/TestRangerPolicyValidator.java > 2c1de4e > > > Diff: https://reviews.apache.org/r/70654/diff/2/ > > > Testing > --- > > User is no more able to create zone policy using curl command within service > which are not associated to zone. > > > Thanks, > > bhavik patel > >
Review Request 70677: RANGER-2432: Upgrade Hadoop Version to 3.1.1
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70677/ --- Review request for ranger. Bugs: RANGER-2432 https://issues.apache.org/jira/browse/RANGER-2432 Repository: ranger Description --- **Following libraries can be upgraded:** Hadoop version from 3.0.0 to 3.1.1 HBase version from 2.0.0 to 2.0.2 Hive version from 3.0.0 to 3.1.0 Atlas version from 1.1.0 to 2.0.0 **Note:** Disabled Hbase test cases as setup() in HBaseRangerAuthorizationTest is failing. It could be due to any of the issues mentioned in https://issues.apache.org/jira/browse/HBASE-21688. If its so, then probably fix is not available in the released hbase versions as I have tried all released Hbase > 2.0.0. Also have tried with all the combination of hadoop > 3.0.0 It appears this issue is due to changes in hadoop after 3.0.0 release but not fixed from hbase end in released version. I could not find any work around to get them working as earlier. If you have any approach to fix these testcases please leave your comment here or in the Jira. **Stacktrace after the testcase failure:** Thread 1 (main): State: RUNNABLE Blocked count: 50 Waited count: 499 Stack: sun.management.ThreadImpl.getThreadInfo1(Native Method) sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:178) sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:139) org.apache.hadoop.util.ReflectionUtils.printThreadInfo(ReflectionUtils.java:169) sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) java.lang.reflect.Method.invoke(Method.java:498) org.apache.hadoop.hbase.util.Threads$PrintThreadInfoLazyHolder$1.printThreadInfo(Threads.java:294) org.apache.hadoop.hbase.util.Threads.printThreadInfo(Threads.java:341) org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:221) org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:413) org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:258) org.apache.hadoop.hbase.MiniHBaseCluster.(MiniHBaseCluster.java:115) org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1038) org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:984) org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:855) org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:849) org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:778) org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.setup(HBaseRangerAuthorizationTest.java:100) [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 207.69 s <<< FAILURE! - in org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest [ERROR] org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest Time elapsed: 207.69 s <<< ERROR! java.io.IOException: Shutting down at org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.setup(HBaseRangerAuthorizationTest.java:100) Caused by: java.lang.RuntimeException: Master not initialized after 20ms seconds at org.apache.ranger.authorization.hbase.HBaseRangerAuthorizationTest.setup(HBaseRangerAuthorizationTest.java:100) Diffs - hbase-agent/src/test/java/org/apache/ranger/authorization/hbase/HBaseRangerAuthorizationTest.java 172b310f1 hive-agent/src/test/java/org/apache/ranger/services/hive/HIVERangerAuthorizerTest.java 62381743f pom.xml 40b053bd2 Diff: https://reviews.apache.org/r/70677/diff/1/ Testing --- Tested Ranger Build, Ranger installation and Ranger admin service, zone, policy, user and group crud operation. Thanks, Pradeep Agrawal
Re: Review Request 70616: RANGER-2347 : Restrict capabilities of security zone administrator and auditor
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70616/#review215181 --- Ship it! Ship It! - Pradeep Agrawal On May 10, 2019, 9:35 a.m., bhavik patel wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70616/ > --- > > (Updated May 10, 2019, 9:35 a.m.) > > > Review request for ranger, Ankita Sinha, Don Bosco Durai, Gautam Borad, Abhay > Kulkarni, Madhan Neethiraj, Oliver Szabo, Pradeep Agrawal, Ramesh Mani, > Selvamohan Neethiraj, Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2347 > https://issues.apache.org/jira/browse/RANGER-2347 > > > Repository: ranger > > > Description > --- > > Zone Admin and zone auditor can see their own zone specific audit access logs. > > > Diffs > - > > security-admin/src/main/java/org/apache/ranger/biz/AssetMgr.java 41b42ca > security-admin/src/main/java/org/apache/ranger/biz/ServiceMgr.java 429c450 > security-admin/src/main/java/org/apache/ranger/rest/AssetREST.java 703d30b > security-admin/src/main/java/org/apache/ranger/rest/SecurityZoneREST.java > f0909ab > security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java > c4ccee9 > > security-admin/src/main/java/org/apache/ranger/solr/SolrAccessAuditsService.java > 9be2ef4 > security-admin/src/test/java/org/apache/ranger/rest/TestAssetREST.java > 1f73709 > > > Diff: https://reviews.apache.org/r/70616/diff/2/ > > > Testing > --- > > Zone admin and zone auditor will only be able to see thier zone associated > logs. > Functioanlity will be same as before for rest of all users. > > > Thanks, > > bhavik patel > >
Re: Review Request 70615: RANGER-2421 - add http client to atlas plugin
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70615/#review215409 --- Ship it! Ship It! - Pradeep Agrawal On May 9, 2019, 9:25 a.m., Zsombor Gegesy wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70615/ > --- > > (Updated May 9, 2019, 9:25 a.m.) > > > Review request for ranger. > > > Bugs: RANGER-2421 > https://issues.apache.org/jira/browse/RANGER-2421 > > > Repository: ranger > > > Description > --- > > Due to http client version difference, and the lack of httpclient-4.5.3.jar > inside the ranger-atlas-plugin/lib/ranger-atlas-plugin-impl/ folder, the > following exception can be seen: > > ava.lang.NoSuchMethodError: > org.apache.http.impl.client.HttpClientBuilder.evictIdleConnections(JLjava/util/concurrent/TimeUnit;)Lorg/apache/http/impl/client/HttpClientBuilder; > at > org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:311) > at > org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:330) > at > org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:268) > at > org.apache.solr.client.solrj.impl.HttpClientUtil.createClient(HttpClientUtil.java:255) > at > org.apache.solr.client.solrj.impl.CloudSolrClient.(CloudSolrClient.java:280) > at > org.apache.solr.client.solrj.impl.CloudSolrClient$Builder.build(CloudSolrClient.java:1600) > at > org.apache.ranger.audit.destination.SolrAuditDestination$1.run(SolrAuditDestination.java:126) > at > org.apache.ranger.audit.destination.SolrAuditDestination$1.run(SolrAuditDestination.java:123) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730) > at > org.apache.ranger.audit.provider.MiscUtil.executePrivilegedAction(MiscUtil.java:516) > at > org.apache.ranger.audit.destination.SolrAuditDestination.connect(SolrAuditDestination.java:123) > at > org.apache.ranger.audit.destination.SolrAuditDestination.init(SolrAuditDestination.java:72) > at > org.apache.ranger.audit.provider.AuditProviderFactory.init(AuditProviderFactory.java:179) > at > org.apache.ranger.plugin.service.RangerBasePlugin.init(RangerBasePlugin.java:217) > > Atlas has a httpclient-4.4.x, which lacks the needed method. > > > Diffs > - > > src/main/assembly/plugin-atlas.xml 4de27b071 > > > Diff: https://reviews.apache.org/r/70615/diff/1/ > > > Testing > --- > > Added the necessary jars to a deployed Atlas server, and noticed that the > exception disapears, and audit correctly sent. > > > Thanks, > > Zsombor Gegesy > >
Review Request 70649: RANGER-2429: Ranger KMS is not starting properly
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70649/ --- Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2429 https://issues.apache.org/jira/browse/RANGER-2429 Repository: ranger Description --- Ranger KMS start is failing with java.lang.NoClassDefFoundError as zookeeper, common-codec jar is not available. Diffs - agents-common/pom.xml 27be45f9d agents-common/src/main/java/org/apache/ranger/plugin/util/RangerSslHelper.java 5fc3358c6 kms/pom.xml 856eebd91 pom.xml c0a480c65 Diff: https://reviews.apache.org/r/70649/diff/1/ Testing --- Build Ranger with proposed changes and can see required jars in ranger-kms module. After installation java.lang.NoClassDefFoundError is not coming and ranger-kms is loading properly. Thanks, Pradeep Agrawal
Re: Review Request 70493: RANGER-2351 : Implement Import / Export of Policies by Zone
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70493/#review214939 --- Ship it! Ship It! - Pradeep Agrawal On April 25, 2019, 12:25 p.m., Nikhil P wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70493/ > --- > > (Updated April 25, 2019, 12:25 p.m.) > > > Review request for ranger, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, > Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, and > Velmurugan Periasamy. > > > Bugs: RANGER-2351 > https://issues.apache.org/jira/browse/RANGER-2351 > > > Repository: ranger > > > Description > --- > > Implement Import / Export of Policies by Zone. > Similar to, how we have Import Export by Service,Import/export by zone > feature is required. > > > Diffs > - > > > security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyRetriever.java > 1bd8dc0 > security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java > 17be098 > security-admin/src/main/java/org/apache/ranger/db/XXPolicyDao.java 6cb85f9 > > security-admin/src/main/java/org/apache/ranger/db/XXSecurityZoneRefServiceDao.java > d568d81 > security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java > 6ddb359 > > security-admin/src/main/java/org/apache/ranger/service/RangerBaseModelService.java > acba730 > > security-admin/src/main/java/org/apache/ranger/service/RangerPolicyService.java > 771feec > > security-admin/src/main/java/org/apache/ranger/service/RangerPolicyServiceBase.java > 8dfbf41 > security-admin/src/main/resources/META-INF/jpa_named_queries.xml 97cc58b > security-admin/src/main/webapp/scripts/modules/globalize/message/en.js > 19bb04b > security-admin/src/main/webapp/scripts/utils/XAGlobals.js 8036d2f > security-admin/src/main/webapp/scripts/views/DownloadServicePolicy.js > 2b5cfdd > security-admin/src/main/webapp/scripts/views/UploadServicePolicy.js 68022e9 > security-admin/src/main/webapp/scripts/views/policymanager/ServiceLayout.js > fd3bb4e > security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js 7589bfa > > security-admin/src/main/webapp/scripts/views/reports/PlugableServiceDiffDetail.js > 3df0ec4 > > security-admin/src/main/webapp/templates/common/uploadservicepolicy_tmpl.html > 150a885 > security-admin/src/test/java/org/apache/ranger/rest/TestServiceREST.java > 8f39607 > > > Diff: https://reviews.apache.org/r/70493/diff/4/ > > > Testing > --- > > 1.tested If policies export after selecting a zone on "Access based policy" > page, exports only policies in that zone. > 2.Validated If no zone is selected on "Access based policy" page, exports > only policies which are not present in any zone. > 3.validated If policy import works properly and policies gets imported to > destination zone , provided correct mapping is given and zones are already > mapped to the services at destination. > > > File Attachments > > > RANGER-2351-Implement-Import-Export-of-Policies-by-Z.patch > > https://reviews.apache.org/media/uploaded/files/2019/04/18/beb14579-4600-4950-af31-2a758b5dfca5__RANGER-2351-Implement-Import-Export-of-Policies-by-Z.patch > > > Thanks, > > Nikhil P > >
Re: Review Request 70565: RANGER-2411 : Restrict Admin role user to create Zone for KMS service
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70565/#review215057 --- Ship it! Ship It! - Pradeep Agrawal On May 6, 2019, 5:50 a.m., Nikhil P wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70565/ > --- > > (Updated May 6, 2019, 5:50 a.m.) > > > Review request for ranger, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, > Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja Polavarapu, and > Velmurugan Periasamy. > > > Bugs: RANGER-2411 > https://issues.apache.org/jira/browse/RANGER-2411 > > > Repository: ranger > > > Description > --- > > Restrict Admin role user to create Zone for KMS service > > > Diffs > - > > security-admin/src/main/java/org/apache/ranger/rest/SecurityZoneREST.java > 4f6fa89 > > security-admin/src/test/java/org/apache/ranger/rest/TestSecurityZoneREST.java > edb3102 > > > Diff: https://reviews.apache.org/r/70565/diff/4/ > > > Testing > --- > > Tested if Admin user is not able to create zone for KMS service. > > > Thanks, > > Nikhil P > >
Re: Review Request 68667: RANGER-2205 Optimize view 'vx_trx_log' to improve query performance
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/68667/#review215041 --- Can you update this patch and make the db patch number to 042 rather 035. - Pradeep Agrawal On Sept. 7, 2018, 6:53 a.m., Xing Peng wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/68667/ > --- > > (Updated Sept. 7, 2018, 6:53 a.m.) > > > Review request for ranger. > > > Bugs: RANGER-2205 > https://issues.apache.org/jira/browse/RANGER-2205 > > > Repository: ranger > > > Description > --- > > When the number of records in the table 'x_trx_log' is 1000, the view > 'vx_trx_log' query takes about 14.651 seconds. > > > Diffs > - > > security-admin/db/mysql/optimized/current/ranger_core_db_mysql.sql > 3f23b0082 > > security-admin/db/mysql/patches/035-update-view-vx_trx_log-and-add-index-on-table-x_trx_log.sql > PRE-CREATION > security-admin/db/oracle/optimized/current/ranger_core_db_oracle.sql > bafdb9676 > > security-admin/db/oracle/patches/035-update-view-vx_trx_log-and-add-index-on-table-x_trx_log.sql > PRE-CREATION > security-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql > 2bc58acfe > > security-admin/db/postgres/patches/035-update-view-vx_trx_log-and-add-index-on-table-x_trx_log.sql > PRE-CREATION > > security-admin/db/sqlanywhere/optimized/current/ranger_core_db_sqlanywhere.sql > 1b64eeac8 > > security-admin/db/sqlanywhere/patches/035-update-view-vx_trx_log-and-add-index-on-table-x_trx_log.sql > PRE-CREATION > security-admin/db/sqlserver/optimized/current/ranger_core_db_sqlserver.sql > 4a216fed9 > > security-admin/db/sqlserver/patches/035-update-view-vx_trx_log-and-add-index-on-table-x_trx_log.sql > PRE-CREATION > > > Diff: https://reviews.apache.org/r/68667/diff/2/ > > > Testing > --- > > > Thanks, > > Xing Peng > >
Review Request 70590: RANGER-2413: Python script to update rangertagsync config properties
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70590/ --- Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2413 https://issues.apache.org/jira/browse/RANGER-2413 Repository: ranger Description --- update_property file might be required to update tagsync config so need to be available under ranger-tagsync module same as ranger-admin and tagsync. Diffs - tagsync/scripts/update_property.py PRE-CREATION Diff: https://reviews.apache.org/r/70590/diff/1/ Testing --- After the build can see the update_properties.py file ranger-tagsync module. Thanks, Pradeep Agrawal
Review Request 71007: RANGER-2493: Ranger takes long time to override a service with many policies
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71007/ --- Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2493 https://issues.apache.org/jira/browse/RANGER-2493 Repository: ranger Description --- ** Problem Statement ** Ranger takes an extremely long time to override a service with many policies, will crash ranger if admin JVM heap size is the 1 GB default Diffs - security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java bf50df3a8 security-admin/src/main/java/org/apache/ranger/common/db/BaseDao.java 51c0de56c security-admin/src/main/java/org/apache/ranger/db/XXPolicyDao.java baf6b6ea4 security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 171d73bfa security-admin/src/main/resources/META-INF/jpa_named_queries.xml eb0384b14 security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java dc845cf80 security-admin/src/test/java/org/apache/ranger/rest/TestServiceREST.java 9b9aa8377 Diff: https://reviews.apache.org/r/71007/diff/1/ Testing --- Tested 500 policy import json and unable to reproduce JVM issue. Thanks, Pradeep Agrawal
Review Request 70952: RANGER-2484: Improve import API to merge the policies if resources are exactly same
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70952/ --- Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2484 https://issues.apache.org/jira/browse/RANGER-2484 Repository: ranger Description --- **Problem Statement:** Observing failure while importing permissions into ranger using ranger import API( /service/plugins/policies/importPoliciesFromFile?updateIfExists=true). Create a ranger policy for resource "db1/table1/column1" with policy name policy-1 in service hivedev. Import permissions for resource "db1/table1/column1" which has policy name policy-2 into hivedev service using import API mentioned above. This results in below failure Validation failure: error code[3010], reason[Another policy already exists for matching resource: policy-name=[policy-1], service=[hivedev]], field[resources], subfield[null], type[semantically incorrect] This issue will be seen only when there is a policy that already exists for the resource with different policy name from the one that is being imported. If the policy names match, the policy is updated properly. ** Proposed Solution: ** Import API can accept an additional parameter 'ignorePolicyName' along with 'updateIfExists'. 'ignorePolicyName' parameter shall be helpful to decide whether policies should be merged or not irrespective of policy name. **Updated Behaviour shall be:** (please add or ask to add if anything is missing below) 1 ) if 'override' flag is true then import api shall delete existing policy from the given service and create the policies received from json file. 2) if 'updateIfExists' flag is true then import api shall update the old policy object with new policy json if policy guid or policy name is matching ; if policy does not exist with given name and guid then it will create the policy but policy creation may fail if a policy exist with the same resource having different name. 3) if 'updateIfExists' flag is true and 'ignorePolicyName' flag is true and if the new policy does not have policy conditions ; then ranger will try to find existing policy for the exactly matching resource ; if exactly matching resource policy is found then the new policy will be merged to existing policy and existing policy name will remain same. if same resource policies are not found then it will try to act as 'updateIfExists' true (mentioned in step-2) Diffs - security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 1d9391f20 security-admin/src/main/java/org/apache/ranger/rest/ServiceRESTUtil.java c49c42698 Diff: https://reviews.apache.org/r/70952/diff/1/ Testing --- 1) Build Ranger with the attached patch, untar the ranger, updated install.properties and installed and started Ranger-admin. 2) Created testuser1 and testuser2. 3) Created a hive service : hivedev 4) Created a policy 'policy2' for resource : db1/table1/column1 , added a policy item for user 'testuser1' with 'select' permissions. 5) Exported the policy json from UI. 6) Updated the exported policy json and updated policy name to 'policy3' , updated policy item by replacing 'testuser1' with 'testuser2' and added 'create' and 'update' permissions. 7) Now tried to import policies with param '' : curl -i -X POST --header "Accept:*/*" -H "Content-Type: multipart/form-data" -F 'file=@/Users/pradeep/Downloads/Ranger_Policies_20190625_104505.json' -u admin:ranger1234 'http://172.22.111.117:6080/service/plugins/policies/importPoliciesFromFile?updateIfExists=true=hive' **Expected Behaviour :** policy2 should be updated with additional policy item having testuser2 permissions of 'create' and 'update'. **Actual Behaviour :** policy2 updated with additional policy item having testuser2 permissions of 'create' and 'update'. Thanks, Pradeep Agrawal
Review Request 70953: RANGER-2485: Security zone filter is causing Ranger audit access request waiting for longer
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70953/ --- Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2485 https://issues.apache.org/jira/browse/RANGER-2485 Repository: ranger Description --- ** Problem Statement ** Currently for logged-in user his all groups information is being pulled to collect on which security zones the user is zone admin or zone audit. if a user is member of large no. of groups then pulling all the info shall take time. Moreover, later part of the current code fetches above details for logged in user irrespective of his role but does not run for the admin user. ** Proposed Solution: ** Is logged user is not an admin user then only should fetch the details. Also there are nested sql calls for reference entries which can be reduced by using join queries. Diffs - security-admin/src/main/java/org/apache/ranger/biz/AssetMgr.java 1a787909c security-admin/src/main/java/org/apache/ranger/db/XXSecurityZoneDao.java c0f06668f security-admin/src/main/resources/META-INF/jpa_named_queries.xml 54e34d5bb Diff: https://reviews.apache.org/r/70953/diff/1/ Testing --- Tested the ranger access audit page from admin,service admin, keyadmin , user, security-zone admin users and its working fine. Thanks, Pradeep Agrawal
Re: Review Request 71015: RANGER-2490:Add https support while using Solr API to upload config set
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71015/#review216552 --- Ship it! Ship It! - Pradeep Agrawal On July 10, 2019, 10:59 a.m., Fatima Khan wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71015/ > --- > > (Updated July 10, 2019, 10:59 a.m.) > > > Review request for ranger, Gautam Borad, Kevin Risden, Oliver Szabo, Pradeep > Agrawal, and Velmurugan Periasamy. > > > Bugs: RANGER-2490 > https://issues.apache.org/jira/browse/RANGER-2490 > > > Repository: ranger > > > Description > --- > > Add https support while using Solr API to upload config set > > > Diffs > - > > > embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/SolrCollectionBootstrapper.java > 02a3f6c > > > Diff: https://reviews.apache.org/r/71015/diff/3/ > > > Testing > --- > > Tested Below Scenario in Ranger with https support > 1.Solr configuration were uploaded successfully > 2.Solr collections were created successfully > > > Thanks, > > Fatima Khan > >
Re: Review Request 71007: RANGER-2493: Ranger takes long time to delete a service with many policies
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71007/ --- (Updated July 9, 2019, 8:40 a.m.) Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Changes --- Updated RR as policy deletion is taking time because it has to create trx logs entry. Added few overridden method in the BaseDao class to avoid the flush() call during each trx log creation. Summary (updated) - RANGER-2493: Ranger takes long time to delete a service with many policies Bugs: RANGER-2493 https://issues.apache.org/jira/browse/RANGER-2493 Repository: ranger Description (updated) --- ** Problem Statement ** Ranger takes an extremely long time to delete a service with many policies, will crash ranger if admin JVM heap size is the 1 GB default Diffs (updated) - security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java 055cd382b security-admin/src/main/java/org/apache/ranger/biz/RangerBizUtil.java 0effa6789 security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java bf50df3a8 security-admin/src/main/java/org/apache/ranger/common/db/BaseDao.java 51c0de56c security-admin/src/main/java/org/apache/ranger/db/XXPolicyDao.java baf6b6ea4 security-admin/src/main/java/org/apache/ranger/patch/PatchForUpdatingPolicyJson_J10019.java 1a1cc23bd security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 171d73bfa security-admin/src/main/java/org/apache/ranger/service/RangerBaseModelService.java b16bfe7b3 security-admin/src/main/java/org/apache/ranger/service/RangerDataHistService.java 7bd0681b1 security-admin/src/main/resources/META-INF/jpa_named_queries.xml eb0384b14 security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java dc845cf80 security-admin/src/test/java/org/apache/ranger/rest/TestServiceREST.java 9b9aa8377 security-admin/src/test/java/org/apache/ranger/service/TestRangerDataHistService.java 65e273c4e Diff: https://reviews.apache.org/r/71007/diff/4/ Changes: https://reviews.apache.org/r/71007/diff/3-4/ Testing (updated) --- Tested with deleting a service having 2000 policies and unable to reproduce JVM issue. Tested with importing policy json having 2000 policies and override option. 500 policies deletion took 100 seconds while 1000 policies deletion took 4 minutes. Thanks, Pradeep Agrawal
Re: Review Request 71007: RANGER-2493: Ranger takes long time to override a service with many policies
rsistence-context/ https://en.wikibooks.org/wiki/Java_Persistence/Persisting#Example_flush I came across few articles(mentioned above) and thought that if we are not going to reuse those deleted object then we don't need to call flush() for them as flush() call will immediately try to send delete entity statements to the DB. Also if all the bulk policy delete is going to be in single transactions then probably JPA will keep all these objects in a cache and update entities cascading in the memory untill the transaction completes. In the first articles its mentioned that the flush() call forces JPA/Eclipselink to perform a dirty check on all managed entities and to create and execute SQL statements for all pending insert, update or delete operations. That slows down the application because it prevents JPA/Eclipselink from using several internal optimizations. - Pradeep --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71007/#review216348 --- On July 4, 2019, 7:17 a.m., Pradeep Agrawal wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71007/ > --- > > (Updated July 4, 2019, 7:17 a.m.) > > > Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay > Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh > Mani, Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2493 > https://issues.apache.org/jira/browse/RANGER-2493 > > > Repository: ranger > > > Description > --- > > ** Problem Statement ** Ranger takes an extremely long time to override a > service with many policies, will crash ranger if admin JVM heap size is the 1 > GB default > > > Diffs > - > > security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java > bf50df3a8 > security-admin/src/main/java/org/apache/ranger/common/db/BaseDao.java > 51c0de56c > security-admin/src/main/java/org/apache/ranger/db/XXPolicyDao.java > baf6b6ea4 > security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java > 171d73bfa > security-admin/src/main/resources/META-INF/jpa_named_queries.xml eb0384b14 > security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java > dc845cf80 > security-admin/src/test/java/org/apache/ranger/rest/TestServiceREST.java > 9b9aa8377 > > > Diff: https://reviews.apache.org/r/71007/diff/2/ > > > Testing > --- > > Tested 500 policy import json and unable to reproduce JVM issue. > > > Thanks, > > Pradeep Agrawal > >
Re: Review Request 71007: RANGER-2493: Ranger takes long time to override a service with many policies
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71007/ --- (Updated July 4, 2019, 7:17 a.m.) Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Changes --- Addressed review comments and added required note. Bugs: RANGER-2493 https://issues.apache.org/jira/browse/RANGER-2493 Repository: ranger Description --- ** Problem Statement ** Ranger takes an extremely long time to override a service with many policies, will crash ranger if admin JVM heap size is the 1 GB default Diffs (updated) - security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java bf50df3a8 security-admin/src/main/java/org/apache/ranger/common/db/BaseDao.java 51c0de56c security-admin/src/main/java/org/apache/ranger/db/XXPolicyDao.java baf6b6ea4 security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 171d73bfa security-admin/src/main/resources/META-INF/jpa_named_queries.xml eb0384b14 security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java dc845cf80 security-admin/src/test/java/org/apache/ranger/rest/TestServiceREST.java 9b9aa8377 Diff: https://reviews.apache.org/r/71007/diff/2/ Changes: https://reviews.apache.org/r/71007/diff/1-2/ Testing --- Tested 500 policy import json and unable to reproduce JVM issue. Thanks, Pradeep Agrawal
Review Request 70986: RANGER-2496: Update Spring Security version to 4.2.13
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70986/ --- Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2496 https://issues.apache.org/jira/browse/RANGER-2496 Repository: ranger Description --- Here I am proposing to Upgrade Spring Security version to 4.2.13 Diffs - pom.xml b5832bd66 Diff: https://reviews.apache.org/r/70986/diff/1/ Testing --- Tested Ranger build, installation, start and login. Thanks, Pradeep Agrawal
Review Request 71023: RANGER-2500: Zone Policies not getting imported when 'updateIfExists=true' is passed through curl
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71023/ --- Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2500 https://issues.apache.org/jira/browse/RANGER-2500 Repository: ranger Description --- **Problem Statement:** Zone Policies import may fail when 'updateIfExists=true' is passed through curl. After zone implementation its possible that same policy may exist in a zone and unzone. if it exists unzone then the current implementation will bring that existing db object in memory and try to update the same with new request. since the request will try to update unzone policy to a zone policy; the request will fail during the policy update validation as there is a check that policy zone can't be updated. **Proposed Solution:** The proposed solution will bring policy from respective zone only. This way policy update request will happen with in the same zone policy and update validation shall pass. **Other notes:** When updateIfExists true is passed then following flow will happen. 1) First it will check if the new policy guid and the existing policy guid is same or not. if found same then it will try to update that record. please note that if existing object is in unzone and the new request is for the zone then request may fail due to policy update validation check as mentioned above. 2) if first condition is not applied then it will try to get a policy by name and zone. if record is found then it will try to update that record. 3) if first and second condition is not applied then it will try to get a policy by name. if record is found then it will try to update that record. 4) if any of the above condition fail then policy update will fail but if all three conditions are not met then it will try to create the policy. Diffs - security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java 171d73bfa Diff: https://reviews.apache.org/r/71023/diff/1/ Testing --- Tested with 'updateIfExists=true' param and unable to reproduce the case. Thanks, Pradeep Agrawal
Review Request 70522: RANGER-2407: Update default header values sent from Ranger
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70522/ --- Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Bugs: RANGER-2407 https://issues.apache.org/jira/browse/RANGER-2407 Repository: ranger Description --- Update/Remove the default server entry. Since its not possible to remove the default entry completely, I am proposing to update the default value and the new value shall be "Apache Ranger" Diffs - embeddedwebserver/src/main/java/org/apache/ranger/server/tomcat/EmbeddedServer.java 3963df19e security-admin/src/main/resources/conf.dist/security-applicationContext.xml 39f236d2a Diff: https://reviews.apache.org/r/70522/diff/1/ Testing --- After Ranger setup able to receive value "Apache Ranger" from Ranger REST call. Thanks, Pradeep Agrawal
Re: Review Request 70954: RANGER-2487 : Resource policy names with a characters that are typically HTML escaped mutate and grow as they are saved.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70954/#review216183 --- Ship it! Ship It! - Pradeep Agrawal On June 26, 2019, 3:04 p.m., Nitin Galave wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70954/ > --- > > (Updated June 26, 2019, 3:04 p.m.) > > > Review request for ranger, Gautam Borad, Mehul Parikh, Pradeep Agrawal, and > Velmurugan Periasamy. > > > Bugs: RANGER-2487 > https://issues.apache.org/jira/browse/RANGER-2487 > > > Repository: ranger > > > Description > --- > > If a resource based policy is given a name that contains a character that is > typically HTML escape, such as the greater than sign '>', which is HTML > escaped '>', then whenever that policy is saved, the name of the policy will > be changed by the ranger you to contain the HTML escape characters. > > For example, if I name a policy mydb->mytable, then when that policy is > edited and saved in the UI, its name will change to mydb-mytable. Because > the ampersand is itself an HTML escaped character, if I save the policy > again, this name will be changed to mydb-gt;mytable.Etc.. > > > Diffs > - > > security-admin/src/main/webapp/scripts/models/RangerPolicy.js 30e36ac > security-admin/src/main/webapp/scripts/modules/globalize/message/en.js > 8d921f7 > security-admin/src/main/webapp/scripts/utils/XAUtils.js 79f397e > security-admin/src/main/webapp/scripts/views/policies/RangerPolicyForm.js > b82654e > > > Diff: https://reviews.apache.org/r/70954/diff/1/ > > > Testing > --- > > Verified CRUD operation for policy. > > > Thanks, > > Nitin Galave > >
Re: Review Request 70938: RANGER-2467 : similar to clusterName custom condition, add clusterType custome condition.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70938/#review216239 --- Ship it! Ship It! - Pradeep Agrawal On June 28, 2019, 10:04 a.m., Mateen Mansoori wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70938/ > --- > > (Updated June 28, 2019, 10:04 a.m.) > > > Review request for ranger, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, > Nikhil P, Pradeep Agrawal, and Velmurugan Periasamy. > > > Bugs: RANGER-2467 > https://issues.apache.org/jira/browse/RANGER-2467 > > > Repository: ranger > > > Description > --- > > Need to add clusterType custom condition,This will help with custom > conditions like “Accessed from cluster type?” > > > Diffs > - > > > agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerAccessedFromClusterTypeCondition.java > PRE-CREATION > > agents-common/src/main/java/org/apache/ranger/plugin/conditionevaluator/RangerAccessedNotFromClusterTypeCondition.java > PRE-CREATION > > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessRequest.java > cb06d26 > > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessRequestImpl.java > 1f2f8ea > > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessRequestReadOnly.java > d5563bd > > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPluginContext.java > 36dcec1 > > agents-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyEngineImpl.java > 0edf149 > > agents-common/src/main/java/org/apache/ranger/plugin/service/RangerAuthContext.java > 67c068b > > agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestPolicyACLs.java > 33b26e0 > > agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestPolicyDb.java > f373339 > > agents-common/src/test/java/org/apache/ranger/plugin/policyengine/TestPolicyEngine.java > cce5129 > > hbase-agent/src/test/java/org/apache/ranger/authorization/hbase/TestPolicyEngine.java > 6dd81fa > > hive-agent/src/main/java/org/apache/ranger/authorization/hive/authorizer/RangerHiveAccessRequest.java > ce5cf64 > > > Diff: https://reviews.apache.org/r/70938/diff/2/ > > > Testing > --- > > Tested for hive plugin, whether clusterType is being set properly in > RangerAccessRequest. > > > Thanks, > > Mateen Mansoori > >
Re: Review Request 70893: RANGER-2477: Ranger KnoxSSO authentication when x-forwarded-host header is not forwarded
> On June 29, 2019, 5:35 a.m., Don Bosco Durai wrote: > > security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java > > Lines 281 (patched) > > <https://reviews.apache.org/r/70893/diff/2/?file=2152885#file2152885line281> > > > > To be on the safe side, you might want to consider using relaceFirst(). Thanks. - Pradeep --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70893/#review216245 --- On June 29, 2019, 5:50 a.m., Pradeep Agrawal wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70893/ > --- > > (Updated June 29, 2019, 5:50 a.m.) > > > Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay > Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh > Mani, Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2477 > https://issues.apache.org/jira/browse/RANGER-2477 > > > Repository: ranger > > > Description > --- > > Ranger is unable to forward the request to Ranger if LB is SSL and KnoxSSO is > enabled and x-forwarded-host header is not forwarded from LB. Usually Ranger > expects that x-forwarded-host shall be provided by LB so current > implementation forward the request to the same host but does not change the > protocol to https if LB is also SSL(x-forwarded-proto) > > Proposed solution: proposed patch contains changes which shall replace the > x-forwarded-proto value in the request URL if request URL contains protocol > http while x-forwarded-proto value is https. > > > Diffs > - > > > security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java > c3fbe9c23 > > > Diff: https://reviews.apache.org/r/70893/diff/3/ > > > Testing > --- > > Tested knoxsso, knox proxy and ranger HA based authentications. > > > Thanks, > > Pradeep Agrawal > >
Re: Review Request 70893: RANGER-2477: Ranger KnoxSSO authentication when x-forwarded-host header is not forwarded
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70893/ --- (Updated June 29, 2019, 5:50 a.m.) Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Changes --- Addressed review comment: Changed replace() with replaceFirst() to avoid unrequired replaces. Bugs: RANGER-2477 https://issues.apache.org/jira/browse/RANGER-2477 Repository: ranger Description --- Ranger is unable to forward the request to Ranger if LB is SSL and KnoxSSO is enabled and x-forwarded-host header is not forwarded from LB. Usually Ranger expects that x-forwarded-host shall be provided by LB so current implementation forward the request to the same host but does not change the protocol to https if LB is also SSL(x-forwarded-proto) Proposed solution: proposed patch contains changes which shall replace the x-forwarded-proto value in the request URL if request URL contains protocol http while x-forwarded-proto value is https. Diffs (updated) - security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java c3fbe9c23 Diff: https://reviews.apache.org/r/70893/diff/3/ Changes: https://reviews.apache.org/r/70893/diff/2-3/ Testing --- Tested knoxsso, knox proxy and ranger HA based authentications. Thanks, Pradeep Agrawal
Re: Review Request 70893: RANGER-2477: Ranger KnoxSSO authentication when x-forwarded-host header is not forwarded
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70893/ --- (Updated June 29, 2019, 5:04 a.m.) Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh Mani, Sailaja Polavarapu, and Velmurugan Periasamy. Changes --- Addressed review comment Bugs: RANGER-2477 https://issues.apache.org/jira/browse/RANGER-2477 Repository: ranger Description --- Ranger is unable to forward the request to Ranger if LB is SSL and KnoxSSO is enabled and x-forwarded-host header is not forwarded from LB. Usually Ranger expects that x-forwarded-host shall be provided by LB so current implementation forward the request to the same host but does not change the protocol to https if LB is also SSL(x-forwarded-proto) Proposed solution: proposed patch contains changes which shall replace the x-forwarded-proto value in the request URL if request URL contains protocol http while x-forwarded-proto value is https. Diffs (updated) - security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java c3fbe9c23 Diff: https://reviews.apache.org/r/70893/diff/2/ Changes: https://reviews.apache.org/r/70893/diff/1-2/ Testing --- Tested knoxsso, knox proxy and ranger HA based authentications. Thanks, Pradeep Agrawal
Re: Review Request 70893: RANGER-2477: Ranger KnoxSSO authentication when x-forwarded-host header is not forwarded
> On June 29, 2019, 2:58 a.m., Don Bosco Durai wrote: > > security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java > > Lines 280 (patched) > > <https://reviews.apache.org/r/70893/diff/1/?file=2151380#file2151380line280> > > > > Just curios, what happens if the request URL is https and > > xForwardedProt is http? Is it a valid combination? > > > > Also, any reason, we are not checking just for "http:"? Instead, 2 > > conditions? =>I don't think its a valid combination but it may happen. Usually, xForwardedProto is either provided as header by user or it can be overridden by proxy/load-balancer. if its decided by proxy/load-balancer then it will be according to the request but user may make a mistake by putting http rather actually https is needed. line 283 will make the forwardURL similar to the requestURL and will ignore the xForwardedProto value. => When load balancer is in https and ranger is in http and knoxSSO is enabled and if x-forwarded-host is not provided then we can assume that the request can be forwarded to the same host from where the request is coming here though LB is in ssl, received requestURL was in http(bit strange may be LB issue probably similar to https://stackoverflow.com/questions/29469929/why-does-request-getrequesturl-return-non-https-url) so to handle this situation I am considering xForwardedProto value which was https so replacing http with https. since i am using startsWith() method and https starts with http so just for http case i need to add extra condition here as i want to replace http only and avoid wrong replaces like https -> httpss if requestURL contains https then line 283 shall make the requestURL to be a forwardURL. If there is a better way to handle this please advice. - Pradeep --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70893/#review216240 ----------- On June 19, 2019, 4:22 p.m., Pradeep Agrawal wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70893/ > --- > > (Updated June 19, 2019, 4:22 p.m.) > > > Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay > Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh > Mani, Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2477 > https://issues.apache.org/jira/browse/RANGER-2477 > > > Repository: ranger > > > Description > --- > > Ranger is unable to forward the request to Ranger if LB is SSL and KnoxSSO is > enabled and x-forwarded-host header is not forwarded from LB. Usually Ranger > expects that x-forwarded-host shall be provided by LB so current > implementation forward the request to the same host but does not change the > protocol to https if LB is also SSL(x-forwarded-proto) > > Proposed solution: proposed patch contains changes which shall replace the > x-forwarded-proto value in the request URL if request URL contains protocol > http while x-forwarded-proto value is https. > > > Diffs > - > > > security-admin/src/main/java/org/apache/ranger/security/web/filter/RangerSSOAuthenticationFilter.java > c3fbe9c23 > > > Diff: https://reviews.apache.org/r/70893/diff/1/ > > > Testing > --- > > Tested knoxsso, knox proxy and ranger HA based authentications. > > > Thanks, > > Pradeep Agrawal > >
Re: Review Request 71166: RANGER-2518: Allow service creator to delete the service
> On July 31, 2019, 8:13 a.m., Madhan Neethiraj wrote: > > security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java > > Lines 805 (patched) > > <https://reviews.apache.org/r/71166/diff/3/?file=2158473#file2158473line811> > > > > Consider checking if 'session == null' - as most callers of > > ContextUtil.getCurrentUserSession() seem to do. Please review update other > > such instances as well - like #827. Added null check in few other methods as well though they are not related to this jira. - Pradeep --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71166/#review216980 ------- On July 31, 2019, 7:26 a.m., Pradeep Agrawal wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71166/ > --- > > (Updated July 31, 2019, 7:26 a.m.) > > > Review request for ranger, Ankita Sinha, bhavik patel, Gautam Borad, Abhay > Kulkarni, Madhan Neethiraj, Mehul Parikh, Nikhil P, Nitin Galave, Ramesh > Mani, Sailaja Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2518 > https://issues.apache.org/jira/browse/RANGER-2518 > > > Repository: ranger > > > Description > --- > > **Problem Statement:** Current service admin user can not delete the ranger > service. > > **Proposed Solution:** > > As during service/repo creation, creator reference get added in added_by_id > field of x_service table. we can compare the logged in user id and service > creator id. if both matches then no need to check the admin permissions. This > will allow service creator user to delete the service. > > > Diffs > - > > security-admin/src/main/java/org/apache/ranger/biz/RangerBizUtil.java > 0ad7df2dd > security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java > 84202335d > security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java > b06273cc5 > > security-admin/src/main/java/org/apache/ranger/service/XResourceService.java > 43a855e6d > > security-admin/src/main/java/org/apache/ranger/service/XUgsyncAuditInfoService.java > d613c700a > security-admin/src/test/java/org/apache/ranger/rest/TestServiceREST.java > 34be7e914 > > > Diff: https://reviews.apache.org/r/71166/diff/4/ > > > Testing > --- > > Built, Installed and started ranger with the patch. > Login from admin user. > Create a user 'testuser1' with 'admin' role > Logout from admin user and login from 'testuser1' > create hive service 'hivedev' > Logout from 'testuser1' user and login from 'admin' > Change the role of 'testuser1' user from 'admin' to 'user' > Logout from admin user > execute below curl command by using 'testuser1' user's credential > curl -i --header "Accept:application/json" -H "Content-Type: > application/json" -u testuser1:user1234 -X DELETE > 'http://172.22.111.117:6080/service/plugins/services/5' > > > **Expected behaviour:** > service should get deleted and return http response code 204 with no content. > > **Actual behaviour:** > Response received: > > HTTP/1.1 204 No Content > Set-Cookie: RANGERADMINSESSIONID=3F481200366A0823073FFE27FF982A84; Path=/; > HttpOnly > X-Frame-Options: DENY > X-XSS-Protection: 1; mode=block > Strict-Transport-Security: max-age=31536000; includeSubDomains > Content-Security-Policy: default-src 'none'; script-src 'self' > 'unsafe-inline' 'unsafe-eval'; connect-src 'self'; img-src 'self'; style-src > 'self' 'unsafe-inline';font-src 'self' > Cache-Control: no-cache, no-store, max-age=0, must-revalidate > Pragma: no-cache > Expires: 0 > X-Content-Type-Options: nosniff > Content-Type: application/json > Date: Thu, 25 Jul 2019 13:50:13 GMT > Server: Apache Ranger > > > Thanks, > > Pradeep Agrawal > >
Re: Review Request 71155: RANGER-2516 Update Ranger default policies to provide entity-read access to public group
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71155/#review216941 --- Ship it! Ship It! - Pradeep Agrawal On July 26, 2019, 8:01 a.m., Nikhil P wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71155/ > --- > > (Updated July 26, 2019, 8:01 a.m.) > > > Review request for ranger, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, > Nixon Rodrigues, Pradeep Agrawal, Ramesh Mani, Selvamohan Neethiraj, Sailaja > Polavarapu, and Velmurugan Periasamy. > > > Bugs: RANGER-2516 > https://issues.apache.org/jira/browse/RANGER-2516 > > > Repository: ranger > > > Description > --- > > Update Ranger default policies to provide entity-read access to public group > > > Diffs > - > > > plugin-atlas/src/main/java/org/apache/ranger/services/atlas/RangerServiceAtlas.java > 459e791c3 > > > Diff: https://reviews.apache.org/r/71155/diff/3/ > > > Testing > --- > > Tested on local if additional policy is getting created for required > resources. > > > Thanks, > > Nikhil P > >
Re: Review Request 71114: RANGER-2506 Add cluster name on plugin status tab.
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/71114/#review216940 --- Ship it! Ship It! - Pradeep Agrawal On July 25, 2019, 12:37 p.m., Mateen Mansoori wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/71114/ > --- > > (Updated July 25, 2019, 12:37 p.m.) > > > Review request for ranger, Gautam Borad, Abhay Kulkarni, Madhan Neethiraj, > Mehul Parikh, Nikhil P, Pradeep Agrawal, and Velmurugan Periasamy. > > > Bugs: RANGER-2506 > https://issues.apache.org/jira/browse/RANGER-2506 > > > Repository: ranger > > > Description > --- > > Add cluster name on plugin status tab. > > > Diffs > - > > agents-common/src/main/java/org/apache/ranger/plugin/util/SearchFilter.java > 7e672c3 > security-admin/src/main/java/org/apache/ranger/biz/AssetMgr.java f5fce93 > security-admin/src/main/java/org/apache/ranger/common/RangerSearchUtil.java > fc57601 > security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java > 990fc2b > security-admin/src/main/java/org/apache/ranger/rest/TagREST.java e650968 > > security-admin/src/main/java/org/apache/ranger/service/RangerPluginInfoService.java > 3ca3b8f > security-admin/src/main/webapp/scripts/views/reports/AuditLayout.js f7237fd > > > Diff: https://reviews.apache.org/r/71114/diff/2/ > > > Testing > --- > > Cluster Name is getting displayed on Audit>>plugin status tab. > Checked Cluster Name Search Filtering on Audit>>plugin status tab. > > > Thanks, > > Mateen Mansoori > >
