-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68881/
-----------------------------------------------------------
(Updated Oct. 4, 2018, 3:20 p.m.)
Review request for ranger, Madhan Neethiraj, Mehul Parikh, Nikhil P, Pradeep
Agrawal, and Velmurugan Periasamy.
Summary (updated)
-----------------
RANGER-2203: Review and update database schema for ranger policies to minimize
database queries/updates
Bugs: RANGER-2186, RANGER-2195 and RANGER-2219
https://issues.apache.org/jira/browse/RANGER-2186
https://issues.apache.org/jira/browse/RANGER-2195
https://issues.apache.org/jira/browse/RANGER-2219
Repository: ranger
Description
-------
Notes:
This is a composite patch for updating database schema for Ranger policy and
tag related objects.
Notes:
This patch is more complete than the one posted earlier
(https://reviews.apache.org/r/68876/). Database scripts to drop foreign-keys
are missing for some database types. They need to be added.
This patch also subsumes back-ports of RANGER-2186 and RANGER-2195 from master
to ranger-0.7 branch. Porting those JIRA separately involved fixing conflicts
in many common files.
Description:
Currently, ranger policies are fully normalized and stored in a multiple
Relational database tables. There is a performance overhead incurred when
retrieving a ranger policy, as multiple database accesses are required to fully
reconstruct it. This is significant when there are large ranger policies (that
is, the number of resources addressed by the policy is large), and/or when
there is a large number of ranger policies in an installation.
This Jira tracks alternate design of database schema, where a policy is stored
in a de-normalized way, in its entirely, in one database table (preferably as a
JSON string).
Diffs
-----
agents-common/src/main/java/org/apache/ranger/authorization/utils/JsonUtils.java
PRE-CREATION
agents-common/src/main/java/org/apache/ranger/plugin/model/RangerTag.java
743b02825
agents-common/src/main/java/org/apache/ranger/plugin/store/AbstractServiceStore.java
a75ca59e8
agents-common/src/main/java/org/apache/ranger/plugin/store/TagStore.java
5918b1292
embeddedwebserver/scripts/ranger-admin-services.sh 19dab2a9d
security-admin/db/mysql/patches/035-update-schema-for-x-policy.sql
PRE-CREATION
security-admin/db/mysql/patches/040-denormalize-tag-tables.sql PRE-CREATION
security-admin/db/oracle/patches/035-update-schema-for-x-policy.sql
PRE-CREATION
security-admin/db/oracle/patches/040-denormalize-tag-tables.sql PRE-CREATION
security-admin/db/postgres/patches/035-update-schema-for-x-policy.sql
PRE-CREATION
security-admin/db/postgres/patches/040-denormalize-tag-tables.sql
PRE-CREATION
security-admin/db/sqlanywhere/patches/035-update-schema-for-x-policy.sql
PRE-CREATION
security-admin/db/sqlanywhere/patches/040-denormalize-tag-tables.sql
PRE-CREATION
security-admin/db/sqlserver/patches/035-update-schema-for-x-policy.sql
PRE-CREATION
security-admin/db/sqlserver/patches/040-denormalize-tag-tables.sql
PRE-CREATION
security-admin/scripts/db_setup.py d4f37ed6f
security-admin/src/main/java/org/apache/ranger/biz/PolicyRefUpdater.java
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/biz/RangerPolicyRetriever.java
354ab1cba
security-admin/src/main/java/org/apache/ranger/biz/RangerTagDBRetriever.java
53683ec04
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java
ceee8cef2
security-admin/src/main/java/org/apache/ranger/biz/TagDBStore.java 3234be6a2
security-admin/src/main/java/org/apache/ranger/common/db/RangerTransactionSynchronizationAdapter.java
2a62fb408
security-admin/src/main/java/org/apache/ranger/db/RangerDaoManager.java
e3b878b3e
security-admin/src/main/java/org/apache/ranger/db/RangerDaoManagerBase.java
d718441ee
security-admin/src/main/java/org/apache/ranger/db/XXAccessTypeDefDao.java
8f3a50687
security-admin/src/main/java/org/apache/ranger/db/XXDataMaskTypeDefDao.java
f6e1aff0b
security-admin/src/main/java/org/apache/ranger/db/XXGroupDao.java 19e2e11f3
security-admin/src/main/java/org/apache/ranger/db/XXPolicyConditionDefDao.java
21afcac89
security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemAccessDao.java
de37e10cd
security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemConditionDao.java
11596ef4d
security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemDataMaskInfoDao.java
a8418c626
security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemGroupPermDao.java
8c05699b2
security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemRowFilterInfoDao.java
4618e7dc7
security-admin/src/main/java/org/apache/ranger/db/XXPolicyItemUserPermDao.java
40a0da13b
security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefAccessTypeDao.java
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefConditionDao.java
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefDataMaskTypeDao.java
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefGroupDao.java
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefResourceDao.java
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/db/XXPolicyRefUserDao.java
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/db/XXPolicyResourceDao.java
4b04b9664
security-admin/src/main/java/org/apache/ranger/db/XXPolicyResourceMapDao.java
70657370a
security-admin/src/main/java/org/apache/ranger/db/XXResourceDefDao.java
aee56e833
security-admin/src/main/java/org/apache/ranger/db/XXServiceResourceDao.java
f87c0ae59
security-admin/src/main/java/org/apache/ranger/db/XXServiceResourceElementDao.java
72fe2140b
security-admin/src/main/java/org/apache/ranger/db/XXServiceResourceElementValueDao.java
e726d047c
security-admin/src/main/java/org/apache/ranger/db/XXServiceVersionInfoDao.java
0098bff23
security-admin/src/main/java/org/apache/ranger/db/XXTagAttributeDao.java
e361b33bd
security-admin/src/main/java/org/apache/ranger/db/XXTagAttributeDefDao.java
145399f1c
security-admin/src/main/java/org/apache/ranger/db/XXTagDao.java 77428dbf9
security-admin/src/main/java/org/apache/ranger/db/XXTagDefDao.java dfd2fcb5d
security-admin/src/main/java/org/apache/ranger/db/XXTagResourceMapDao.java
bbcd54618
security-admin/src/main/java/org/apache/ranger/db/XXUserDao.java 7af1bf96a
security-admin/src/main/java/org/apache/ranger/entity/XXDBBase.java 8405eb3e4
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyBase.java
69d28bb54
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefAccessType.java
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefCondition.java
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefDataMaskType.java
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefGroup.java
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefResource.java
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/entity/XXPolicyRefUser.java
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/entity/XXServiceResource.java
961627a3c
security-admin/src/main/java/org/apache/ranger/entity/XXTag.java 9155385ec
security-admin/src/main/java/org/apache/ranger/entity/XXTagDef.java 818908ba8
security-admin/src/main/java/org/apache/ranger/patch/PatchForUpdatingPolicyJson_J10017.java
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/patch/PatchForUpdatingTagsJson_J10020.java
PRE-CREATION
security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java
6685e1a58
security-admin/src/main/java/org/apache/ranger/service/RangerAuditFields.java
7223f109c
security-admin/src/main/java/org/apache/ranger/service/RangerPolicyServiceBase.java
0195c72a7
security-admin/src/main/java/org/apache/ranger/service/RangerServiceResourceService.java
d7256802a
security-admin/src/main/java/org/apache/ranger/service/RangerServiceResourceServiceBase.java
6af682a81
security-admin/src/main/java/org/apache/ranger/service/RangerTagDefService.java
82eb252e6
security-admin/src/main/java/org/apache/ranger/service/RangerTagService.java
28b9115fa
security-admin/src/main/resources/META-INF/jpa_named_queries.xml 4a7055dee
security-admin/src/test/java/org/apache/ranger/biz/TestServiceDBStore.java
44523fb22
security-admin/src/test/java/org/apache/ranger/rest/TestServiceREST.java
68908b205
security-admin/src/test/java/org/apache/ranger/service/TestRangerServiceDefService.java
d65b0d6ee
security-admin/src/test/java/org/apache/ranger/service/TestRangerTagDefService.java
2ca0fd2fb
Diff: https://reviews.apache.org/r/68881/diff/7/
Testing
-------
Thanks,
Abhay Kulkarni
