mneethiraj commented on code in PR #584:
URL: https://github.com/apache/ranger/pull/584#discussion_r2226391712
##########
security-admin/src/main/java/org/apache/ranger/common/RangerServicePoliciesCache.java:
##########
@@ -358,12 +358,13 @@ ServicePolicies getLatestOrCached(String serviceName,
ServiceStore serviceStore,
if (servicePoliciesForDeltas != null &&
servicePoliciesForDeltas.getPolicyDeltas() != null) {
LOG.debug("Deltas were requested. Returning
deltas from lastKnownVersion:[{}]", lastKnownVersion);
-
if (isDeltaCacheReinitialized) {
this.deltaCache = new
ServicePolicyDeltasCache(lastKnownVersion, servicePoliciesForDeltas);
}
+ LOG.debug("servicePoliciesForDeltas = " +
servicePoliciesForDeltas.getServiceConfig());
ret = servicePoliciesForDeltas;
+ LOG.debug("ret = " + ret.getServiceConfig());
Review Comment:
replace string concat with parameterized log message.
```
LOG.debug("ret = {}", ret.getServiceConfig());
```
##########
security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java:
##########
@@ -2672,6 +2672,9 @@ public ServicePolicies
getSecureServicePoliciesIfUpdated(@PathParam("serviceName
throw restErrorUtil.createRESTException(httpCode, logMsg,
logError);
}
+ if (ret != null) {
+ LOG.debug("<== ServiceREST.getSecureServicePoliciesIfUpdated():
configs =" + ret.getServiceConfig());
Review Comment:
- to be consistent, remove `<== ` from the log message
- replace string concat with parameterized message; also replace `if (ret !=
null)` at line 2675 with an inline check:
```
LOG.debug("getSecureServicePoliciesIfUpdated(): configs ={}", ret == null ?
ret : ret.getServiceConfig());
```
##########
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java:
##########
@@ -222,147 +222,113 @@
@Component
public class ServiceDBStore extends AbstractServiceStore {
- private static final Logger LOG =
LoggerFactory.getLogger(ServiceDBStore.class);
-
- public static final String SERVICE_ADMIN_USERS =
"service.admin.users";
- public static final String SERVICE_ADMIN_GROUPS =
"service.admin.groups";
- public static final String GDS_SERVICE_NAME = "_gds";
- public static final String CRYPT_ALGO =
PropertiesUtil.getProperty("ranger.password.encryption.algorithm",
PasswordUtils.DEFAULT_CRYPT_ALGO);
- public static final String ENCRYPT_KEY =
PropertiesUtil.getProperty("ranger.password.encryption.key",
PasswordUtils.DEFAULT_ENCRYPT_KEY);
- public static final String SALT =
PropertiesUtil.getProperty("ranger.password.salt", PasswordUtils.DEFAULT_SALT);
- public static final Integer ITERATION_COUNT =
PropertiesUtil.getIntProperty("ranger.password.iteration.count",
PasswordUtils.DEFAULT_ITERATION_COUNT);
- public static final String RANGER_PLUGIN_AUDIT_FILTERS =
"ranger.plugin.audit.filters";
- public static final String HIDDEN_PASSWORD_STR = "*****";
- public static final String CONFIG_KEY_PASSWORD = "password";
- public static final String ACCESS_TYPE_DECRYPT_EEK = "decrypteek";
- public static final String ACCESS_TYPE_GENERATE_EEK = "generateeek";
- public static final String ACCESS_TYPE_GET_METADATA = "getmetadata";
-
- private static final String POLICY_ALLOW_EXCLUDE = "Policy
Allow:Exclude";
- private static final String POLICY_ALLOW_INCLUDE = "Policy
Allow:Include";
- private static final String POLICY_DENY_EXCLUDE = "Policy
Deny:Exclude";
- private static final String POLICY_DENY_INCLUDE = "Policy
Deny:Include";
- private static final String POLICY_TYPE_ACCESS = "Access";
- private static final String POLICY_TYPE_DATAMASK = "Masking";
- private static final String POLICY_TYPE_ROWFILTER = "Row Level
Filter";
- private static final String HOSTNAME = "Host name";
- private static final String USER_NAME = "Exported by";
- private static final String RANGER_VERSION = "Ranger apache
version";
- private static final String TIMESTAMP = "Export time";
- private static final String EXPORT_COUNT = "Exported count";
- private static final String SERVICE_CHECK_USER =
"service.check.user";
- private static final String AMBARI_SERVICE_CHECK_USER =
"ambari.service.check.user";
- private static final String RANGER_PLUGIN_CONFIG_PREFIX = "ranger.plugin.";
- private static final String LINE_SEPARATOR = "\n";
- private static final String FILE_HEADER =
"ID|Name|Resources|Roles|Groups|Users|Accesses|Service Type|Status|Policy
Type|Delegate Admin|isRecursive|isExcludes|Service
Name|Description|isAuditEnabled|Policy Conditions|Policy Condition Type|Masking
Options|Row Filter Expr|Policy Label Name";
- private static final String COMMA_DELIMITER = "|";
-
- private static final String DEFAULT_CSV_SANITIZATION_PATTERN =
"^[=+\\-@\\t\\r]";
- private static final Pattern CSV_SANITIZATION_PATTERN =
Pattern.compile(PropertiesUtil.getProperty("ranger.admin.csv.sanitization.pattern",
DEFAULT_CSV_SANITIZATION_PATTERN));
-
- private static final Comparator<RangerPolicyDelta>
POLICY_DELTA_ID_COMPARATOR = new RangerPolicyDeltaComparator();
-
- public static boolean SUPPORTS_POLICY_DELTAS;
- public static boolean SUPPORTS_IN_PLACE_POLICY_UPDATES;
- public static Integer RETENTION_PERIOD_IN_DAYS = 7;
- public static Integer TAG_RETENTION_PERIOD_IN_DAYS = 3;
- public static boolean SUPPORTS_PURGE_LOGIN_RECORDS;
- public static Integer LOGIN_RECORDS_RETENTION_PERIOD_IN_DAYS;
- public static boolean SUPPORTS_PURGE_TRANSACTION_RECORDS;
- public static Integer TRANSACTION_RECORDS_RETENTION_PERIOD_IN_DAYS;
- public static boolean SUPPORTS_PURGE_POLICY_EXPORT_LOGS;
- public static Integer POLICY_EXPORT_LOGS_RETENTION_PERIOD_IN_DAYS;
-
- private static String LOCAL_HOSTNAME;
- private static boolean isRolesDownloadedByService;
-
- private static volatile boolean legacyServiceDefsInitDone;
-
+ public static final String SERVICE_ADMIN_USERS
= "service.admin.users";
+ public static final String SERVICE_ADMIN_GROUPS
= "service.admin.groups";
+ public static final String GDS_SERVICE_NAME
= "_gds";
+ public static final String CRYPT_ALGO
=
PropertiesUtil.getProperty("ranger.password.encryption.algorithm",
PasswordUtils.DEFAULT_CRYPT_ALGO);
+ public static final String ENCRYPT_KEY
= PropertiesUtil.getProperty("ranger.password.encryption.key",
PasswordUtils.DEFAULT_ENCRYPT_KEY);
+ public static final String SALT
= PropertiesUtil.getProperty("ranger.password.salt",
PasswordUtils.DEFAULT_SALT);
+ public static final Integer ITERATION_COUNT
= PropertiesUtil.getIntProperty("ranger.password.iteration.count",
PasswordUtils.DEFAULT_ITERATION_COUNT);
+ public static final String
RANGER_PLUGIN_AUDIT_FILTERS = "ranger.plugin.audit.filters";
+ public static final String
RANGER_PLUGINS_CONFIG_CONF_PREFIX = "ranger.plugins.conf.";
+ public static final String HIDDEN_PASSWORD_STR
= "*****";
+ public static final String CONFIG_KEY_PASSWORD
= "password";
+ public static final String
ACCESS_TYPE_DECRYPT_EEK = "decrypteek";
+ public static final String
ACCESS_TYPE_GENERATE_EEK = "generateeek";
+ public static final String
ACCESS_TYPE_GET_METADATA = "getmetadata";
+ private static final Logger LOG
= LoggerFactory.getLogger(ServiceDBStore.class);
Review Comment:
@dhavalshah9131 - to be consistent with rest of the code base, I suggest
keeping instantiation of Logger instances at start of the class, even if they
are private static.
##########
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java:
##########
@@ -1818,6 +1785,10 @@ public ServicePolicies
getServicePoliciesIfUpdated(String serviceName, Long last
}
}
+ if (ret != null) {
Review Comment:
- replace `if (ret != null)` with inline check as shown below
- remove `<==` in the log message
```
LOG.debug("getServicePoliciesIfUpdated({}, {}, {}): configs = {}",
serviceName, lastKnownVersion, needsBackwardCompatibility, ret == null ? null ?
ret.getServiceConfig());
```
##########
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java:
##########
@@ -1274,6 +1275,76 @@ private RangerAdminClient getAdminClient() throws
Exception {
return admin;
}
+ private void
configurePluginContextFromServicePoliciesForUserGroupName(ServicePolicies
servicePolicies) {
+ if (servicePolicies != null) {
+ Map<String, String> serviceConfigMap =
servicePolicies.getServiceConfig();
+ if (MapUtils.isNotEmpty(serviceConfigMap)) {
+ LOG.debug("==> RangerBasePlugin({})",
serviceConfigMap.keySet());
+
pluginContext.setUserNameCaseConversion(serviceConfigMap.get(RangerCommonConstants.PLUGINS_CONF_USERNAME_CASE_CONVERSION_PARAM));
+
pluginContext.setGroupNameCaseConversion(serviceConfigMap.get(RangerCommonConstants.PLUGINS_CONF_GROUPNAME_CASE_CONVERSION_PARAM));
+ String mappingUserNameHandler =
serviceConfigMap.get(RangerCommonConstants.PLUGINS_CONF_MAPPING_USERNAME_HANDLER);
+ try {
+ if (mappingUserNameHandler != null) {
+ Class<Mapper> regExClass = (Class<Mapper>)
Class.forName(mappingUserNameHandler);
+ Mapper userNameRegExInst =
regExClass.newInstance();
+ if (userNameRegExInst != null) {
+ String baseProperty =
RangerCommonConstants.PLUGINS_CONF_MAPPING_USERNAME;
+ userNameRegExInst.init(baseProperty,
getAllRegexPatterns(baseProperty, serviceConfigMap),
+
serviceConfigMap.get(RangerCommonConstants.PLUGINS_CONF_MAPPING_SEPARATOR));
+
pluginContext.setUserNameTransformInst(userNameRegExInst);
+ } else {
+ LOG.error("RegEx handler instance for username is
null!");
+ }
+ }
+ } catch (ClassNotFoundException cne) {
+ LOG.error("Failed to load " + mappingUserNameHandler + "
", cne);
+ } catch (Throwable te) {
+ LOG.error("Failed to instantiate " +
mappingUserNameHandler + " ", te);
+ }
+ String mappingGroupNameHandler =
serviceConfigMap.get(RangerCommonConstants.PLUGINS_CONF_MAPPING_GROUPNAME_HANDLER);
+ try {
+ if (mappingGroupNameHandler != null) {
+ Class<Mapper> regExClass = (Class<Mapper>)
Class.forName(mappingGroupNameHandler);
+ Mapper groupNameRegExInst =
regExClass.newInstance();
+ if (groupNameRegExInst != null) {
+ String baseProperty =
RangerCommonConstants.PLUGINS_CONF_MAPPING_GROUPNAME;
+ groupNameRegExInst.init(baseProperty,
getAllRegexPatterns(baseProperty, serviceConfigMap),
+
serviceConfigMap.get(RangerCommonConstants.PLUGINS_CONF_MAPPING_SEPARATOR));
+
pluginContext.setGroupNameTransformInst(groupNameRegExInst);
+ } else {
+ LOG.error("RegEx handler instance for groupname is
null!");
+ }
+ }
+ } catch (ClassNotFoundException cne) {
+ LOG.error("Failed to load " + mappingGroupNameHandler + "
", cne);
+ } catch (Throwable te) {
+ LOG.error("Failed to instantiate " +
mappingGroupNameHandler + " ", te);
+ }
+ }
+ }
+ }
+
+ private List<String> getAllRegexPatterns(String baseProperty, Map<String,
String> serviceConfig) throws Throwable {
+ List<String> regexPatterns = new ArrayList<String>();
+ String baseRegex = serviceConfig.get(baseProperty);
+ LOG.debug("==> getAllRegexPatterns({})", baseProperty);
+ LOG.debug("baseRegex = {}", baseRegex);
+ LOG.debug("pluginConfig = {}", serviceConfig != null ?
serviceConfig.keySet() : "null");
+ if (baseRegex == null) {
+ return regexPatterns;
Review Comment:
I suggest avoiding multiple `return` statements in a method.
```
if (baseRegEx != null) {
...
}
LOG.debug("<== getAllRegexPatterns({})", regexPatterns);
return regexPatterns;
```
##########
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java:
##########
@@ -1842,6 +1813,10 @@ public ServicePolicies getServicePolicyDeltas(String
serviceName, Long lastKnown
ret = getServicePolicies(serviceName, lastKnownVersion, true,
SUPPORTS_POLICY_DELTAS, cachedPolicyVersion);
}
+ if (ret != null) {
Review Comment:
- replace if (ret != null) with inline check as shown below
- replace `<===` with `<==`
```
LOG.debug("<== ServiceDBStore.getServicePolicyDeltas({}, {}): ret = {}",
serviceName, lastKnownVersion, ret == null ? ret : ret.getServiceConfig());
```
##########
security-admin/src/main/java/org/apache/ranger/biz/ServiceDBStore.java:
##########
@@ -4112,6 +4099,10 @@ private ServicePolicies getServicePolicies(String
serviceName, Long lastKnownVer
ret.setTagPolicies(tagPolicies);
}
+ if (ret != null) {
Review Comment:
- replace `if (ret != null)` with inline check as shown below
- remove `<==` in the log message
```
LOG.debug("ServiceDBStore.getServicePolicies({}, {}): ret = {}",
serviceName, lastKnownVersion, ret == null ? null : ret.getServiceConfig());
```
##########
ugsync-util/src/main/java/org/apache/ranger/ugsyncutil/transform/RegEx.java:
##########
@@ -17,34 +17,27 @@
* under the License.
*/
-package org.apache.ranger.usergroupsync;
-
-import org.apache.ranger.unixusersync.config.UserGroupSyncConfig;
+package org.apache.ranger.ugsyncutil.transform;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
public class RegEx extends AbstractMapper {
- private final UserGroupSyncConfig config =
UserGroupSyncConfig.getInstance();
- private LinkedHashMap<String, String> replacementPattern;
+ private LinkedHashMap<String, String> replacementPattern;
public LinkedHashMap<String, String> getReplacementPattern() {
return replacementPattern;
}
@Override
- public void init(String baseProperty) {
- logger.info("Initializing for {}", baseProperty);
-
+ public void init(String baseProperty, List<String> regexPatterns, String
regexSeparator) {
+ AbstractMapper.logger.info("Initializing for {}", baseProperty);
Review Comment:
Why is it necessary to add class name qualifier when referencing static
member `logger`? Alternatively, consider adding a static import:
```
import static org.apache.ranger.ugsyncutil.transform.AbstractMapper.logger;
```
##########
agents-common/src/main/java/org/apache/ranger/plugin/service/RangerBasePlugin.java:
##########
@@ -1274,6 +1275,76 @@ private RangerAdminClient getAdminClient() throws
Exception {
return admin;
}
+ private void
configurePluginContextFromServicePoliciesForUserGroupName(ServicePolicies
servicePolicies) {
+ if (servicePolicies != null) {
+ Map<String, String> serviceConfigMap =
servicePolicies.getServiceConfig();
+ if (MapUtils.isNotEmpty(serviceConfigMap)) {
+ LOG.debug("==> RangerBasePlugin({})",
serviceConfigMap.keySet());
+
pluginContext.setUserNameCaseConversion(serviceConfigMap.get(RangerCommonConstants.PLUGINS_CONF_USERNAME_CASE_CONVERSION_PARAM));
+
pluginContext.setGroupNameCaseConversion(serviceConfigMap.get(RangerCommonConstants.PLUGINS_CONF_GROUPNAME_CASE_CONVERSION_PARAM));
+ String mappingUserNameHandler =
serviceConfigMap.get(RangerCommonConstants.PLUGINS_CONF_MAPPING_USERNAME_HANDLER);
+ try {
+ if (mappingUserNameHandler != null) {
+ Class<Mapper> regExClass = (Class<Mapper>)
Class.forName(mappingUserNameHandler);
+ Mapper userNameRegExInst =
regExClass.newInstance();
+ if (userNameRegExInst != null) {
+ String baseProperty =
RangerCommonConstants.PLUGINS_CONF_MAPPING_USERNAME;
+ userNameRegExInst.init(baseProperty,
getAllRegexPatterns(baseProperty, serviceConfigMap),
+
serviceConfigMap.get(RangerCommonConstants.PLUGINS_CONF_MAPPING_SEPARATOR));
+
pluginContext.setUserNameTransformInst(userNameRegExInst);
+ } else {
+ LOG.error("RegEx handler instance for username is
null!");
+ }
+ }
+ } catch (ClassNotFoundException cne) {
+ LOG.error("Failed to load " + mappingUserNameHandler + "
", cne);
+ } catch (Throwable te) {
+ LOG.error("Failed to instantiate " +
mappingUserNameHandler + " ", te);
+ }
+ String mappingGroupNameHandler =
serviceConfigMap.get(RangerCommonConstants.PLUGINS_CONF_MAPPING_GROUPNAME_HANDLER);
+ try {
+ if (mappingGroupNameHandler != null) {
+ Class<Mapper> regExClass = (Class<Mapper>)
Class.forName(mappingGroupNameHandler);
+ Mapper groupNameRegExInst =
regExClass.newInstance();
+ if (groupNameRegExInst != null) {
+ String baseProperty =
RangerCommonConstants.PLUGINS_CONF_MAPPING_GROUPNAME;
+ groupNameRegExInst.init(baseProperty,
getAllRegexPatterns(baseProperty, serviceConfigMap),
+
serviceConfigMap.get(RangerCommonConstants.PLUGINS_CONF_MAPPING_SEPARATOR));
+
pluginContext.setGroupNameTransformInst(groupNameRegExInst);
+ } else {
+ LOG.error("RegEx handler instance for groupname is
null!");
+ }
+ }
+ } catch (ClassNotFoundException cne) {
+ LOG.error("Failed to load " + mappingGroupNameHandler + "
", cne);
+ } catch (Throwable te) {
+ LOG.error("Failed to instantiate " +
mappingGroupNameHandler + " ", te);
+ }
+ }
+ }
+ }
+
+ private List<String> getAllRegexPatterns(String baseProperty, Map<String,
String> serviceConfig) throws Throwable {
+ List<String> regexPatterns = new ArrayList<String>();
+ String baseRegex = serviceConfig.get(baseProperty);
+ LOG.debug("==> getAllRegexPatterns({})", baseProperty);
+ LOG.debug("baseRegex = {}", baseRegex);
Review Comment:
Consider consoldating debug log messages in 1331 and 1332:
```
LOG.debug("baseRegex = {}, pluginConfig = {}", baseRegex, serviceConfig ==
null ? null : serviceConfig.keySet());
``
##########
security-admin/src/main/java/org/apache/ranger/common/RangerServicePoliciesCache.java:
##########
@@ -358,12 +358,13 @@ ServicePolicies getLatestOrCached(String serviceName,
ServiceStore serviceStore,
if (servicePoliciesForDeltas != null &&
servicePoliciesForDeltas.getPolicyDeltas() != null) {
LOG.debug("Deltas were requested. Returning
deltas from lastKnownVersion:[{}]", lastKnownVersion);
-
if (isDeltaCacheReinitialized) {
this.deltaCache = new
ServicePolicyDeltasCache(lastKnownVersion, servicePoliciesForDeltas);
}
+ LOG.debug("servicePoliciesForDeltas = " +
servicePoliciesForDeltas.getServiceConfig());
Review Comment:
replace string concat with parameterized log message.
```
LOG.debug("servicePoliciesForDeltas = {}",
servicePoliciesForDeltas.getServiceConfig());
```
##########
agents-common/src/main/java/org/apache/ranger/plugin/model/UgsyncNameTransformRules.java:
##########
@@ -0,0 +1,69 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ranger.plugin.model;
+
+import com.fasterxml.jackson.annotation.JsonAutoDetect;
+import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+import com.fasterxml.jackson.databind.annotation.JsonSerialize;
+import org.apache.ranger.plugin.util.RangerUserStoreUtil;
+
+import javax.xml.bind.annotation.XmlAccessType;
+import javax.xml.bind.annotation.XmlAccessorType;
+import javax.xml.bind.annotation.XmlRootElement;
+
+import java.util.Map;
+
+@JsonAutoDetect(fieldVisibility = JsonAutoDetect.Visibility.ANY)
+@JsonSerialize(include = JsonSerialize.Inclusion.NON_NULL)
+@JsonIgnoreProperties(ignoreUnknown = true)
+@XmlRootElement
+@XmlAccessorType(XmlAccessType.FIELD)
+public class UgsyncNameTransformRules extends RangerBaseModelObject implements
java.io.Serializable {
+ private static final long serialVersionUID = 1L;
+ private final String name;
Review Comment:
- are instances of `UgsyncNameTransformRules` serialized/deserialied in REST
API calls i.e. parameters or return values? If not, I suggest removing @Json
annotations.
- why is the member `name` final? If this is necessary, why not
`nameTransformRules` as well?
- Is the default constructor at line 43 necessary, if the members are final?
##########
security-admin/src/main/java/org/apache/ranger/rest/ServiceREST.java:
##########
@@ -194,7 +191,8 @@ public class ServiceREST {
public static final String PURGE_RECORD_TYPE_LOGIN_LOGS =
"login_records";
public static final String PURGE_RECORD_TYPE_TRX_LOGS =
"trx_records";
public static final String PURGE_RECORD_TYPE_POLICY_EXPORT_LOGS =
"policy_export_logs";
-
+ private static final Logger LOG =
LoggerFactory.getLogger(ServiceREST.class);
Review Comment:
To be consistent with rest of the sources, have initialization of `Logger`
instances at the beginning of the class.
##########
ugsync-util/src/test/java/org/apache/ranger/ugsynutil/transform/TestRegEx.java:
##########
@@ -49,21 +49,21 @@ public void setUp() {
public void testUserNameTransform() {
userRegexPatterns.add("s/\\s/_/");
userNameRegEx.populateReplacementPatterns(userNameBaseProperty,
userRegexPatterns, mappingSeparator);
- assertEquals("test_user", userNameRegEx.transform("test user"));
+ Assert.assertEquals("test_user", userNameRegEx.transform("test user"));
Review Comment:
Instead of qualifying each reference to a static method with the class name,
consider statically importing the methods, like:
```
import static org.junit.Assert.assertEquals;
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscr...@ranger.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
