Thanks, Madhan! On 3/13/15, 7:50 PM, "[email protected]" <[email protected]> wrote:
>Repository: incubator-ranger >Updated Branches: > refs/heads/master d4f2eb20f -> 2e0be82df > > >RANGER-307: Policy evaluation optimization: reorder policies and >short-circuit evaluation > > >Project: http://git-wip-us.apache.org/repos/asf/incubator-ranger/repo >Commit: >http://git-wip-us.apache.org/repos/asf/incubator-ranger/commit/2e0be82d >Tree: >http://git-wip-us.apache.org/repos/asf/incubator-ranger/tree/2e0be82d >Diff: >http://git-wip-us.apache.org/repos/asf/incubator-ranger/diff/2e0be82d > >Branch: refs/heads/master >Commit: 2e0be82df973a6f37e766b7e9fff2b41431d0db1 >Parents: d4f2eb2 >Author: Abhay Kulkarni <[email protected]> >Authored: Fri Mar 13 19:50:19 2015 -0700 >Committer: Madhan Neethiraj <[email protected]> >Committed: Fri Mar 13 19:50:19 2015 -0700 > >---------------------------------------------------------------------- > .../ranger/plugin/policyengine/CacheMap.java | 40 +++ > .../plugin/policyengine/RangerAccessData.java | 39 +++ > .../plugin/policyengine/RangerAccessResult.java | 32 ++- > .../policyengine/RangerPolicyEngineImpl.java | 149 ++--------- > .../RangerPolicyEvaluatorFacade.java | 128 ++++++++++ > .../policyengine/RangerPolicyRepository.java | 231 +++++++++++++++++ > .../RangerDefaultPolicyEvaluator.java | 223 ++++++++++------- > .../RangerOptimizedPolicyEvaluator.java | 247 +++++++++++++++++++ > .../RangerDefaultPolicyEvaluatorTest.java | 8 +- > .../policyengine/test_policyengine_hdfs.json | 2 +- > pom.xml | 2 +- > src/main/assembly/ranger-src.xml | 83 ++++--- > 12 files changed, 913 insertions(+), 271 deletions(-) >---------------------------------------------------------------------- > > >http://git-wip-us.apache.org/repos/asf/incubator-ranger/blob/2e0be82d/agen >t >s-common/src/main/java/org/apache/ranger/plugin/policyengine/CacheMap.java >---------------------------------------------------------------------- >diff --git >a/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/CacheM >ap >.java >b/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/CacheM >ap >.java >new file mode 100644 >index 0000000..0a1566c >--- /dev/null >+++ >b/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/CacheM >ap >.java >@@ -0,0 +1,40 @@ >+/* >+ * Licensed to the Apache Software Foundation (ASF) under one >+ * or more contributor license agreements. See the NOTICE file >+ * distributed with this work for additional information >+ * regarding copyright ownership. The ASF licenses this file >+ * to you under the Apache License, Version 2.0 (the >+ * "License"); you may not use this file except in compliance >+ * with the License. You may obtain a copy of the License at >+ * >+ * http://www.apache.org/licenses/LICENSE-2.0 >+ * >+ * Unless required by applicable law or agreed to in writing, >+ * software distributed under the License is distributed on an >+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY >+ * KIND, either express or implied. See the License for the >+ * specific language governing permissions and limitations >+ * under the License. >+ */ >+package org.apache.ranger.plugin.policyengine; >+ >+import java.util.LinkedHashMap; >+import java.util.Map; >+ >+ >+public class CacheMap<K, V> extends LinkedHashMap<K, V> { >+ private static final float RANGER_CACHE_DEFAULT_LOAD_FACTOR = 0.75f; >+ >+ protected int maxCapacity; >+ >+ public CacheMap(int maxCapacity) { >+ super(maxCapacity, CacheMap.RANGER_CACHE_DEFAULT_LOAD_FACTOR, >true); // true for access-order >+ >+ this.maxCapacity = maxCapacity; >+ } >+ >+ @Override >+ protected boolean removeEldestEntry(Map.Entry eldest) { >+ return size() > maxCapacity; >+ } >+} > >http://git-wip-us.apache.org/repos/asf/incubator-ranger/blob/2e0be82d/agen >t >s-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessD >at >a.java >---------------------------------------------------------------------- >diff --git >a/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/Ranger >Ac >cessData.java >b/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/Ranger >Ac >cessData.java >new file mode 100644 >index 0000000..34f7428 >--- /dev/null >+++ >b/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/Ranger >Ac >cessData.java >@@ -0,0 +1,39 @@ >+/* >+ * Licensed to the Apache Software Foundation (ASF) under one >+ * or more contributor license agreements. See the NOTICE file >+ * distributed with this work for additional information >+ * regarding copyright ownership. The ASF licenses this file >+ * to you under the Apache License, Version 2.0 (the >+ * "License"); you may not use this file except in compliance >+ * with the License. You may obtain a copy of the License at >+ * >+ * http://www.apache.org/licenses/LICENSE-2.0 >+ * >+ * Unless required by applicable law or agreed to in writing, >+ * software distributed under the License is distributed on an >+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY >+ * KIND, either express or implied. See the License for the >+ * specific language governing permissions and limitations >+ * under the License. >+ */ >+ >+package org.apache.ranger.plugin.policyengine; >+ >+ >+public class RangerAccessData<T> { >+ private String accessFDN = null; >+ private T accessDetails = null; >+ >+ public RangerAccessData(String accessFDN) { >+ this.accessFDN = accessFDN; >+ } >+ public String getAccessFDN() { >+ return accessFDN; >+ } >+ public T getAccessDetails() { >+ return accessDetails; >+ } >+ public void setAccessDetails(T accessDetails) { >+ this.accessDetails = accessDetails; >+ } >+} > >http://git-wip-us.apache.org/repos/asf/incubator-ranger/blob/2e0be82d/agen >t >s-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerAccessR >es >ult.java >---------------------------------------------------------------------- >diff --git >a/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/Ranger >Ac >cessResult.java >b/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/Ranger >Ac >cessResult.java >index 2eaec16..3a1fa1d 100644 >--- >a/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/Ranger >Ac >cessResult.java >+++ >b/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/Ranger >Ac >cessResult.java >@@ -27,23 +27,23 @@ public class RangerAccessResult { > private RangerServiceDef serviceDef = null; > private RangerAccessRequest request = null; > >+ private boolean isAccessDetermined = false; > private boolean isAllowed = false; >+ private boolean isAuditedDetermined = false; > private boolean isAudited = false; > private long policyId = -1; > private String reason = null; > > public RangerAccessResult(String serviceName, RangerServiceDef >serviceDef, RangerAccessRequest request) { >- this(serviceName, serviceDef, request, false, false, -1, null); >- } >- >- public RangerAccessResult(String serviceName, RangerServiceDef >serviceDef, RangerAccessRequest request, boolean isAllowed, boolean >isAudited, long policyId, String reason) { > this.serviceName = serviceName; > this.serviceDef = serviceDef; > this.request = request; >- this.isAllowed = isAllowed; >- this.isAudited = isAudited; >- this.policyId = policyId; >- this.reason = reason; >+ this.isAccessDetermined = false; >+ this.isAllowed = false; >+ this.isAuditedDetermined = false; >+ this.isAudited = false; >+ this.policyId = -1; >+ this.reason = null; > } > > /** >@@ -67,6 +67,10 @@ public class RangerAccessResult { > return request; > } > >+ public boolean getIsAccessDetermined() { return isAccessDetermined; } >+ >+ private void setIsAccessDetermined(boolean value) { isAccessDetermined = >value; } >+ > /** > * @return the isAllowed > */ >@@ -78,6 +82,7 @@ public class RangerAccessResult { > * @param isAllowed the isAllowed to set > */ > public void setIsAllowed(boolean isAllowed) { >+ setIsAccessDetermined(true); > this.isAllowed = isAllowed; > } > >@@ -87,6 +92,10 @@ public class RangerAccessResult { > public void setReason(String reason) { > this.reason = reason; > } >+ >+ public boolean getIsAuditedDetermined() { return isAuditedDetermined; } >+ >+ private void setIsAuditedDetermined(boolean value) { isAuditedDetermined >= value; } > > /** > * @return the isAudited >@@ -95,10 +104,13 @@ public class RangerAccessResult { > return isAudited; > } > >+ >+ > /** > * @param isAudited the isAudited to set > */ > public void setIsAudited(boolean isAudited) { >+ setIsAuditedDetermined(true); > this.isAudited = isAudited; > } > >@@ -145,8 +157,10 @@ public class RangerAccessResult { > public StringBuilder toString(StringBuilder sb) { > sb.append("RangerAccessResult={"); > >+ >sb.append("isAccessDetermined={").append(isAccessDetermined).append("} "); > sb.append("isAllowed={").append(isAllowed).append("} "); >- sb.append("isAudited={").append(isAudited).append("} "); >+ >sb.append("isAuditedDetermined={").append(isAuditedDetermined).append("} >"); >+ sb.append("isAudited={").append(isAudited).append("} "); > sb.append("policyId={").append(policyId).append("} "); > sb.append("reason={").append(reason).append("} "); > > >http://git-wip-us.apache.org/repos/asf/incubator-ranger/blob/2e0be82d/agen >t >s-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyE >ng >ineImpl.java >---------------------------------------------------------------------- >diff --git >a/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/Ranger >Po >licyEngineImpl.java >b/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/Ranger >Po >licyEngineImpl.java >index 51787ac..abae1fe 100644 >--- >a/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/Ranger >Po >licyEngineImpl.java >+++ >b/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/Ranger >Po >licyEngineImpl.java >@@ -19,31 +19,25 @@ > > package org.apache.ranger.plugin.policyengine; > >-import java.util.ArrayList; >-import java.util.Collection; >-import java.util.List; >- >-import org.apache.commons.collections.CollectionUtils; >-import org.apache.commons.lang.StringUtils; > import org.apache.commons.logging.Log; > import org.apache.commons.logging.LogFactory; > import org.apache.ranger.plugin.audit.RangerAuditHandler; > import org.apache.ranger.plugin.contextenricher.RangerContextEnricher; > import org.apache.ranger.plugin.model.RangerPolicy; > import org.apache.ranger.plugin.model.RangerServiceDef; >-import >org.apache.ranger.plugin.model.RangerServiceDef.RangerContextEnricherDef; >-import >org.apache.ranger.plugin.policyevaluator.RangerDefaultPolicyEvaluator; > import org.apache.ranger.plugin.policyevaluator.RangerPolicyEvaluator; > >+import java.util.ArrayList; >+import java.util.Collection; >+import java.util.List; >+ > > public class RangerPolicyEngineImpl implements RangerPolicyEngine { > private static final Log LOG = >LogFactory.getLog(RangerPolicyEngineImpl.class); > >- private String serviceName = null; >- private RangerServiceDef serviceDef = null; >- private List<RangerContextEnricher> contextEnrichers = null; >- private List<RangerPolicyEvaluator> policyEvaluators = null; >- private RangerAuditHandler defaultAuditHandler = null; >+ private String serviceName = null; >+ private RangerPolicyRepository policyRepository = null; >+ private RangerAuditHandler defaultAuditHandler = null; > > > public RangerPolicyEngineImpl() { >@@ -63,12 +57,13 @@ public class RangerPolicyEngineImpl implements >RangerPolicyEngine { > > @Override > public RangerServiceDef getServiceDef() { >- return serviceDef; >+ return policyRepository == null ? null : >policyRepository.getServiceDef(); > } > > @Override > public List<RangerContextEnricher> getContextEnrichers() { >- return contextEnrichers; >+ >+ return policyRepository == null ? null : getContextEnrichers(); > } > > @Override >@@ -77,51 +72,12 @@ public class RangerPolicyEngineImpl implements >RangerPolicyEngine { > LOG.debug("==> RangerPolicyEngineImpl.setPolicies(" + > serviceName + ", >" + serviceDef + ", policies.count=" + (policies == null ? 0 : >policies.size()) + ")"); > } > >- if(serviceName != null && serviceDef != null && policies != >null) { >- List<RangerContextEnricher> contextEnrichers = new >ArrayList<RangerContextEnricher>(); >- >- >if(!CollectionUtils.isEmpty(serviceDef.getContextEnrichers())) { >- for(RangerContextEnricherDef enricherDef : >serviceDef.getContextEnrichers()) { >- if(enricherDef == null) { >- continue; >- } >- >- RangerContextEnricher contextEnricher = >getContextEnricher(enricherDef); >- >- contextEnrichers.add(contextEnricher); >- } >- } >- >- List<RangerPolicyEvaluator> evaluators = new >ArrayList<RangerPolicyEvaluator>(); >- >- for(RangerPolicy policy : policies) { >- if(! policy.getIsEnabled()) { >- continue; >- } >- >- RangerPolicyEvaluator evaluator = >getPolicyEvaluator(policy, >serviceDef); >- >- if(evaluator != null) { >- evaluators.add(evaluator); >- } >- } >- >- /* TODO: >- * sort evaluators list for faster completion of >isAccessAllowed() >method >- * 1. Global policies: the policies that cover for >any resource (for >example: database=*; table=*; column=*) >- * 2. Policies that cover all resources under level-1 >(for example: >every thing in one or more databases) >- * 3. Policies that cover all resources under level-2 >(for example: >every thing in one or more tables) >- * ... >- * 4. Policies that cover all resources under level-n >(for example: >one or more columns) >- * >- */ >- >- this.serviceName = serviceName; >- this.serviceDef = serviceDef; >- this.contextEnrichers = contextEnrichers; >- this.policyEvaluators = evaluators; >+ if (serviceName != null && serviceDef != null && policies != >null) { >+ policyRepository = new >RangerPolicyRepository(serviceName); >+ policyRepository.init(serviceDef, policies); >+ this.serviceName = serviceName; > } else { >- LOG.error("RangerPolicyEngineImpl.setPolicies(): >invalid arguments - >null serviceDef/policies"); >+ LOG.error("RangerPolicyEngineImpl.setPolicies ->Invalid >arguments: >serviceName, serviceDef, or policies is null"); > } > > if(LOG.isDebugEnabled()) { >@@ -141,7 +97,7 @@ public class RangerPolicyEngineImpl implements >RangerPolicyEngine { > > @Override > public RangerAccessResult createAccessResult(RangerAccessRequest >request) >{ >- return new RangerAccessResult(serviceName, serviceDef, >request); >+ return policyRepository == null ? null : new >RangerAccessResult(serviceName, policyRepository.getServiceDef(), >request); > } > > @Override >@@ -207,73 +163,26 @@ public class RangerPolicyEngineImpl implements >RangerPolicyEngine { > > RangerAccessResult ret = createAccessResult(request); > >- if(request != null) { >- List<RangerPolicyEvaluator> evaluators = >policyEvaluators; >+ if(policyRepository != null && ret != null && request != null) { >+ List<RangerPolicyEvaluatorFacade> evaluators = >policyRepository.getPolicyEvaluators(); > > if(evaluators != null) { >+ policyRepository.retrieveAuditEnabled(request, >ret); > for(RangerPolicyEvaluator evaluator : > evaluators) { > evaluator.evaluate(request, ret); > >- // stop once allowed=true && >audited==true >- if(ret.getIsAllowed() && >ret.getIsAudited()) { >+ // stop once allowed==true && >auditedDetermined==true >+ if(ret.getIsAccessDetermined() && >ret.getIsAuditedDetermined()) { > break; > } > } >- } >- } >+ policyRepository.storeAuditEnabled(request, >ret); > >- if(LOG.isDebugEnabled()) { >- LOG.debug("<== >RangerPolicyEngineImpl.isAccessAllowedNoAudit(" + >request + "): " + ret); >- } >- >- return ret; >- } >- >- private RangerContextEnricher >getContextEnricher(RangerContextEnricherDef >enricherDef) { >- if(LOG.isDebugEnabled()) { >- LOG.debug("==> >RangerPolicyEngineImpl.getContextEnricher(" + >enricherDef + ")"); >- } >- >- RangerContextEnricher ret = null; >- >- String name = enricherDef != null ? enricherDef.getName() > : >null; >- String clsName = enricherDef != null ? >enricherDef.getEnricher() : >null; >- >- if(! StringUtils.isEmpty(clsName)) { >- try { >- @SuppressWarnings("unchecked") >- Class<RangerContextEnricher> enricherClass = >(Class<RangerContextEnricher>)Class.forName(clsName); >- >- ret = enricherClass.newInstance(); >- } catch(Exception excp) { >- LOG.error("failed to instantiate context >enricher '" + clsName + "' >for '" + name + "'", excp); > } > } > >- if(ret != null) { >- ret.init(enricherDef); >- } >- > if(LOG.isDebugEnabled()) { >- LOG.debug("<== >RangerPolicyEngineImpl.getContextEnricher(" + >enricherDef + "): " + ret); >- } >- >- return ret; >- } >- >- private RangerPolicyEvaluator getPolicyEvaluator(RangerPolicy policy, >RangerServiceDef serviceDef) { >- if(LOG.isDebugEnabled()) { >- LOG.debug("==> >RangerPolicyEngineImpl.getPolicyEvaluator(" + policy + >"," + serviceDef + ")"); >- } >- >- RangerPolicyEvaluator ret = null; >- >- ret = new RangerDefaultPolicyEvaluator(); // TODO: configurable >evaluator class? >- >- ret.init(policy, serviceDef); >- >- if(LOG.isDebugEnabled()) { >- LOG.debug("<== >RangerPolicyEngineImpl.getPolicyEvaluator(" + policy + >"," + serviceDef + "): " + ret); >+ LOG.debug("<== >RangerPolicyEngineImpl.isAccessAllowedNoAudit(" + >request + "): " + ret); > } > > return ret; >@@ -292,17 +201,7 @@ public class RangerPolicyEngineImpl implements >RangerPolicyEngine { > sb.append("RangerPolicyEngineImpl={"); > > sb.append("serviceName={").append(serviceName).append("} "); >- sb.append("serviceDef={").append(serviceDef).append("} "); >- >- sb.append("policyEvaluators={"); >- if(policyEvaluators != null) { >- for(RangerPolicyEvaluator policyEvaluator : >policyEvaluators) { >- if(policyEvaluator != null) { >- sb.append(policyEvaluator).append(" "); >- } >- } >- } >- sb.append("} "); >+ sb.append(policyRepository); > > sb.append("}"); > > >http://git-wip-us.apache.org/repos/asf/incubator-ranger/blob/2e0be82d/agen >t >s-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyE >va >luatorFacade.java >---------------------------------------------------------------------- >diff --git >a/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/Ranger >Po >licyEvaluatorFacade.java >b/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/Ranger >Po >licyEvaluatorFacade.java >new file mode 100644 >index 0000000..5c30df3 >--- /dev/null >+++ >b/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/Ranger >Po >licyEvaluatorFacade.java >@@ -0,0 +1,128 @@ >+/* >+ * Licensed to the Apache Software Foundation (ASF) under one >+ * or more contributor license agreements. See the NOTICE file >+ * distributed with this work for additional information >+ * regarding copyright ownership. The ASF licenses this file >+ * to you under the Apache License, Version 2.0 (the >+ * "License"); you may not use this file except in compliance >+ * with the License. You may obtain a copy of the License at >+ * >+ * http://www.apache.org/licenses/LICENSE-2.0 >+ * >+ * Unless required by applicable law or agreed to in writing, >+ * software distributed under the License is distributed on an >+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY >+ * KIND, either express or implied. See the License for the >+ * specific language governing permissions and limitations >+ * under the License. >+ */ >+ >+package org.apache.ranger.plugin.policyengine; >+ >+import org.apache.commons.logging.Log; >+import org.apache.commons.logging.LogFactory; >+import >org.apache.ranger.plugin.conditionevaluator.RangerConditionEvaluator; >+import org.apache.ranger.plugin.model.RangerPolicy; >+import org.apache.ranger.plugin.model.RangerServiceDef; >+import >org.apache.ranger.plugin.policyevaluator.RangerDefaultPolicyEvaluator; >+import >org.apache.ranger.plugin.policyevaluator.RangerOptimizedPolicyEvaluator; >+import org.apache.ranger.plugin.policyevaluator.RangerPolicyEvaluator; >+ >+import java.util.Map; >+ >+public class RangerPolicyEvaluatorFacade implements >RangerPolicyEvaluator, >Comparable<RangerPolicyEvaluatorFacade> { >+ private static final Log LOG = >LogFactory.getLog(RangerPolicyEvaluatorFacade.class); >+ >+ RangerDefaultPolicyEvaluator delegate = null; >+ int computedPriority = 0; >+ boolean useCachePolicyEngine = false; >+ >+ RangerPolicyEvaluatorFacade(boolean useCachePolicyEngine) { >+ super(); >+ this.useCachePolicyEngine = useCachePolicyEngine; >+ delegate = new RangerOptimizedPolicyEvaluator(); >+ } >+ >+ RangerPolicyEvaluator getPolicyEvaluator() { >+ return delegate; >+ } >+ >+ @Override >+ public void init(RangerPolicy policy, RangerServiceDef serviceDef) { >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("==> RangerPolicyEvaluatorFacade.init(), >useCachePolicyEngine:" + useCachePolicyEngine); >+ } >+ delegate.init(policy, serviceDef); >+ computedPriority = computePriority(); >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("<== RangerPolicyEvaluatorFacade.init(), >useCachePolicyEngine:" + useCachePolicyEngine); >+ } >+ } >+ >+ @Override >+ public RangerPolicy getPolicy() { >+ return delegate.getPolicy(); >+ } >+ >+ @Override >+ public RangerServiceDef getServiceDef() { >+ return delegate.getServiceDef(); >+ } >+ >+ @Override >+ public void evaluate(RangerAccessRequest request, RangerAccessResult >result) { >+ delegate.evaluate(request, result); >+ } >+ >+ @Override >+ public boolean isMatch(RangerResource resource) { >+ return false; >+ } >+ >+ @Override >+ public boolean isSingleAndExactMatch(RangerResource resource) { >+ return false; >+ } >+ >+ @Override >+ public int compareTo(RangerPolicyEvaluatorFacade other) { >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("==> RangerPolicyEvaluatorFacade.compareTo()"); >+ } >+ int result; >+ >+ if (this.getComputedPriority() == other.getComputedPriority()) { >+ Map<String, RangerConditionEvaluator> myConditionEvaluators = >this.delegate.getConditionEvaluators(); >+ Map<String, RangerConditionEvaluator> >otherConditionEvaluators >= other.delegate.getConditionEvaluators(); >+ >+ int myConditionEvaluatorCount = myConditionEvaluators == null >? 0 : myConditionEvaluators.size(); >+ int otherConditionEvaluatorCount = otherConditionEvaluators >== >null ? 0 : otherConditionEvaluators.size(); >+ >+ result = Integer.compare(myConditionEvaluatorCount, >otherConditionEvaluatorCount); >+ } else { >+ int myComputedPriority = this.getComputedPriority(); >+ int otherComputedPriority = other.getComputedPriority(); >+ result = Integer.compare(myComputedPriority, >otherComputedPriority); >+ } >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("<== RangerPolicyEvaluatorFacade.compareTo(), >result:" + result); >+ } >+ >+ return result; >+ } >+ >+ private int getComputedPriority() { >+ return computedPriority; >+ } >+ >+ private int computePriority() { >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("==> >RangerPolicyEvaluatorFacade.computePriority()"); >+ } >+ int result = delegate.computePriority(); >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("<==RangerPolicyEvaluatorFacade.computePriority(), >result:" + result); >+ } >+ return result; >+ } >+} > >http://git-wip-us.apache.org/repos/asf/incubator-ranger/blob/2e0be82d/agen >t >s-common/src/main/java/org/apache/ranger/plugin/policyengine/RangerPolicyR >ep >ository.java >---------------------------------------------------------------------- >diff --git >a/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/Ranger >Po >licyRepository.java >b/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/Ranger >Po >licyRepository.java >new file mode 100644 >index 0000000..700b3ad >--- /dev/null >+++ >b/agents-common/src/main/java/org/apache/ranger/plugin/policyengine/Ranger >Po >licyRepository.java >@@ -0,0 +1,231 @@ >+/* >+ * Licensed to the Apache Software Foundation (ASF) under one >+ * or more contributor license agreements. See the NOTICE file >+ * distributed with this work for additional information >+ * regarding copyright ownership. The ASF licenses this file >+ * to you under the Apache License, Version 2.0 (the >+ * "License"); you may not use this file except in compliance >+ * with the License. You may obtain a copy of the License at >+ * >+ * http://www.apache.org/licenses/LICENSE-2.0 >+ * >+ * Unless required by applicable law or agreed to in writing, >+ * software distributed under the License is distributed on an >+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY >+ * KIND, either express or implied. See the License for the >+ * specific language governing permissions and limitations >+ * under the License. >+ */ >+ >+package org.apache.ranger.plugin.policyengine; >+ >+import org.apache.commons.collections.CollectionUtils; >+import org.apache.commons.lang.StringUtils; >+import org.apache.commons.logging.Log; >+import org.apache.commons.logging.LogFactory; >+import org.apache.ranger.authorization.hadoop.config.RangerConfiguration; >+import org.apache.ranger.plugin.contextenricher.RangerContextEnricher; >+import org.apache.ranger.plugin.model.RangerPolicy; >+import org.apache.ranger.plugin.model.RangerServiceDef; >+import org.apache.ranger.plugin.policyevaluator.RangerPolicyEvaluator; >+ >+import java.util.ArrayList; >+import java.util.Collections; >+import java.util.List; >+import java.util.Map; >+ >+public class RangerPolicyRepository { >+ private static final Log LOG = >LogFactory.getLog(RangerPolicyRepository.class); >+ >+ private String serviceName = null; >+ private List<RangerPolicyEvaluatorFacade> policyEvaluators = null; >+ private List<RangerContextEnricher> contextEnrichers = null; >+ private RangerServiceDef serviceDef = null; >+ // Not used at this time >+ private boolean useCachePolicyEngine = >false; >+ private Map<String, RangerAccessData<Boolean>> accessAuditCache = >null; >+ >+ private static int RANGER_POLICYENGINE_AUDITRESULT_CACHE_SIZE = >64*1024; >+ >+ RangerPolicyRepository(String serviceName) { >+ super(); >+ this.serviceName = serviceName; >+ } >+ String getRepositoryName() { >+ return serviceName; >+ } >+ List<RangerPolicyEvaluatorFacade> getPolicyEvaluators() { >+ return policyEvaluators; >+ } >+ List<RangerContextEnricher> getContextEnrichers() { >+ return contextEnrichers; >+ } >+ RangerServiceDef getServiceDef() { >+ return serviceDef; >+ } >+ >+ void init(RangerServiceDef serviceDef, List<RangerPolicy> policies) { >+ >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("==> RangerPolicyRepository.init(" + serviceDef + >", >policies.count=" + (policies == null ? 0 : policies.size()) + ")"); >+ } >+ >+ this.serviceDef = serviceDef; >+ >+ contextEnrichers = new ArrayList<RangerContextEnricher>(); >+ >+ if (!CollectionUtils.isEmpty(serviceDef.getContextEnrichers())) { >+ for (RangerServiceDef.RangerContextEnricherDef enricherDef : >serviceDef.getContextEnrichers()) { >+ if (enricherDef == null) { >+ continue; >+ } >+ >+ RangerContextEnricher contextEnricher = >buildContextEnricher(enricherDef); >+ >+ contextEnrichers.add(contextEnricher); >+ } >+ } >+ >+ policyEvaluators = new ArrayList<RangerPolicyEvaluatorFacade>(); >+ >+ for (RangerPolicy policy : policies) { >+ if (!policy.getIsEnabled()) { >+ continue; >+ } >+ >+ RangerPolicyEvaluatorFacade evaluator = >buildPolicyEvaluator(policy, serviceDef); >+ >+ if (evaluator != null) { >+ policyEvaluators.add(evaluator); >+ } >+ >+ Collections.sort(policyEvaluators); >+ >+ } >+ >+ String propertyName = "ranger.plugin." + serviceName + >".policyengine.auditcachesize"; >+ >+ int auditResultCacheSize = >RangerConfiguration.getInstance().getInt(propertyName, >RANGER_POLICYENGINE_AUDITRESULT_CACHE_SIZE); >+ >+ accessAuditCache = new CacheMap<String, >RangerAccessData<Boolean>>(auditResultCacheSize); >+ >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("<== RangerPolicyRepository.init(" + serviceDef + >", >policies.count=" + (policies == null ? 0 : policies.size()) + ")"); >+ } >+ } >+ >+ private RangerContextEnricher >buildContextEnricher(RangerServiceDef.RangerContextEnricherDef >enricherDef) >{ >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("==> RangerPolicyRepository.buildContextEnricher(" >+ >enricherDef + ")"); >+ } >+ >+ RangerContextEnricher ret = null; >+ >+ String name = enricherDef != null ? enricherDef.getName() >: >null; >+ String clsName = enricherDef != null ? enricherDef.getEnricher() >: >null; >+ >+ if(! StringUtils.isEmpty(clsName)) { >+ try { >+ @SuppressWarnings("unchecked") >+ Class<RangerContextEnricher> enricherClass = >(Class<RangerContextEnricher>)Class.forName(clsName); >+ >+ ret = enricherClass.newInstance(); >+ } catch(Exception excp) { >+ LOG.error("failed to instantiate context enricher '" + >clsName + "' for '" + name + "'", excp); >+ } >+ } >+ >+ if(ret != null) { >+ ret.init(enricherDef); >+ } >+ >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("<== RangerPolicyRepository.buildContextEnricher(" >+ >enricherDef + "): " + ret); >+ } >+ return ret; >+ } >+ >+ private RangerPolicyEvaluatorFacade buildPolicyEvaluator(RangerPolicy >policy, RangerServiceDef serviceDef) { >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("==> RangerPolicyRepository.buildPolicyEvaluator(" >+ >policy + "," + serviceDef + ")"); >+ } >+ >+ RangerPolicyEvaluatorFacade ret = null; >+ >+ ret = new RangerPolicyEvaluatorFacade(useCachePolicyEngine); >+ ret.init(policy, serviceDef); >+ >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("<== RangerPolicyRepository.buildPolicyEvaluator(" >+ >policy + "," + serviceDef + "): " + ret); >+ } >+ return ret; >+ } >+ >+ synchronized void retrieveAuditEnabled(RangerAccessRequest request, >RangerAccessResult ret) { >+ if (LOG.isDebugEnabled()) { >+ LOG.debug("==> >RangerPolicyRepository.retrieveAuditEnabled()"); >+ } >+ RangerAccessData<Boolean> value = >accessAuditCache.get(request.getResource().toString()); >+ if ((value != null)) { >+ ret.setIsAudited(value.getAccessDetails()); >+ } >+ >+ if (LOG.isDebugEnabled()) { >+ LOG.debug("<== >RangerPolicyRepository.retrieveAuditEnabled()"); >+ } >+ } >+ >+ synchronized void storeAuditEnabled(RangerAccessRequest request, >RangerAccessResult ret) { >+ if (LOG.isDebugEnabled()) { >+ LOG.debug("==> RangerPolicyRepository.storeAuditEnabled()"); >+ } >+ RangerAccessData<Boolean> lookup = >accessAuditCache.get(request.getResource().toString()); >+ if ((lookup == null && ret.getIsAuditedDetermined() == true)) { >+ RangerAccessData<Boolean> value = new >RangerAccessData<Boolean>(request.toString()); >+ value.setAccessDetails(ret.getIsAudited()); >+ accessAuditCache.put(request.getResource().toString(), >value); >+ } >+ >+ if (LOG.isDebugEnabled()) { >+ LOG.debug("<== RangerPolicyRepository.storeAuditEnabled()"); >+ } >+ } >+ >+ @Override >+ public String toString( ) { >+ StringBuilder sb = new StringBuilder(); >+ >+ toString(sb); >+ >+ return sb.toString(); >+ } >+ >+ public StringBuilder toString(StringBuilder sb) { >+ >+ sb.append("RangerPolicyRepository={"); >+ >+ sb.append("serviceName={").append(serviceName).append("} "); >+ sb.append("serviceDef={").append(serviceDef).append("} "); >+ sb.append("policyEvaluators={"); >+ if (policyEvaluators != null) { >+ for (RangerPolicyEvaluator policyEvaluator : >policyEvaluators) >{ >+ if (policyEvaluator != null) { >+ sb.append(policyEvaluator).append(" "); >+ } >+ } >+ } >+ if (contextEnrichers != null) { >+ for (RangerContextEnricher contextEnricher : >contextEnrichers) >{ >+ if (contextEnricher != null) { >+ sb.append(contextEnricher).append(" "); >+ } >+ } >+ } >+ >sb.append("useCachePolicyEngine={").append(useCachePolicyEngine).append("} > >"); >+ >+ sb.append("} "); >+ >+ return sb; >+ } >+ >+} > >http://git-wip-us.apache.org/repos/asf/incubator-ranger/blob/2e0be82d/agen >t >s-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerDefa >ul >tPolicyEvaluator.java >---------------------------------------------------------------------- >diff --git >a/agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/Ran >ge >rDefaultPolicyEvaluator.java >b/agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/Ran >ge >rDefaultPolicyEvaluator.java >index 0ac5eed..30b36fa 100644 >--- >a/agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/Ran >ge >rDefaultPolicyEvaluator.java >+++ >b/agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/Ran >ge >rDefaultPolicyEvaluator.java >@@ -19,13 +19,7 @@ > > package org.apache.ranger.plugin.policyevaluator; > >-import java.util.Collection; >-import java.util.Collections; >-import java.util.HashMap; >-import java.util.HashSet; >-import java.util.Iterator; >-import java.util.Map; >- >+import com.google.common.base.Strings; > import org.apache.commons.collections.CollectionUtils; > import org.apache.commons.lang.StringUtils; > import org.apache.commons.logging.Log; >@@ -47,7 +41,7 @@ import >org.apache.ranger.plugin.policyengine.RangerResource; > import >org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher; > import org.apache.ranger.plugin.resourcematcher.RangerResourceMatcher; > >-import com.google.common.base.Strings; >+import java.util.*; > > > public class RangerDefaultPolicyEvaluator extends >RangerAbstractPolicyEvaluator { >@@ -92,7 +86,13 @@ public class RangerDefaultPolicyEvaluator extends >RangerAbstractPolicyEvaluator > } > } > >- /** >+ public Map<String, RangerConditionEvaluator> >getConditionEvaluators() >{ >+ return conditionEvaluators; >+ } >+ public int computePriority() { return 0;} >+ >+ >+ /** > * Non-private only for testability. > * @param policy > * @param serviceDef >@@ -129,7 +129,7 @@ public class RangerDefaultPolicyEvaluator extends >RangerAbstractPolicyEvaluator > if > (Strings.isNullOrEmpty(evaluatorClassName)) { > > LOG.error("initializeConditionEvaluators: Serious Configuration >error: Couldn't get condition evaluator class name for condition[" + >conditionName + "]! Disabling all checks for this condition."); > } else { >- >RangerConditionEvaluator anEvaluator = >newConditionEvauator(evaluatorClassName); >+ >RangerConditionEvaluator anEvaluator = >newConditionEvaluator(evaluatorClassName); > if (anEvaluator > == null) { > > LOG.error("initializeConditionEvaluators: Serious Configuration >error: Couldn't instantiate condition evaluator for class[" + >evaluatorClassName + "]. All checks for condition[" + conditionName + "] >disabled."); > } else { >@@ -179,9 +179,9 @@ public class RangerDefaultPolicyEvaluator extends >RangerAbstractPolicyEvaluator > return result; > } > >- RangerConditionEvaluator newConditionEvauator(String className) { >+ RangerConditionEvaluator newConditionEvaluator(String className) { > if(LOG.isDebugEnabled()) { >- LOG.debug(String.format("==> >RangerDefaultPolicyEvaluator.newConditionEvauator(%s)", className)); >+ LOG.debug(String.format("==> >RangerDefaultPolicyEvaluator.newConditionEvaluator(%s)", className)); > } > > RangerConditionEvaluator evaluator = null; >@@ -195,92 +195,135 @@ public class RangerDefaultPolicyEvaluator extends >RangerAbstractPolicyEvaluator > } > > if(LOG.isDebugEnabled()) { >- LOG.debug(String.format("<== >RangerDefaultPolicyEvaluator.newConditionEvauator(%s)", evaluator == null >? >null : evaluator.toString())); >+ LOG.debug(String.format("<== >RangerDefaultPolicyEvaluator.newConditionEvaluator(%s)", evaluator == >null ? >null : evaluator.toString())); > } > return evaluator; > } > > @Override >- public void evaluate(RangerAccessRequest request, RangerAccessResult >result) { >- if(LOG.isDebugEnabled()) { >- LOG.debug("==> RangerDefaultPolicyEvaluator.evaluate(" >+ request + ", >" >+ result + ")"); >- } >- >- RangerPolicy policy = getPolicy(); >- >- if(policy != null && request != null && result != null) { >- boolean isResourceMatch = >isMatch(request.getResource()); >- boolean isResourceHeadMatch = isResourceMatch || >matchResourceHead(request.getResource()); >- String accessType = request.getAccessType(); >- >- if(StringUtils.isEmpty(accessType)) { >- accessType = RangerPolicyEngine.ANY_ACCESS; >- } >- >- boolean isAnyAccess = StringUtils.equals(accessType, >RangerPolicyEngine.ANY_ACCESS); >- boolean isAdminAccess = StringUtils.equals(accessType, >RangerPolicyEngine.ADMIN_ACCESS); >- >- if(isResourceMatch || (isResourceHeadMatch && >isAnyAccess)) { >- if(policy.getIsAuditEnabled()) { >- result.setIsAudited(true); >- } >- >- for(RangerPolicyItem policyItem : >policy.getPolicyItems()) { >- if(isAdminAccess) { >- >if(policyItem.getDelegateAdmin()) { >- >result.setIsAllowed(true); >- >result.setPolicyId(policy.getId()); >- break; >- } >- >- continue; >- } >- >- >if(CollectionUtils.isEmpty(policyItem.getAccesses())) { >- continue; >- } >- >- boolean isUserGroupMatch = >matchUserGroup(policyItem, >request.getUser(), request.getUserGroups()); >- >- if(! isUserGroupMatch) { >- continue; >- } >- >- boolean isCustomConditionsMatch = >matchCustomConditions(policyItem, >request, conditionEvaluators); >- >- if(! isCustomConditionsMatch) { >- continue; >- } >- >- if(isAnyAccess) { >- for(RangerPolicyItemAccess >access : policyItem.getAccesses()) { >- >if(access.getIsAllowed()) { >- >result.setIsAllowed(true); >- >result.setPolicyId(policy.getId()); >- break; >- } >- } >- } else { >- RangerPolicyItemAccess access = >getAccess(policyItem, accessType); >- >- if(access != null && >access.getIsAllowed()) { >- >result.setIsAllowed(true); >- >result.setPolicyId(policy.getId()); >- } >- } >- >- if(result.getIsAllowed()) { >- break; >- } >- } >- } >- } >- >- if(LOG.isDebugEnabled()) { >+ public void evaluate(RangerAccessRequest request, RangerAccessResult >result) { >+ if (LOG.isDebugEnabled()) { >+ LOG.debug("==> RangerDefaultPolicyEvaluator.evaluate(" + >request + ", " + result + ")"); >+ } >+ RangerPolicy policy = getPolicy(); >+ >+ if (policy != null && request != null && result != null) { >+ >+ String accessType = request.getAccessType(); >+ if (StringUtils.isEmpty(accessType)) { >+ accessType = RangerPolicyEngine.ANY_ACCESS; >+ } >+ boolean isAnyAccess = StringUtils.equals(accessType, >RangerPolicyEngine.ANY_ACCESS); >+ >+ boolean isMatchAttempted = false; >+ boolean matchResult = false; >+ boolean headMatchResult = false; >+ >+ if (!result.getIsAuditedDetermined()) { >+ // Need to match request.resource first. If it matches >(or >head matches), then only more progress can be made >+ matchResult = isMatch(request.getResource()); >+ isMatchAttempted = true; >+ >+ if (matchResult) { >+ // Do all stuff. >+ if (policy.getIsAuditEnabled()) { >+ result.setIsAudited(true); >+ } >+ } >+ } >+ >+ if (!result.getIsAccessDetermined()) { >+ if (!isMatchAttempted) { >+ // Need to match request.resource first. If it >matches >(or head matches), then only more progress can be made >+ matchResult = isMatch(request.getResource()); >+ isMatchAttempted = true; >+ } >+ >+ // Try head match only if it is useful >+ if (isAnyAccess) { >+ headMatchResult = matchResult || >matchResourceHead(request.getResource()); >+ } >+ >+ if (matchResult || (isAnyAccess && headMatchResult)) { >+ // A match was found earlier >+ evaluatePolicyItemsForAccess(request, result); >+ } >+ } >+ } >+ >+ if(LOG.isDebugEnabled()) { > LOG.debug("<== RangerDefaultPolicyEvaluator.evaluate(" > + request + ", >" >+ result + ")"); > } > } > >+ protected void evaluatePolicyItemsForAccess(RangerAccessRequest >request, RangerAccessResult result) { >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("==> >RangerDefaultPolicyEvaluator.evaluatePolicyItemsForAccess()"); >+ } >+ String accessType = request.getAccessType(); >+ if (StringUtils.isEmpty(accessType)) { >+ accessType = RangerPolicyEngine.ANY_ACCESS; >+ } >+ boolean isAnyAccess = StringUtils.equals(accessType, >RangerPolicyEngine.ANY_ACCESS); >+ boolean isAdminAccess = StringUtils.equals(accessType, >RangerPolicyEngine.ADMIN_ACCESS); >+ >+ for (RangerPolicy.RangerPolicyItem policyItem : >getPolicy().getPolicyItems()) { >+ >+ boolean isUserGroupMatch = matchUserGroup(policyItem, >request.getUser(), request.getUserGroups()); >+ >+ if (!isUserGroupMatch) { >+ continue; >+ } >+ // This is only for Grant and Revoke access requests sent by >the component. For those cases >+ // Our plugin will fill in the accessType as ADMIN_ACCESS. >+ >+ if (isAdminAccess) { >+ if (policyItem.getDelegateAdmin()) { >+ result.setIsAllowed(true); >+ result.setPolicyId(getPolicy().getId()); >+ break; >+ } >+ continue; >+ } >+ >+ if (CollectionUtils.isEmpty(policyItem.getAccesses())) { >+ continue; >+ } >+ >+ boolean accessAllowed = false; >+ if (isAnyAccess) { >+ for (RangerPolicy.RangerPolicyItemAccess access : >policyItem.getAccesses()) { >+ if (access.getIsAllowed()) { >+ accessAllowed = true; >+ break; >+ } >+ } >+ } else { >+ RangerPolicy.RangerPolicyItemAccess access = >getAccess(policyItem, accessType); >+ >+ if (access != null && access.getIsAllowed()) { >+ accessAllowed = true; >+ } >+ } >+ if (accessAllowed == false) { >+ continue; >+ } >+ >+ boolean isCustomConditionsMatch = >matchCustomConditions(policyItem, request, getConditionEvaluators()); >+ >+ if (!isCustomConditionsMatch) { >+ continue; >+ } >+ >+ result.setIsAllowed(true); >+ result.setPolicyId(getPolicy().getId()); >+ break; >+ } >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("<== >RangerDefaultPolicyEvaluator.evaluatePolicyItemsForAccess()"); >+ } >+ } >+ > @Override > public boolean isMatch(RangerResource resource) { > if(LOG.isDebugEnabled()) { >@@ -554,7 +597,7 @@ public class RangerDefaultPolicyEvaluator extends >RangerAbstractPolicyEvaluator > } > > if(LOG.isDebugEnabled()) { >- LOG.debug("==> >RangerDefaultPolicyEvaluator.createResourceMatcher(" + >resourceDef + ", " + resource + "): " + ret); >+ LOG.debug("<== >RangerDefaultPolicyEvaluator.createResourceMatcher(" + >resourceDef + ", " + resource + "): " + ret); > } > > return ret; > >http://git-wip-us.apache.org/repos/asf/incubator-ranger/blob/2e0be82d/agen >t >s-common/src/main/java/org/apache/ranger/plugin/policyevaluator/RangerOpti >mi >zedPolicyEvaluator.java >---------------------------------------------------------------------- >diff --git >a/agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/Ran >ge >rOptimizedPolicyEvaluator.java >b/agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/Ran >ge >rOptimizedPolicyEvaluator.java >new file mode 100644 >index 0000000..59fb87c >--- /dev/null >+++ >b/agents-common/src/main/java/org/apache/ranger/plugin/policyevaluator/Ran >ge >rOptimizedPolicyEvaluator.java >@@ -0,0 +1,247 @@ >+/* >+ * Licensed to the Apache Software Foundation (ASF) under one >+ * or more contributor license agreements. See the NOTICE file >+ * distributed with this work for additional information >+ * regarding copyright ownership. The ASF licenses this file >+ * to you under the Apache License, Version 2.0 (the >+ * "License"); you may not use this file except in compliance >+ * with the License. You may obtain a copy of the License at >+ * >+ * http://www.apache.org/licenses/LICENSE-2.0 >+ * >+ * Unless required by applicable law or agreed to in writing, >+ * software distributed under the License is distributed on an >+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY >+ * KIND, either express or implied. See the License for the >+ * specific language governing permissions and limitations >+ * under the License. >+ */ >+ >+package org.apache.ranger.plugin.policyevaluator; >+ >+import org.apache.commons.collections.CollectionUtils; >+import org.apache.commons.lang.StringUtils; >+import org.apache.commons.logging.Log; >+import org.apache.commons.logging.LogFactory; >+import org.apache.ranger.plugin.model.RangerPolicy; >+import org.apache.ranger.plugin.model.RangerServiceDef; >+import org.apache.ranger.plugin.policyengine.RangerAccessRequest; >+import org.apache.ranger.plugin.policyengine.RangerAccessResult; >+import org.apache.ranger.plugin.policyengine.RangerPolicyEngine; >+ >+import java.util.*; >+import java.lang.Math; >+ >+public class RangerOptimizedPolicyEvaluator extends >RangerDefaultPolicyEvaluator { >+ private static final Log LOG = >LogFactory.getLog(RangerOptimizedPolicyEvaluator.class); >+ >+ private Set<String> groups = null; >+ private Set<String> users = null; >+ private Set<String> accessPerms = null; >+ private boolean delegateAdmin = false; >+ private boolean hasAllPerms = false; >+ private boolean hasPublicGroup = false; >+ >+ >+ // For computation of priority >+ private static final String >RANGER_POLICY_EVAL_MATCH_ANY_PATTERN_STRING = "*"; >+ private static final String >RANGER_POLICY_EVAL_MATCH_ONE_CHARACTER_STRING = "?"; >+ private static final int >RANGER_POLICY_EVAL_MATCH_ANY_WILDCARD_PREMIUM >= 25; >+ private static final int >RANGER_POLICY_EVAL_CONTAINS_MATCH_ANY_WILDCARD_PREMIUM = 10; >+ private static final int >RANGER_POLICY_EVAL_CONTAINS_MATCH_ONE_CHARACTER_WILDCARD_PREMIUM = 10; >+ private static final int RANGER_POLICY_EVAL_HAS_EXCLUDES_PREMIUM > >= 25; >+ private static final int RANGER_POLICY_EVAL_IS_RECURSIVE_PREMIUM > >= 25; >+ private static final int >RANGER_POLICY_EVAL_PUBLIC_GROUP_ACCESS_PREMIUM = 25; >+ private static final int RANGER_POLICY_EVAL_ALL_ACCESS_TYPES_PREMIUM > >= 25; >+ private static final int RANGER_POLICY_EVAL_RESERVED_SLOTS_NUMBER > >= 10000; >+ private static final int >RANGER_POLICY_EVAL_RESERVED_SLOTS_PER_LEVEL_NUMBER = 1000; >+ >+ @Override >+ public void init(RangerPolicy policy, RangerServiceDef serviceDef) { >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("==> RangerOptimizedPolicyEvaluator.init()"); >+ } >+ >+ super.init(policy, serviceDef); >+ >+ accessPerms = new HashSet<String>(); >+ groups = new HashSet<String>(); >+ users = new HashSet<String>(); >+ >+ for (RangerPolicy.RangerPolicyItem item : >policy.getPolicyItems()) >{ >+ delegateAdmin = delegateAdmin || item.getDelegateAdmin(); >+ >+ List<RangerPolicy.RangerPolicyItemAccess> policyItemAccesses >= >item.getAccesses(); >+ for(RangerPolicy.RangerPolicyItemAccess policyItemAccess : >policyItemAccesses) { >+ >+ if (policyItemAccess.getIsAllowed()) { >+ String accessType = policyItemAccess.getType(); >+ accessPerms.add(accessType); >+ } >+ } >+ >+ groups.addAll(item.getGroups()); >+ users.addAll(item.getUsers()); >+ } >+ >+ hasAllPerms = checkIfHasAllPerms(); >+ >+ for (String group : groups) { >+ if (group.equalsIgnoreCase(RangerPolicyEngine.GROUP_PUBLIC)) >{ >+ hasPublicGroup = true; >+ } >+ } >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("<== RangerOptimizedPolicyEvaluator.init()"); >+ } >+ } >+ >+ @Override >+ public int computePriority() { >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("==> >RangerOptimizedPolicyEvaluator.computePriority()"); >+ } >+ RangerServiceDef serviceDef = getServiceDef(); >+ RangerPolicy policy = getPolicy(); >+ >+ class LevelResourceNames implements >Comparable<LevelResourceNames> >{ >+ int level; >+ RangerPolicy.RangerPolicyResource policyResource; >+ >+ @Override >+ public int compareTo(LevelResourceNames other) { >+ // Sort in ascending order of level numbers >+ return Integer.compare(this.level, other.level); >+ } >+ } >+ List<LevelResourceNames> tmpList = new >ArrayList<LevelResourceNames>(); >+ >+ List<RangerServiceDef.RangerResourceDef> resourceDefs = >serviceDef.getResources(); >+ >+ for (Map.Entry<String, RangerPolicy.RangerPolicyResource> >keyValuePair : policy.getResources().entrySet()) { >+ String serviceDefResourceName = keyValuePair.getKey(); >+ RangerPolicy.RangerPolicyResource policyResource = >keyValuePair.getValue(); >+ List<String> policyResourceNames = >policyResource.getValues(); >+ >+ RangerServiceDef.RangerResourceDef found = null; >+ for (RangerServiceDef.RangerResourceDef resourceDef : >resourceDefs) { >+ if >(serviceDefResourceName.equals(resourceDef.getName())) >{ >+ found = resourceDef; >+ break; >+ } >+ } >+ if (found != null) { >+ int level = found.getLevel(); >+ if (policyResourceNames != null) { >+ LevelResourceNames item = new LevelResourceNames(); >+ item.level = level; >+ item.policyResource = policyResource; >+ tmpList.add(item); >+ } >+ >+ } >+ >+ } >+ Collections.sort(tmpList); // Sort in ascending order of levels >+ >+ CharSequence matchesAnySeq = >RANGER_POLICY_EVAL_MATCH_ANY_PATTERN_STRING.subSequence(0, 1); >+ CharSequence matchesSingleCharacterSeq = >RANGER_POLICY_EVAL_MATCH_ONE_CHARACTER_STRING.subSequence(0, 1); >+ >+ int priorityLevel = RANGER_POLICY_EVAL_RESERVED_SLOTS_NUMBER; >+ boolean seenFirstMatchAny = false; >+ >+ for (LevelResourceNames item : tmpList) { >+ // Expect lowest level first >+ List<String> resourceNames = item.policyResource.getValues(); >+ boolean foundStarWildcard = false; >+ boolean foundQuestionWildcard = false; >+ boolean foundMatchAny = false; >+ >+ for (String resourceName : resourceNames) { >+ if (resourceName.isEmpty() >||resourceName.equals(RANGER_POLICY_EVAL_MATCH_ANY_PATTERN_STRING)) { >+ foundMatchAny = true; >+ break; >+ } >+ if (resourceName.contains(matchesAnySeq)) >+ foundStarWildcard = true; >+ else if >(resourceName.contains(matchesSingleCharacterSeq)) >+ foundQuestionWildcard = true; >+ } >+ if (foundMatchAny) { >+ if (seenFirstMatchAny) >+ priorityLevel -= >RANGER_POLICY_EVAL_MATCH_ANY_WILDCARD_PREMIUM; >+ else { >+ seenFirstMatchAny = true; >+ } >+ } else { >+ priorityLevel += >RANGER_POLICY_EVAL_RESERVED_SLOTS_PER_LEVEL_NUMBER; >+ if (foundStarWildcard) priorityLevel -= >RANGER_POLICY_EVAL_CONTAINS_MATCH_ANY_WILDCARD_PREMIUM; >+ else if (foundQuestionWildcard) priorityLevel -= >RANGER_POLICY_EVAL_CONTAINS_MATCH_ONE_CHARACTER_WILDCARD_PREMIUM; >+ >+ RangerPolicy.RangerPolicyResource resource = >item.policyResource; >+ if (resource.getIsExcludes()) priorityLevel -= >RANGER_POLICY_EVAL_HAS_EXCLUDES_PREMIUM; >+ if (resource.getIsRecursive()) priorityLevel -= >RANGER_POLICY_EVAL_IS_RECURSIVE_PREMIUM; >+ } >+ } >+ >+ if (hasPublicGroup) { >+ priorityLevel -= >RANGER_POLICY_EVAL_PUBLIC_GROUP_ACCESS_PREMIUM; >+ } else { >+ priorityLevel -= groups.size(); >+ } >+ priorityLevel -= users.size(); >+ >+ priorityLevel -= >Math.round(((float)RANGER_POLICY_EVAL_ALL_ACCESS_TYPES_PREMIUM * >accessPerms.size()) / serviceDef.getAccessTypes().size()); >+ >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("<== >RangerOptimizedPolicyEvaluator.computePriority(), policyName:" + >policy.getName() + ", priority:" + priorityLevel); >+ } >+ return priorityLevel; >+ } >+ @Override >+ protected void evaluatePolicyItemsForAccess(RangerAccessRequest >request, RangerAccessResult result) { >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("==> >RangerOptimizedPolicyEvaluator.evaluatePolicyItemsForAccess()"); >+ } >+ String accessType = request.getAccessType(); >+ if (StringUtils.isEmpty(accessType)) { >+ accessType = RangerPolicyEngine.ANY_ACCESS; >+ } >+ boolean isAnyAccess = StringUtils.equals(accessType, >RangerPolicyEngine.ANY_ACCESS); >+ boolean isAdminAccess = StringUtils.equals(accessType, >RangerPolicyEngine.ADMIN_ACCESS); >+ >+ if (hasPublicGroup || users.contains(request.getUser()) || >CollectionUtils.containsAny(groups, request.getUserGroups())) { >+ // No need to reject based on users and groups >+ >+ if (isAnyAccess || (isAdminAccess && delegateAdmin) || >hasAllPerms || accessPerms.contains(accessType)) { >+ // No need to reject based on aggregated access >permissions >+ super.evaluatePolicyItemsForAccess(request, result); >+ } >+ } >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("<== >RangerOptimizedPolicyEvaluator.evaluatePolicyItemsForAccess()"); >+ } >+ >+ } >+ private boolean checkIfHasAllPerms() { >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("==> >RangerOptimizedPolicyEvaluator.checkIfHasAllPerms()"); >+ } >+ >+ boolean result = true; >+ >+ List<RangerServiceDef.RangerAccessTypeDef> serviceAccessTypes = >getServiceDef().getAccessTypes(); >+ for (RangerServiceDef.RangerAccessTypeDef serviceAccessType : >serviceAccessTypes) { >+ if(! accessPerms.contains(serviceAccessType.getName())) { >+ return false; >+ } >+ } >+ >+ if(LOG.isDebugEnabled()) { >+ LOG.debug("==> >RangerOptimizedPolicyEvaluator.checkIfHasAllPerms(), " + result); >+ } >+ >+ return result; >+ } >+ >+} > >http://git-wip-us.apache.org/repos/asf/incubator-ranger/blob/2e0be82d/agen >t >s-common/src/test/java/org/apache/ranger/plugin/policyevaluator/RangerDefa >ul >tPolicyEvaluatorTest.java >---------------------------------------------------------------------- >diff --git >a/agents-common/src/test/java/org/apache/ranger/plugin/policyevaluator/Ran >ge >rDefaultPolicyEvaluatorTest.java >b/agents-common/src/test/java/org/apache/ranger/plugin/policyevaluator/Ran >ge >rDefaultPolicyEvaluatorTest.java >index 9256995..9efbcaf 100644 >--- >a/agents-common/src/test/java/org/apache/ranger/plugin/policyevaluator/Ran >ge >rDefaultPolicyEvaluatorTest.java >+++ >b/agents-common/src/test/java/org/apache/ranger/plugin/policyevaluator/Ran >ge >rDefaultPolicyEvaluatorTest.java >@@ -59,15 +59,15 @@ public class RangerDefaultPolicyEvaluatorTest { > } > > @Test >- public void test_newConditionEvauator() { >+ public void test_newConditionEvaulator() { > RangerDefaultPolicyEvaluator evaluator = new >RangerDefaultPolicyEvaluator(); >- RangerConditionEvaluator ipMatcher = >evaluator.newConditionEvauator("org.apache.ranger.plugin.conditionevaluato >r. >RangerIpMatcher"); >+ RangerConditionEvaluator ipMatcher = >evaluator.newConditionEvaluator("org.apache.ranger.plugin.conditionevaluat >or >.RangerIpMatcher"); > assertTrue(ipMatcher.getClass().equals(RangerIpMatcher.class)); > > // bogus value will lead to null object from coming back >- ipMatcher = >evaluator.newConditionEvauator("org.apache.ranger.plugin.conditionevaluato >r. >RangerIpMatcha"); >+ ipMatcher = >evaluator.newConditionEvaluator("org.apache.ranger.plugin.conditionevaluat >or >.RangerIpMatcha"); > assertNull(ipMatcher); >- ipMatcher = evaluator.newConditionEvauator("RangerIpMatcher"); >+ ipMatcher = evaluator.newConditionEvaluator("RangerIpMatcher"); > assertNull(ipMatcher); > } > > >http://git-wip-us.apache.org/repos/asf/incubator-ranger/blob/2e0be82d/agen >t >s-common/src/test/resources/policyengine/test_policyengine_hdfs.json >---------------------------------------------------------------------- >diff --git >a/agents-common/src/test/resources/policyengine/test_policyengine_hdfs.jso >n >b/agents-common/src/test/resources/policyengine/test_policyengine_hdfs.jso >n >index 943fe80..2acf868 100644 >--- >a/agents-common/src/test/resources/policyengine/test_policyengine_hdfs.jso >n >+++ >b/agents-common/src/test/resources/policyengine/test_policyengine_hdfs.jso >n >@@ -23,7 +23,7 @@ > } > , > {"id":2,"name":"allow-read-to-all under >/public/","isEnabled":true,"isAuditEnabled":false, >- "resources":{"path":{"values":["/public/"],"isRecursive":true}}, >+ "resources":{"path":{"values":["/public/*"],"isRecursive":true}}, > "policyItems":[ > >{"accesses":[{"type":"read","isAllowed":true},{"type":"execute","isAllowed >": >true}],"users":[],"groups":["public"],"delegateAdmin":false} > ] > >http://git-wip-us.apache.org/repos/asf/incubator-ranger/blob/2e0be82d/pom. >x >ml >---------------------------------------------------------------------- >diff --git a/pom.xml b/pom.xml >index 7df033d..fe340c4 100644 >--- a/pom.xml >+++ b/pom.xml >@@ -333,9 +333,9 @@ > <descriptor>src/main/assembly/plugin-yarn.xml</descriptor> > <descriptor>src/main/assembly/admin-web.xml</descriptor> > <descriptor>src/main/assembly/usersync.xml</descriptor> >- <descriptor>src/main/assembly/ranger-src.xml</descriptor> > ><descriptor>src/main/assembly/migration-util.xml</descriptor> > <descriptor>src/main/assembly/kms.xml</descriptor> >+ <descriptor>src/main/assembly/ranger-src.xml</descriptor> > </descriptors> > </configuration> > </plugin> > >http://git-wip-us.apache.org/repos/asf/incubator-ranger/blob/2e0be82d/src/ >m >ain/assembly/ranger-src.xml >---------------------------------------------------------------------- >diff --git a/src/main/assembly/ranger-src.xml >b/src/main/assembly/ranger-src.xml >index cf6b1da..077fa42 100644 >--- a/src/main/assembly/ranger-src.xml >+++ b/src/main/assembly/ranger-src.xml >@@ -16,45 +16,46 @@ > limitations under the License. > --> > <assembly> >- <id>src</id> >- <formats> >- <format>tar.gz</format> >- <format>zip</format> >- </formats> >- <baseDirectory>${project.name}-${project.version}-src</baseDirectory> >- <includeBaseDirectory>true</includeBaseDirectory> >- <fileSets> >- <fileSet> >- <directory>.</directory> >- <includes> >- <include>LICENCE.txt</include> >- <include>README.txt</include> >- <include>NOTICE.txt</include> >- <include>DISCLAIMER.txt</include> >- </includes> >- </fileSet> >- <fileSet> >- <directory>.</directory> >- <useDefaultExcludes>true</useDefaultExcludes> >- <excludes> >- <exclude>.git/**</exclude> >- <exclude>**/.gitignore</exclude> >- <exclude>**/.svn</exclude> >- <exclude>**/*.iws</exclude> >- <exclude>**/*.ipr</exclude> >- <exclude>**/*.iml</exclude> >- <exclude>**/.classpath</exclude> >- <exclude>**/.project</exclude> >- <exclude>**/.settings</exclude> >- <exclude>**/target/**</exclude> >- <!-- until the code that does this is fixed --> >- <exclude>**/*.log</exclude> >- <exclude>**/build/**</exclude> >- <exclude>**/file:/**</exclude> >- <exclude>**/debugfiles.list</exclude> >- <exclude>**/debuglinks.list</exclude> >- <exclude>**/debugsources.list</exclude> >- </excludes> >- </fileSet> >- </fileSets> >+ <id>src</id> >+ <formats> >+ <format>tar.gz</format> >+ <format>zip</format> >+ </formats> >+ <baseDirectory>${project.name}-${project.version}-src</baseDirectory> >+ <includeBaseDirectory>true</includeBaseDirectory> >+ <fileSets> >+ <fileSet> >+ <directory>.</directory> >+ <includes> >+ <include>LICENCE.txt</include> >+ <include>README.txt</include> >+ <include>NOTICE.txt</include> >+ <include>DISCLAIMER.txt</include> >+ </includes> >+ <useDefaultExcludes>true</useDefaultExcludes> >+ </fileSet> >+ <fileSet> >+ <directory>.</directory> >+ <excludes> >+ <exclude>.git/**</exclude> >+ <exclude>**/.gitignore</exclude> >+ <exclude>**/.svn</exclude> >+ <exclude>**/*.iws</exclude> >+ <exclude>**/*.ipr</exclude> >+ <exclude>**/*.iml</exclude> >+ <exclude>**/.classpath</exclude> >+ <exclude>**/.project</exclude> >+ <exclude>**/.settings</exclude> >+ <exclude>**/target/**</exclude> >+ <exclude>.idea/**</exclude> >+ <!-- until the code that does this is fixed --> >+ <exclude>**/*.log</exclude> >+ <exclude>**/build/**</exclude> >+ <exclude>**/file:/**</exclude> >+ <exclude>**/debugfiles.list</exclude> >+ <exclude>**/debuglinks.list</exclude> >+ <exclude>**/debugsources.list</exclude> >+ </excludes> >+ </fileSet> >+ </fileSets> > </assembly> > > >
