-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54531/
-----------------------------------------------------------
Review request for ranger.
Repository: ranger
Description
-------
# One failed embedded ServiceDef creating prevent any service from being
created in Ranger
EmbeddedServiceDefsUtil have this method (I removed some log statements to make
the code more clear)
```java
public class EmbeddedServiceDefsUtil {
...
private RangerServiceDef getOrCreateServiceDef(ServiceStore store,
String serviceDefName) {
....
RangerServiceDef ret = null;
boolean createServiceDef =
(CollectionUtils.isEmpty(supportedServiceDefs) ||
supportedServiceDefs.contains(serviceDefName));
try {
ret = store.getServiceDefByName(serviceDefName);
if(ret == null && createEmbeddedServiceDefs &&
createServiceDef) {
ret = loadEmbeddedServiceDef(serviceDefName);
if (ret.getId() != null) {
store.setPopulateExistingBaseFields(true);
ret = store.createServiceDef(ret);
store.setPopulateExistingBaseFields(false);
} else {
ret = store.createServiceDef(ret);
}
LOG.info("created embedded service-def " +
serviceDefName);
}
} catch(Exception excp) {
LOG.fatal("EmbeddedServiceDefsUtil.getOrCreateServiceDef(): failed to
load/create serviceType " + serviceDefName, excp);
}
...
return ret;
}
}
```
When creating a service def, the flag PopulateExistingBaseFields must be set to
true, but it should be set to false afterwards.
If the statement ``` ret = store.createServiceDef(ret); ``` throws an
exception, the flag is never reset. But since the catch statement do not
rethrow the exception, null will be returned and Ranger will start normally.
Now, the flag PopulateExistingBaseFields are used by a lot of methods in
ServiceDBStore
* public RangerServiceDef **createServiceDef**(RangerServiceDef serviceDef)
throws Exception
* public RangerService **createService**(RangerService service) throws Exception
* public RangerService **updateService**(RangerService service) throws Exception
* public RangerPolicy **createPolicy**(RangerPolicy policy) throws Exception {
For createService and updateService, it will cause the method to select between
svcServiceWithAssignedId and svcService. svcServiceWithAssignedId require that
the incoming service have an ID. Services created via the web gui and via the
ambari integration do NOT have a pre-selected ID. They cannot be created, and
thus you cannot actually use Ranger.
Now, how can ```ret = store.createServiceDef(ret);``` fail? I do not know, but
it happened on my system with this exception log entry
2016-12-06 09:34:25,850 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG
org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil
(EmbeddedServiceDefsUtil.java:136) - ==>
EmbeddedServiceDefsUtil.getOrCreateServiceDef(kms)
2016-12-06 09:34:25,850 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG
org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:934) - ==>
ServiceDefDBStore.getServiceDefByName(kms)
2016-12-06 09:34:25,855 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG
org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:946) - ==
ServiceDefDBStore.getServiceDefByName(kms): null
2016-12-06 09:34:25,855 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG
org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil
(EmbeddedServiceDefsUtil.java:168) - ==>
EmbeddedServiceDefsUtil.loadEmbeddedServiceDef(kms)
2016-12-06 09:34:25,867 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG
org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil
(EmbeddedServiceDefsUtil.java:182) - ==>
EmbeddedServiceDefsUtil.loadEmbeddedServiceDef(kms)
2016-12-06 09:34:25,867 [kact-man-001.kact.sblokalnet-startStop-1] INFO
org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil
(EmbeddedServiceDefsUtil.java:146) - creating embedded service-def kms
2016-12-06 09:34:25,868 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG
org.apache.ranger.biz.ServiceDBStore (ServiceDBStore.java:242) - ==>
ServiceDefDBStore.createServiceDef(RangerServiceDef={id={7} guid={null}
isEnabled={true} createdBy={null} updatedBy={null} createTime={null}
updateTime={null} version={null} name={kms}
implClass={org.apache.ranger.services.kms.RangerServiceKMS} label={KMS}
description={KMS} rbKeyLabel={null} rbKeyDescription={null} c
onfigs={RangerServiceConfigDef={itemId={provider} name={provider}
type={string} subType={null} mandatory={true} defaultValue={null}
validationRegEx={null} validationMessage={null} uiHint={null} label={KMS URL}
description={null} rbKeyLabel={null} rbKeyDescription={null}
rbKeyValidationMessage={null} }RangerServiceConfigDef={itemId={username}
name={username} type={string} subType={null} mandatory={true}
defaultValue={null} validationRegEx={null} validationM
essage={null} uiHint={null} label={Username} description={null}
rbKeyLabel={null} rbKeyDescription={null} rbKeyValidationMessage={null}
}RangerServiceConfigDef={itemId={password} name={password} type={password}
subType={null} mandatory={true} defaultValue={null} validationRegEx={null}
validationMessage={null} uiHint={null} label={Password} description={null}
rbKeyLabel={null} rbKeyDescription={null} rbKeyValidationMessage={null} }}
resources={RangerResourc
eDef={itemId={1} name={keyname} type={string} level={10} parent={}
mandatory={true} lookupSupported={true} recursiveSupported={false}
excludesSupported={false}
matcher={org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher}
matcherOptions={{wildCard=true, ignoreCase=false}} validationRegEx={}
validationMessage={} uiHint={} label={Key Name} description={Key Name}
rbKeyLabel={null} rbKeyDescription={null} rbKeyValidationMessage={null} }} acce
ssTypes={RangerAccessTypeDef={itemId={1} name={create} label={Create}
rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={2}
name={delete} label={Delete} rbKeyLabel={null} impliedGrants={}
}RangerAccessTypeDef={itemId={3} name={rollover} label={Rollover}
rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={4}
name={setkeymaterial} label={Set Key Material} rbKeyLabel={null}
impliedGrants={} }RangerAccessTypeDef={itemId={5} name={get
} label={Get} rbKeyLabel={null} impliedGrants={}
}RangerAccessTypeDef={itemId={6} name={getkeys} label={Get Keys}
rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={7}
name={getmetadata} label={Get Metadata} rbKeyLabel={null} impliedGrants={}
}RangerAccessTypeDef={itemId={8} name={generateeek} label={Generate EEK}
rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={9}
name={decrypteek} label={Decrypt EEK} rbKeyLabel={null} implie
dGrants={} }} policyConditions={} contextEnrichers={} enums={} })
2016-12-06 09:34:25,870 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG
org.apache.ranger.common.db.BaseDao (BaseDao.java:239) - Ignoring
BaseDao.setIdentityInsert(). This should be executed if DB flavor is sqlserver.
2016-12-06 09:34:25,872 [kact-man-001.kact.sblokalnet-startStop-1] FATAL
org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil
(EmbeddedServiceDefsUtil.java:156) -
EmbeddedServiceDefsUtil.getOrCreateServiceDef(): failed to load/create
serviceType kms
java.lang.NullPointerException
at
org.apache.ranger.service.RangerServiceDefServiceBase.mapViewToEntityBean(RangerServiceDefServiceBase.java:158)
at
org.apache.ranger.service.RangerServiceDefWithAssignedIdService.mapViewToEntityBean(RangerServiceDefWithAssignedIdService.java:31)
at
org.apache.ranger.service.RangerServiceDefWithAssignedIdService.mapViewToEntityBean(RangerServiceDefWithAssignedIdService.java:25)
at
org.apache.ranger.service.RangerBaseModelService.populateEntityBeanForCreate(RangerBaseModelService.java:180)
at
org.apache.ranger.service.RangerBaseModelService.preCreate(RangerBaseModelService.java:217)
at
org.apache.ranger.service.RangerBaseModelService.create(RangerBaseModelService.java:225)
at
org.apache.ranger.biz.ServiceDBStore.createServiceDef(ServiceDBStore.java:268)
at
org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil.getOrCreateServiceDef(EmbeddedServiceDefsUtil.java:149)
at
org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil.init(EmbeddedServiceDefsUtil.java:96)
at
org.apache.ranger.biz.ServiceDBStore$1.doInTransaction(ServiceDBStore.java:223)
at
org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:130)
at
org.apache.ranger.biz.ServiceDBStore.initStore(ServiceDBStore.java:220)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:346)
at
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:299)
at
org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:132)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:394)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1448)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:519)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:872)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:814)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:731)
at
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:485)
at
org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:92)
at
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:284)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1106)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:872)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:814)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:731)
at
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:485)
at
org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:92)
at
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:284)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1106)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:872)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:814)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:731)
at
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:485)
at
org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:92)
at
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:284)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1106)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.findAutowireCandidates(DefaultListableBeanFactory.java:872)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:814)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:731)
at
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:485)
at
org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:92)
at
org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:284)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1106)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:517)
at
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
at
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
at
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
at
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
at
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
at
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:605)
at
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:925)
at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:472)
at
org.springframework.web.context.ContextLoader.configureAndRefreshWebApplicationContext(ContextLoader.java:383)
at
org.springframework.web.context.ContextLoader.initWebApplicationContext(ContextLoader.java:283)
at
org.springframework.web.context.ContextLoaderListener.contextInitialized(ContextLoaderListener.java:111)
at
org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4992)
at
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5490)
at
org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
at
org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1575)
at
org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1565)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2016-12-06 09:34:25,875 [kact-man-001.kact.sblokalnet-startStop-1] DEBUG
org.apache.ranger.plugin.store.EmbeddedServiceDefsUtil
(EmbeddedServiceDefsUtil.java:160) - <==
EmbeddedServiceDefsUtil.getOrCreateServiceDef(kms): RangerServiceDef={id={7}
guid={null} isEnabled={true} createdBy={null} updatedBy={null}
createTime={null} updateTime={null} version={1} name={kms}
implClass={org.apache.ranger.services.kms.RangerServiceKMS} label={KMS}
description={KMS} rbKeyLabel={null} rbKeyDescription={null}
configs={RangerServiceConfigDef={itemId={provider} name={provider}
type={string} subType={null} mandatory={true} defaultValue={null}
validationRegEx={null} validationMessage={null} uiHint={null} label={KMS URL}
description={null} rbKeyLabel={null} rbKeyDescription={null}
rbKeyValidationMessage={null} }RangerServiceConfigDef={itemId={username}
name={username} type={string} subType={null} mandatory={true}
defaultValue={null} validationRegEx={null} validationMessage={null}
uiHint={null} label
={Username} description={null} rbKeyLabel={null} rbKeyDescription={null}
rbKeyValidationMessage={null} }RangerServiceConfigDef={itemId={password}
name={password} type={password} subType={null} mandatory={true}
defaultValue={null} validationRegEx={null} validationMessage={null}
uiHint={null} label={Password} description={null} rbKeyLabel={null}
rbKeyDescription={null} rbKeyValidationMessage={null} }}
resources={RangerResourceDef={itemId={1} name={keyname} type={string}
level={10} parent={} mandatory={true} lookupSupported={true}
recursiveSupported={false} excludesSupported={false}
matcher={org.apache.ranger.plugin.resourcematcher.RangerDefaultResourceMatcher}
matcherOptions={{wildCard=true, ignoreCase=false}} validationRegEx={}
validationMessage={} uiHint={} label={Key Name} description={Key Name}
rbKeyLabel={null} rbKeyDescription={null} rbKeyValidationMessage={null} }}
accessTypes={RangerAccessTypeDef={itemId={1} name={create} label={Create}
rbKeyLabel={null} impliedGrants={} }Rang
erAccessTypeDef={itemId={2} name={delete} label={Delete} rbKeyLabel={null}
impliedGrants={} }RangerAccessTypeDef={itemId={3} name={rollover}
label={Rollover} rbKeyLabel={null} impliedGrants={}
}RangerAccessTypeDef={itemId={4} name={setkeymaterial} label={Set Key Material}
rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={5} name={get}
label={Get} rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={6}
name={getkeys} label={Get Keys} rbKeyLabel={null} impliedGrants={}
}RangerAccessTypeDef={itemId={7} name={getmetadata} label={Get Metadata}
rbKeyLabel={null} impliedGrants={} }RangerAccessTypeDef={itemId={8}
name={generateeek} label={Generate EEK} rbKeyLabel={null} impliedGrants={}
}RangerAccessTypeDef={itemId={9} name={decrypteek} label={Decrypt EEK}
rbKeyLabel={null} impliedGrants={} }} policyConditions={} contextEnrichers={}
enums={} }
Diffs
-----
agents-common/src/main/java/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.java
adae311
Diff: https://reviews.apache.org/r/54531/diff/
Testing
-------
Installed compiled EmbeddedServiceDefsUtil.class into
```/usr/iop/current/ranger-admin/ews/webapp/WEB-INF/classes/org/apache/ranger/plugin/store/EmbeddedServiceDefsUtil.class```
After this services could be created
Thanks,
Asger Blekinge
