[
https://issues.apache.org/jira/browse/RANGER-1826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pradeep Agrawal updated RANGER-1826:
------------------------------------
Description:
while importing bulk polices around 1000, OOM error is thrown:
{code}
2017-09-05 10:19:53,856 [http-bio-6080-exec-13] ERROR
org.apache.ranger.rest.ServiceREST (ServiceREST.java:1603) - deletePolicy(282)
failed
java.lang.OutOfMemoryError: Java heap space
2017-09-05 10:19:53,857 [http-bio-6080-exec-13] INFO
org.apache.ranger.common.RESTErrorUtil (RESTErrorUtil.java:63) - Request
failed. loginId=admin, logMessage=Java heap space
2017-09-05 10:19:54,481 [http-bio-6080-exec-13] ERROR
org.apache.ranger.rest.ServiceREST (ServiceREST.java:2117) - Error while
importing policy from file!!
java.lang.OutOfMemoryError: GC overhead limit exceeded
2017-09-05 10:19:54,483 [http-bio-6080-exec-13] INFO
org.apache.ranger.common.RESTErrorUtil (RESTErrorUtil.java:63) - Request
failed. loginId=admin, logMessage=GC overhead limit exceeded
javax.ws.rs.WebApplicationException
at
org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:56)
at
org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:311)
at
org.apache.ranger.rest.ServiceREST.importPoliciesFromFile(ServiceREST.java:2125)
at
org.apache.ranger.rest.ServiceREST$$FastClassBySpringCGLIB$$92dab672.invoke(<generated>)
at
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:700)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at
org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:64)
{code}
then Increased the heap size:
${JAVA_OPTS} -XX:MaxPermSize=256m -Xmx2048m -Xms1024m
then restarted the ranger service and imported the policies again:
it causes apparent deadlock
{code}
2017-09-05 13:38:22,966 [Timer-0] WARN
com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector
(ThreadPoolAsynchronousRunner.java:608) -
com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@6726e60e --
APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
{code}
and after sometime it starts giving OOM again:
{code}
2017-09-05 13:50:54,670 [http-bio-6080-exec-4] ERROR
org.apache.ranger.rest.ServiceREST (ServiceREST.java:1603) - deletePolicy(256)
failed
java.lang.OutOfMemoryError: Java heap space
2017-09-05 13:52:08,355 [zkCallback-2-thread-5-EventThread] WARN
org.apache.solr.common.cloud.ConnectionManager (ConnectionManager.java:183) -
zkClient received AuthFailed
2017-09-05 13:52:20,662 [http-bio-6080-exec-4] INFO ? (?:?) - Request failed.
loginId=admin, logMessage=Java heap space
javax.ws.rs.WebApplicationException
at
org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:56)
at
org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:311)
at
org.apache.ranger.rest.ServiceREST.deletePolicy(ServiceREST.java:1605)
at
org.apache.ranger.rest.ServiceREST.deletePoliciesProvidedInServiceMap(ServiceREST.java:224
{code}
> Import of bulk policies is causing OOM and Apparent Deadlock
> ------------------------------------------------------------
>
> Key: RANGER-1826
> URL: https://issues.apache.org/jira/browse/RANGER-1826
> Project: Ranger
> Issue Type: Bug
> Components: Ranger
> Affects Versions: 0.7.0, 1.0.0
> Reporter: Deepak Sharma
> Assignee: Pradeep Agrawal
> Fix For: 1.0.0, 0.7.2
>
> Attachments: RANGER-1826-master-4.patch
>
>
> while importing bulk polices around 1000, OOM error is thrown:
> {code}
> 2017-09-05 10:19:53,856 [http-bio-6080-exec-13] ERROR
> org.apache.ranger.rest.ServiceREST (ServiceREST.java:1603) -
> deletePolicy(282) failed
> java.lang.OutOfMemoryError: Java heap space
> 2017-09-05 10:19:53,857 [http-bio-6080-exec-13] INFO
> org.apache.ranger.common.RESTErrorUtil (RESTErrorUtil.java:63) - Request
> failed. loginId=admin, logMessage=Java heap space
> 2017-09-05 10:19:54,481 [http-bio-6080-exec-13] ERROR
> org.apache.ranger.rest.ServiceREST (ServiceREST.java:2117) - Error while
> importing policy from file!!
> java.lang.OutOfMemoryError: GC overhead limit exceeded
> 2017-09-05 10:19:54,483 [http-bio-6080-exec-13] INFO
> org.apache.ranger.common.RESTErrorUtil (RESTErrorUtil.java:63) - Request
> failed. loginId=admin, logMessage=GC overhead limit exceeded
> javax.ws.rs.WebApplicationException
> at
> org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:56)
> at
> org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:311)
> at
> org.apache.ranger.rest.ServiceREST.importPoliciesFromFile(ServiceREST.java:2125)
> at
> org.apache.ranger.rest.ServiceREST$$FastClassBySpringCGLIB$$92dab672.invoke(<generated>)
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:700)
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
> at
> org.springframework.security.access.intercept.aopalliance.MethodSecurityInterceptor.invoke(MethodSecurityInterceptor.java:64)
> {code}
> then Increased the heap size:
> ${JAVA_OPTS} -XX:MaxPermSize=256m -Xmx2048m -Xms1024m
> then restarted the ranger service and imported the policies again:
> it causes apparent deadlock
> {code}
> 2017-09-05 13:38:22,966 [Timer-0] WARN
> com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector
> (ThreadPoolAsynchronousRunner.java:608) -
> com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector@6726e60e
> -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending
> tasks!
> {code}
> and after sometime it starts giving OOM again:
> {code}
> 2017-09-05 13:50:54,670 [http-bio-6080-exec-4] ERROR
> org.apache.ranger.rest.ServiceREST (ServiceREST.java:1603) -
> deletePolicy(256) failed
> java.lang.OutOfMemoryError: Java heap space
> 2017-09-05 13:52:08,355 [zkCallback-2-thread-5-EventThread] WARN
> org.apache.solr.common.cloud.ConnectionManager (ConnectionManager.java:183) -
> zkClient received AuthFailed
> 2017-09-05 13:52:20,662 [http-bio-6080-exec-4] INFO ? (?:?) - Request
> failed. loginId=admin, logMessage=Java heap space
> javax.ws.rs.WebApplicationException
> at
> org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:56)
> at
> org.apache.ranger.common.RESTErrorUtil.createRESTException(RESTErrorUtil.java:311)
> at
> org.apache.ranger.rest.ServiceREST.deletePolicy(ServiceREST.java:1605)
> at
> org.apache.ranger.rest.ServiceREST.deletePoliciesProvidedInServiceMap(ServiceREST.java:224
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
