[
https://issues.apache.org/jira/browse/DRILL-5922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16236791#comment-16236791
]
Timothy Farkas commented on DRILL-5922:
---------------------------------------
[~ben-zvi] The *QueryContext* is managed by the *Foreman* class, which does not
close the QueryContext. It is also created temporarily for the purpose of
sending an rpc message in *PlanSplitter.planFragments* and is never closed
there as well. In both cases the QueryContexts are not part of the list of
AutoCloseables that we close when a Drillbit is closed. Also this leak doesn't
appear to just be a shutdown issue, this would also cause memory leaks to occur
in long running Drillbits.
That is weird that the error doesn't show in embedded mode. It also doesn't
appear in IntelliJ when the offending unit test is run by itself and only
appears when all the tests in DrillSeparatePlanningTest are run. Maybe stderr
just isn't flushed in some cases?
Closing the QueryContext in the two offending locations has fixed the exception
in the unit test. I'll do more testing to make sure nothing else has broken.
> Intermittent Memory Leaks in the ROOT allocator
> -------------------------------------------------
>
> Key: DRILL-5922
> URL: https://issues.apache.org/jira/browse/DRILL-5922
> Project: Apache Drill
> Issue Type: Bug
> Reporter: Timothy Farkas
> Assignee: Timothy Farkas
> Priority: Minor
>
> This issue was originall found by [~ben-zvi]. I am able to consistently
> reproduce the error on my laptop by running the following unit test:
> org.apache.drill.exec.DrillSeparatePlanningTest#testMultiMinorFragmentComplexQuery
> {code}
> java.lang.IllegalStateException: Allocator[ROOT] closed with outstanding
> child allocators.
> Allocator(ROOT) 0/1048576/10113536/3221225472 (res/actual/peak/limit)
> child allocators: 1
> Allocator(query:26049b50-0cec-0a92-437c-bbe486e1fcbf)
> 1048576/0/0/268435456 (res/actual/peak/limit)
> child allocators: 0
> ledgers: 0
> reservations: 0
> ledgers: 0
> reservations: 0
> at
> org.apache.drill.exec.memory.BaseAllocator.close(BaseAllocator.java:496)
> ~[classes/:na]
> at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76)
> [classes/:na]
> at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64)
> [classes/:na]
> at
> org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:256)
> ~[classes/:na]
> at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:76)
> [classes/:na]
> at org.apache.drill.common.AutoCloseables.close(AutoCloseables.java:64)
> [classes/:na]
> at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:205)
> [classes/:na]
> at org.apache.drill.BaseTestQuery.closeClient(BaseTestQuery.java:315)
> [test-classes/:na]
> at
> org.apache.drill.BaseTestQuery.updateTestCluster(BaseTestQuery.java:157)
> [test-classes/:na]
> at
> org.apache.drill.BaseTestQuery.updateTestCluster(BaseTestQuery.java:148)
> [test-classes/:na]
> at
> org.apache.drill.exec.DrillSeparatePlanningTest.getFragmentsHelper(DrillSeparatePlanningTest.java:185)
> [test-classes/:na]
> at
> org.apache.drill.exec.DrillSeparatePlanningTest.testMultiMinorFragmentComplexQuery(DrillSeparatePlanningTest.java:108)
> [test-classes/:na]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.8.0_144]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[na:1.8.0_144]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_144]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_144]
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> [junit-4.11.jar:na]
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> [junit-4.11.jar:na]
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> [junit-4.11.jar:na]
> at
> mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.executeTestMethod(JUnit4TestRunnerDecorator.java:120)
> [jmockit-1.3.jar:na]
> at
> mockit.integration.junit4.internal.JUnit4TestRunnerDecorator.invokeExplosively(JUnit4TestRunnerDecorator.java:65)
> [jmockit-1.3.jar:na]
> at
> mockit.integration.junit4.internal.MockFrameworkMethod.invokeExplosively(MockFrameworkMethod.java:29)
> [jmockit-1.3.jar:na]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.8.0_144]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[na:1.8.0_144]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_144]
> at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_144]
> at
> mockit.internal.util.MethodReflection.invokeWithCheckedThrows(MethodReflection.java:95)
> [jmockit-1.3.jar:na]
> at
> mockit.internal.annotations.MockMethodBridge.callMock(MockMethodBridge.java:76)
> [jmockit-1.3.jar:na]
> at
> mockit.internal.annotations.MockMethodBridge.invoke(MockMethodBridge.java:41)
> [jmockit-1.3.jar:na]
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java)
> [junit-4.11.jar:na]
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> [junit-4.11.jar:na]
> at
> org.junit.internal.runners.statements.FailOnTimeout$StatementThread.run(FailOnTimeout.java:74)
> [junit-4.11.jar:na]
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
