[ https://issues.apache.org/jira/browse/DRILL-3085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Steven Phillips updated DRILL-3085: ----------------------------------- Fix Version/s: 1.0.0 > In ExternalSortBatch, Memory Leak in Runtime Generation Code > ------------------------------------------------------------ > > Key: DRILL-3085 > URL: https://issues.apache.org/jira/browse/DRILL-3085 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Codegen > Reporter: Sean Hsuan-Yi Chu > Assignee: Sean Hsuan-Yi Chu > Priority: Critical > Fix For: 1.0.0 > > > This case is related to DRILL-3065. > In ExternalSortBatch, we have an MSorter to do sorting thing. In this class, > there are two SelectionVector4, vector4 and aux. If we fail at the time just > after either gets new memory, the close() method would fail to clean their > allocated memory properly. > To reproduce this problem, inject an exception at the last step of > MSortTemplate.setup() > Detailed Information: > 1. Query: > select n_name from cp.`tpch/nation.parquet` order by n_name > *. Using this query alone cannot help reproduce the issue. We still need to > inject the exception at the right place. > 2. Data: > cp.`tpch/nation.parquet` > 3. Log: > java.lang.IllegalStateException: Failure while closing accountor. Expected > private and shared pools to be set to initial values. However, one or more > were not. Stats are > zone init allocated delta > private 0 0 0 > shared 3221225472 3195686243 25539229. > at > org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:200) > at org.apache.drill.exec.memory.Accountor.close(Accountor.java:386) > at > org.apache.drill.exec.memory.TopLevelAllocator.close(TopLevelAllocator.java:175) > at > org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:75) > at com.google.common.io.Closeables.close(Closeables.java:77) > at com.google.common.io.Closeables.closeQuietly(Closeables.java:108) > at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:292) > at org.apache.drill.BaseTestQuery.closeClient(BaseTestQuery.java:238) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) > at > org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) > at org.junit.runners.ParentRunner.run(ParentRunner.java:309) > at org.junit.runner.JUnitCore.run(JUnitCore.java:160) > at > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:74) > at > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:211) > at > com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:67) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134) > Exception in thread "Drillbit-ShutdownHook#0" java.lang.RuntimeException: > Caught exception closing Drillbit started from > org.apache.drill.common.StackTrace.<init>:36 > org.apache.drill.exec.server.Drillbit.run:250 > org.apache.drill.BaseTestQuery.openClient:180 > org.apache.drill.BaseTestQuery.setupDefaultTestCluster:116 > sun.reflect.NativeMethodAccessorImpl.invoke0:-2 > sun.reflect.NativeMethodAccessorImpl.invoke:57 > sun.reflect.DelegatingMethodAccessorImpl.invoke:43 > java.lang.reflect.Method.invoke:606 > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall:47 > org.junit.internal.runners.model.ReflectiveCallable.run:12 > org.junit.runners.model.FrameworkMethod.invokeExplosively:44 > org.junit.internal.runners.statements.RunBefores.evaluate:24 > org.junit.internal.runners.statements.RunAfters.evaluate:27 > org.junit.runners.ParentRunner.run:309 > org.junit.runner.JUnitCore.run:160 > com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs:74 > com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart:211 > com.intellij.rt.execution.junit.JUnitStarter.main:67 > sun.reflect.NativeMethodAccessorImpl.invoke0:-2 > sun.reflect.NativeMethodAccessorImpl.invoke:57 > sun.reflect.DelegatingMethodAccessorImpl.invoke:43 > java.lang.reflect.Method.invoke:606 > at > org.apache.drill.exec.server.Drillbit$ShutdownThread.run(Drillbit.java:333) > Caused by: java.lang.IllegalStateException: Failure while closing accountor. > Expected private and shared pools to be set to initial values. However, one > or more were not. Stats are > zone init allocated delta > private 0 0 0 > shared 3221225472 3195686243 25539229. > at > org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:200) > at org.apache.drill.exec.memory.Accountor.close(Accountor.java:386) > at > org.apache.drill.exec.memory.TopLevelAllocator.close(TopLevelAllocator.java:175) > at > org.apache.drill.exec.server.BootStrapContext.close(BootStrapContext.java:75) > at com.google.common.io.Closeables.close(Closeables.java:77) > at com.google.common.io.Closeables.closeQuietly(Closeables.java:108) > at org.apache.drill.exec.server.Drillbit.close(Drillbit.java:292) > at > org.apache.drill.exec.server.Drillbit$ShutdownThread.run(Drillbit.java:330) -- This message was sent by Atlassian JIRA (v6.3.4#6332)