[
https://issues.apache.org/jira/browse/DRILL-3095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Khurram Faraaz resolved DRILL-3095.
-----------------------------------
Resolution: Fixed
> Memory Leak : Failure while closing accountor.
> ----------------------------------------------
>
> Key: DRILL-3095
> URL: https://issues.apache.org/jira/browse/DRILL-3095
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
> Affects Versions: 1.0.0
> Environment: f7f6efc525cd833ce1530deae32eb9ccb20b664a
> Reporter: Khurram Faraaz
> Assignee: Chris Westin
> Fix For: 1.2.0
>
>
> I am seeing a memory leak when i Cancel a long running query on sqlline. I am
> re running the query with assertion enabled, will add details after the
> second run is complete with assertions.
> Long running query was,
> {code}
> select key1, key2 from `twoKeyJsn.json`;
> {code}
> I did Ctrl-C when the above query was running on sqlline, and then issued the
> below query that returned correct results. And then I see there is a memory
> leak message in drillbit.log
> {code}
> 0: jdbc:drill:schema=dfs.tmp> select count(*) from `twoKeyJsn.json`;
> +------------+
> | EXPR$0 |
> +------------+
> | 26212355 |
> +------------+
> 1 row selected (14.734 seconds)
> {code}
> Stack trace from drillbit.log
> {code}
> 2015-05-15 00:59:01,951 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] WARN
> o.a.drill.exec.ops.SendingAccountor - Interrupted while waiting for send
> complete. Continuing to wait.
> java.lang.InterruptedException: null
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1301)
> ~[na:1.7.0_45]
> at java.util.concurrent.Semaphore.acquire(Semaphore.java:472)
> ~[na:1.7.0_45]
> at
> org.apache.drill.exec.ops.SendingAccountor.waitForSendComplete(SendingAccountor.java:48)
> ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.ops.FragmentContext.waitForSendComplete(FragmentContext.java:436)
> [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.BaseRootExec.close(BaseRootExec.java:112)
> [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.close(ScreenCreator.java:141)
> [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:333)
> [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:278)
> [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
> 2015-05-15 00:59:01,952 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 2aaabb50-3afd-2906-3f48-eb86a315a1f5:0:0: State change requested from
> CANCELLATION_REQUESTED --> FAILED for
> 2015-05-15 00:59:01,952 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 2aaabb50-3afd-2906-3f48-eb86a315a1f5:0:0: State change requested from FAILED
> --> FAILED for
> 2015-05-15 00:59:01,952 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] INFO
> o.a.d.e.w.fragment.FragmentExecutor -
> 2aaabb50-3afd-2906-3f48-eb86a315a1f5:0:0: State change requested from FAILED
> --> FINISHED for
> 2015-05-15 00:59:01,956 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] ERROR
> o.a.d.c.exceptions.UserException - SYSTEM ERROR:
> java.lang.IllegalStateException: Failure while closing accountor. Expected
> private and shared pools to be set to initial values. However, one or more
> were not. Stats are
> zone init allocated delta
> private 1000000 918080 81920
> shared 9999000000 9999000000 0.
> Fragment 0:0
> [Error Id: 90ced8b1-b6db-438f-b193-b7634de31b81 on centos-03.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> java.lang.IllegalStateException: Failure while closing accountor. Expected
> private and shared pools to be set to initial values. However, one or more
> were not. Stats are
> zone init allocated delta
> private 1000000 918080 81920
> shared 9999000000 9999000000 0.
> Fragment 0:0
> [Error Id: 90ced8b1-b6db-438f-b193-b7634de31b81 on centos-03.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:522)
> ~[drill-common-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:315)
> [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:283)
> [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
> Caused by: java.lang.IllegalStateException: Failure while closing accountor.
> Expected private and shared pools to be set to initial values. However, one
> or more were not. Stats are
> zone init allocated delta
> private 1000000 918080 81920
> shared 9999000000 9999000000 0.
> at
> org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:200)
> ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at org.apache.drill.exec.memory.Accountor.close(Accountor.java:386)
> ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.close(TopLevelAllocator.java:325)
> ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.ops.OperatorContextImpl.close(OperatorContextImpl.java:116)
> ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.ops.FragmentContext.suppressingClose(FragmentContext.java:405)
> ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.ops.FragmentContext.close(FragmentContext.java:394)
> ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:340)
> [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:278)
> [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> ... 4 common frames omitted
> 2015-05-15 00:59:01,963 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] INFO
> o.a.drill.exec.work.foreman.Foreman - State change requested.
> CANCELLATION_REQUESTED --> FAILED
> org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR:
> java.lang.IllegalStateException: Failure while closing accountor. Expected
> private and shared pools to be set to initial values. However, one or more
> were not. Stats are
> zone init allocated delta
> private 1000000 918080 81920
> shared 9999000000 9999000000 0.
> Fragment 0:0
> [Error Id: 90ced8b1-b6db-438f-b193-b7634de31b81 on centos-03.qa.lab:31010]
> at
> org.apache.drill.exec.work.foreman.QueryManager$1.statusUpdate(QueryManager.java:481)
> [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.foreman.QueryManager$RootStatusReporter.statusChange(QueryManager.java:461)
> [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:90)
> [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:86)
> [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:319)
> [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:283)
> [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> [na:1.7.0_45]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> [na:1.7.0_45]
> at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
