[
https://issues.apache.org/jira/browse/DRILL-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Khurram Faraaz reassigned DRILL-6229:
-------------------------------------
Assignee: Padma Penumarthy
> memory leak when output_batch_size is set to 4096
> -------------------------------------------------
>
> Key: DRILL-6229
> URL: https://issues.apache.org/jira/browse/DRILL-6229
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
> Affects Versions: 1.12.0
> Reporter: Khurram Faraaz
> Assignee: Padma Penumarthy
> Priority: Major
>
> When drill.exec.memory.operator.output_batch_size` = 4096 (4KB), we
> continuously see in each test run, the memory leak when tests are run from
> test framework.
> Drill 1.13.0 git.commit.id=766315ea17377199897d685ab801edd38394fe01
> alter system set `drill.exec.memory.operator.output_batch_size` = 4096;
> To run flatten tests from drill test framework
> bin/run_tests -s Functional/flatten_operators -g functional -d -t 500 -n 40
> The below two tests time out and we see the memory leak
> Timeouts:
> /root/drillTEST_FRAMEWORK/drill-test-framework/framework/resources/Functional/flatten_operators/100000rows/filter24.q
> Timeouts:
> /root/drillTEST_FRAMEWORK/drill-test-framework/framework/resources/Functional/flatten_operators/100000rows/filter19.q
> oadd.org.apache.drill.common.exceptions.UserException: CONNECTION ERROR:
> Connection ... (user client) closed unexpectedly. Drillbit down?
> Stack trace printed in drill test framework log
> {noformat}
> [Error Id: 54a16569-6555-4e92-aa5f-f76a8c11b31f ]
> at
> oadd.org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
> at
> oadd.org.apache.drill.exec.rpc.user.QueryResultHandler$ChannelClosedHandler$1.operationComplete(QueryResultHandler.java:376)
> at
> oadd.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507)
> at
> oadd.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:500)
> at
> oadd.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:479)
> at
> oadd.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
> at
> oadd.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:104)
> at
> oadd.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
> at
> oadd.io.netty.channel.AbstractChannel$CloseFuture.setClosed(AbstractChannel.java:1003)
> at
> oadd.io.netty.channel.AbstractChannel$AbstractUnsafe.doClose0(AbstractChannel.java:632)
> at
> oadd.io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:610)
> at
> oadd.io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:553)
> at
> oadd.io.netty.channel.DefaultChannelPipeline$HeadContext.close(DefaultChannelPipeline.java:1236)
> at
> oadd.io.netty.channel.AbstractChannelHandlerContext.invokeClose(AbstractChannelHandlerContext.java:618)
> at
> oadd.io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:602)
> at
> oadd.io.netty.channel.ChannelOutboundHandlerAdapter.close(ChannelOutboundHandlerAdapter.java:71)
> at
> oadd.io.netty.channel.AbstractChannelHandlerContext.invokeClose(AbstractChannelHandlerContext.java:618)
> at
> oadd.io.netty.channel.AbstractChannelHandlerContext.close(AbstractChannelHandlerContext.java:602)
> at
> oadd.io.netty.channel.ChannelDuplexHandler.close(ChannelDuplexHandler.java:73)
> at
> oadd.io.netty.channel.AbstractChannelHandlerContext.invokeClose(AbstractChannelHandlerContext.java:618)
> at
> oadd.io.netty.channel.AbstractChannelHandlerContext.access$1100(AbstractChannelHandlerContext.java:35)
> at
> oadd.io.netty.channel.AbstractChannelHandlerContext$13.run(AbstractChannelHandlerContext.java:607)
> at
> oadd.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:399)
> at oadd.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463)
> at
> oadd.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131)
> at java.lang.Thread.run(Thread.java:748)
> Channel is closed, discarding remaining 908 byte(s) in buffer.
> Exception while closing client.
> java.lang.RuntimeException: Exception while closing
> at
> oadd.org.apache.drill.common.DrillAutoCloseables.closeNoChecked(DrillAutoCloseables.java:46)
> at oadd.org.apache.drill.exec.client.DrillClient.close(DrillClient.java:459)
> at
> org.apache.drill.jdbc.impl.DrillConnectionImpl.closeOrWarn(DrillConnectionImpl.java:810)
> at
> org.apache.drill.jdbc.impl.DrillConnectionImpl.cleanup(DrillConnectionImpl.java:823)
> at
> org.apache.drill.jdbc.impl.DrillHandler.onConnectionClose(DrillHandler.java:36)
> at
> oadd.org.apache.calcite.avatica.AvaticaConnection.close(AvaticaConnection.java:205)
> at
> org.apache.drill.test.framework.ConnectionPool.close(ConnectionPool.java:102)
> at org.apache.drill.test.framework.TestDriver.runTests(TestDriver.java:562)
> at org.apache.drill.test.framework.TestDriver.main(TestDriver.java:100)
> Caused by: java.lang.IllegalStateException: Memory was leaked by query.
> Memory leaked: (4198400)
> Allocator(ROOT) 0/4096/6713344/28631367680 (res/actual/peak/limit)
> at
> oadd.org.apache.drill.exec.memory.BaseAllocator.close(BaseAllocator.java:520)
> at
> oadd.org.apache.drill.common.DrillAutoCloseables.closeNoChecked(DrillAutoCloseables.java:44)
> ... 8 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
