[ https://issues.apache.org/jira/browse/DRILL-6132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431408#comment-16431408 ]
Timothy Farkas commented on DRILL-6132: --------------------------------------- DRILL-6125 improved the flow of the code and in the HashPartitionSender and eliminated some race conditions. This issue may be resolved bug needs to be tested. > HashPartitionSender leaks memory > -------------------------------- > > Key: DRILL-6132 > URL: https://issues.apache.org/jira/browse/DRILL-6132 > Project: Apache Drill > Issue Type: Bug > Components: Functions - Drill > Affects Versions: 1.12.0 > Reporter: Chun Chang > Assignee: Timothy Farkas > Priority: Major > > Enable assertion (-ea), I noticed HashPartitionSender leaks memory if > aggregation query fails due to OOM. > {noformat} > message: "SYSTEM ERROR: IllegalStateException: > Allocator[op:2:1:0:HashPartitionSender] closed with outstanding buffers > allocated (1).\nAllocator(op:2:1:0:HashPartitionSender) > 1000000/8192/3300352/10000000000 (res/actual/peak/limit)\n child allocators: > 0\n ledgers: 1\n ledger[703835] allocator: op:2:1:0:HashPartitionSender), > isOwning: true, size: 8192, references: 1, life: 6329151004063490..0, > allocatorManager: [688058, life: 6329151004058252..0] holds 1 buffers. \n > DrillBuf[777552], udle: [688059 0..8192]: , \n reservations: 0\n\n\nFragment > 2:1\n\n[Error Id: c7cc9d37-8881-4db1-8123-2651628c4081 on > 10.10.30.168:31010]\n\n (java.lang.IllegalStateException) > Allocator[op:2:1:0:HashPartitionSender] closed with outstanding buffers > allocated (1).\nAllocator(op:2:1:0:HashPartitionSender) > 1000000/8192/3300352/10000000000 (res/actual/peak/limit)\n child allocators: > 0\n ledgers: 1\n ledger[703835] allocator: op:2:1:0:HashPartitionSender), > isOwning: true, size: 8192, references: 1, life: 6329151004063490..0, > allocatorManager: [688058, life: 6329151004058252..0] holds 1 buffers. \n > DrillBuf[777552], udle: [688059 0..8192]: , \n reservations: 0\n\n > org.apache.drill.exec.memory.BaseAllocator.close():504\n > org.apache.drill.exec.ops.BaseOperatorContext.close():157\n > org.apache.drill.exec.ops.OperatorContextImpl.close():79\n > org.apache.drill.exec.ops.FragmentContext.suppressingClose():429\n > org.apache.drill.exec.ops.FragmentContext.close():418\n > org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources():324\n > org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup():155\n > org.apache.drill.exec.work.fragment.FragmentExecutor.run():267\n > org.apache.drill.common.SelfCleaningRunnable.run():38\n > java.util.concurrent.ThreadPoolExecutor.runWorker():1149\n > java.util.concurrent.ThreadPoolExecutor$Worker.run():624\n > java.lang.Thread.run():748\n" > exception { > exception_class: "java.lang.IllegalStateException" > message: "Allocator[op:2:1:0:HashPartitionSender] closed with outstanding > buffers allocated (1).\nAllocator(op:2:1:0:HashPartitionSender) > 1000000/8192/3300352/10000000000 (res/actual/peak/limit)\n child allocators: > 0\n ledgers: 1\n ledger[703835] allocator: op:2:1:0:HashPartitionSender), > isOwning: true, size: 8192, references: 1, life: 6329151004063490..0, > allocatorManager: [688058, life: 6329151004058252..0] holds 1 buffers. \n > DrillBuf[777552], udle: [688059 0..8192]: , \n reservations: 0\n" > stack_trace { > class_name: "org.apache.drill.exec.memory.BaseAllocator" > file_name: "BaseAllocator.java" > line_number: 504 > method_name: "close" > is_native_method: false > } > stack_trace { > class_name: "org.apache.drill.exec.ops.BaseOperatorContext" > file_name: "BaseOperatorContext.java" > line_number: 157 > method_name: "close" > is_native_method: false > } > stack_trace { > class_name: "org.apache.drill.exec.ops.OperatorContextImpl" > file_name: "OperatorContextImpl.java" > line_number: 79 > method_name: "close" > is_native_method: false > } > stack_trace { > class_name: "org.apache.drill.exec.ops.FragmentContext" > file_name: "FragmentContext.java" > line_number: 429 > method_name: "suppressingClose" > is_native_method: false > } > stack_trace { > class_name: "org.apache.drill.exec.ops.FragmentContext" > file_name: "FragmentContext.java" > line_number: 418 > method_name: "close" > is_native_method: false > } > stack_trace { > class_name: "org.apache.drill.exec.work.fragment.FragmentExecutor" > file_name: "FragmentExecutor.java" > line_number: 324 > method_name: "closeOutResources" > is_native_method: false > } > stack_trace { > class_name: "org.apache.drill.exec.work.fragment.FragmentExecutor" > file_name: "FragmentExecutor.java" > line_number: 155 > method_name: "cleanup" > is_native_method: false > } > stack_trace { > class_name: "org.apache.drill.exec.work.fragment.FragmentExecutor" > file_name: "FragmentExecutor.java" > line_number: 267 > method_name: "run" > is_native_method: false > } > stack_trace { > class_name: "org.apache.drill.common.SelfCleaningRunnable" > file_name: "SelfCleaningRunnable.java" > line_number: 38 > method_name: "run" > is_native_method: false > } > stack_trace { > class_name: "..." > line_number: 0 > method_name: "..." > is_native_method: false > } > } > }{noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)