[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
chenruotao updated HIVE-27854: ------------------------------ Attachment: image-2023-11-06-20-26-07-406.png > Driver compile may causing memory leak > -------------------------------------- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive > Affects Versions: 3.1.2, 4.0.0-beta-1 > Reporter: chenruotao > Assignee: chenruotao > Priority: Major > Labels: pull-request-available > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png, image-2023-11-06-19-30-14-120.png, > image-2023-11-06-19-40-49-488.png, image-2023-11-06-19-45-03-388.png, > image-2023-11-06-20-26-07-406.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute the addShutdownHook method, but now no > thread to call the close method of driver, so it alway stay in > ShutdownHookManager's hooks, causing a memory leak > !image-2023-11-06-18-11-47-832.png|width=943,height=241! > !image-2023-11-06-19-45-03-388.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver after SQLOperation exexute cancel > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! > it will not trigger everytime, but we could do this to trigger it > 1、sleep 5s before compile > !image-2023-11-06-18-21-19-056.png! > 2、set hive.query.timeout.seconds to 4 > !image-2023-11-06-18-22-45-812.png! > 3、now everytime I submit sql , the shutdownHookManager$HookEntry will add one > !image-2023-11-06-18-24-41-982.png|width=842,height=134! > i solve it in hive 3.1.2 by adding this code : > !image-2023-11-06-19-30-14-120.png! > Obviously we see no more memory leak, and shutdownHookManager$HookEntry num > will not increase every time i submit sql > !image-2023-11-06-19-40-49-488.png|width=956,height=140! -- This message was sent by Atlassian Jira (v8.20.10#820010)