[
https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chenruotao updated HIVE-27854:
------------------------------
Attachment: image-2023-11-06-19-40-49-488.png
> Driver compile may causing memory leak
> --------------------------------------
>
> Key: HIVE-27854
> URL: https://issues.apache.org/jira/browse/HIVE-27854
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Affects Versions: 3.1.2, 4.0.0-beta-1
> Reporter: chenruotao
> Assignee: chenruotao
> Priority: Major
> Labels: pull-request-available
> Attachments: image-2023-11-06-18-11-47-832.png,
> image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png,
> image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png,
> image-2023-11-06-18-24-41-982.png, image-2023-11-06-19-30-14-120.png,
> image-2023-11-06-19-40-49-488.png
>
>
> when submit a sql to hiveserver, if the sql query timeout before compile, it
> will continue compile and execute addShutdownHook, but now no thread to call
> the close method of driver, so it alway stay in ShutdownHookManager's hooks,
> causing a memory leak
> !image-2023-11-06-18-11-47-832.png!
> the cancel from SQLOperation will set it's driver = null, so it can not call
> the close method of driver
> !image-2023-11-06-18-13-43-182.png!
> !image-2023-11-06-18-13-00-842.png!
> it will not trigger everytime, but we could do this to trigger it
> 1、sleep 5s before compile
> !image-2023-11-06-18-21-19-056.png!
> 2、set hive.query.timeout.seconds to 4
> !image-2023-11-06-18-22-45-812.png!
> 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one
> !image-2023-11-06-18-24-41-982.png|width=842,height=134!
> i solve it in hive 3.1.2 by add this code :
> !image-2023-11-06-19-30-14-120.png!
> and obviously we see no more memory leak
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
