[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Description: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute the addShutdownHook method, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png|width=943,height=241! !image-2023-11-06-19-45-03-388.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver after SQLOperation exexute cancel !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! and now the driverState is INITIALIZED, so the lDrvState can not set abort to interrupted the compile !image-2023-11-06-20-26-07-406.png|width=479,height=365! !image-2023-11-06-20-28-08-239.png|width=731,height=97! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png|width=763,height=257! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the shutdownHookManager$HookEntry will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! i solve it in hive 3.1.2 by adding this code : !image-2023-11-06-19-30-14-120.png! Obviously we see no more memory leak, and shutdownHookManager$HookEntry num will not increase every time i submit sql !image-2023-11-06-19-40-49-488.png|width=956,height=140! was: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute the addShutdownHook method, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png|width=943,height=241! !image-2023-11-06-19-45-03-388.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver after SQLOperation exexute cancel !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! and now the driverState is INITIALIZED, so the lDrvState can not set abort to interrupted the !image-2023-11-06-20-26-07-406.png|width=479,height=365! !image-2023-11-06-20-28-08-239.png|width=731,height=97! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png|width=763,height=257! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the shutdownHookManager$HookEntry will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! i solve it in hive 3.1.2 by adding this code : !image-2023-11-06-19-30-14-120.png! Obviously we see no more memory leak, and shutdownHookManager$HookEntry num will not increase every time i submit sql !image-2023-11-06-19-40-49-488.png|width=956,height=140! > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Labels: pull-request-available > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png, image-2023-11-06-19-30-14-120.png, > image-2023-11-06-19-40-49-488.png, image-2023-11-06-19-45-03-388.png, > image-2023-11-06-20-26-07-406.png, image-2023-11-06-20-28-08-239.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute the addShutdownHook method, but now no > thread to call the close method of driver, so it alway stay in > ShutdownHookManager's hooks, causing a memory leak > !image-2023-11-06-18-11-47-832.png|width=943,height=241! > !image-2023-11-06-19-45-03-388.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver after SQLOperation exexute cancel > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! > and now the driverState is INITIALIZED, so the lDrvState can not set abort to > interrupted the compile > !image-2023-11-06-20-26-07-406.png|width=479,height=365! > !image-2023-11-06-20-28-08-239.png|width=731,height=97! > it will not trigger everytime, but we could do this to trigger it > 1、sleep 5s before compile > !image-2023-11-06-18-21-19-056.png|width=763,height=257! > 2、set hive.query.timeout.seconds to 4 > !image-2023-11-06-18-22-45-812.png!
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Attachment: image-2023-11-06-20-28-08-239.png > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Labels: pull-request-available > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png, image-2023-11-06-19-30-14-120.png, > image-2023-11-06-19-40-49-488.png, image-2023-11-06-19-45-03-388.png, > image-2023-11-06-20-26-07-406.png, image-2023-11-06-20-28-08-239.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute the addShutdownHook method, but now no > thread to call the close method of driver, so it alway stay in > ShutdownHookManager's hooks, causing a memory leak > !image-2023-11-06-18-11-47-832.png|width=943,height=241! > !image-2023-11-06-19-45-03-388.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver after SQLOperation exexute cancel > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! > it will not trigger everytime, but we could do this to trigger it > 1、sleep 5s before compile > !image-2023-11-06-18-21-19-056.png! > 2、set hive.query.timeout.seconds to 4 > !image-2023-11-06-18-22-45-812.png! > 3、now everytime I submit sql , the shutdownHookManager$HookEntry will add one > !image-2023-11-06-18-24-41-982.png|width=842,height=134! > i solve it in hive 3.1.2 by adding this code : > !image-2023-11-06-19-30-14-120.png! > Obviously we see no more memory leak, and shutdownHookManager$HookEntry num > will not increase every time i submit sql > !image-2023-11-06-19-40-49-488.png|width=956,height=140! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Description: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute the addShutdownHook method, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png|width=943,height=241! !image-2023-11-06-19-45-03-388.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver after SQLOperation exexute cancel !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! and now the driverState is INITIALIZED, so the lDrvState can not set abort to interrupted the !image-2023-11-06-20-26-07-406.png|width=479,height=365! !image-2023-11-06-20-28-08-239.png|width=731,height=97! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png|width=763,height=257! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the shutdownHookManager$HookEntry will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! i solve it in hive 3.1.2 by adding this code : !image-2023-11-06-19-30-14-120.png! Obviously we see no more memory leak, and shutdownHookManager$HookEntry num will not increase every time i submit sql !image-2023-11-06-19-40-49-488.png|width=956,height=140! was: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute the addShutdownHook method, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png|width=943,height=241! !image-2023-11-06-19-45-03-388.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver after SQLOperation exexute cancel !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the shutdownHookManager$HookEntry will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! i solve it in hive 3.1.2 by adding this code : !image-2023-11-06-19-30-14-120.png! Obviously we see no more memory leak, and shutdownHookManager$HookEntry num will not increase every time i submit sql !image-2023-11-06-19-40-49-488.png|width=956,height=140! > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Labels: pull-request-available > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png, image-2023-11-06-19-30-14-120.png, > image-2023-11-06-19-40-49-488.png, image-2023-11-06-19-45-03-388.png, > image-2023-11-06-20-26-07-406.png, image-2023-11-06-20-28-08-239.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute the addShutdownHook method, but now no > thread to call the close method of driver, so it alway stay in > ShutdownHookManager's hooks, causing a memory leak > !image-2023-11-06-18-11-47-832.png|width=943,height=241! > !image-2023-11-06-19-45-03-388.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver after SQLOperation exexute cancel > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! > and now the driverState is INITIALIZED, so the lDrvState can not set abort to > interrupted the > !image-2023-11-06-20-26-07-406.png|width=479,height=365! > !image-2023-11-06-20-28-08-239.png|width=731,height=97! > it will not trigger everytime, but we could do this to trigger it > 1、sleep 5s before compile > !image-2023-11-06-18-21-19-056.png|width=763,height=257! > 2、set hive.query.timeout.seconds to 4 > !image-2023-11-06-18-22-45-812.png! > 3、now everytime I submit sql , the shutdownHookManager$HookEntry will add one > !image-2023-11-06-18-24-41-982.png|width=842,height=134! > i solve it in hive 3.1.2 by adding this code : > !image-2023-11-06-19-30-14-120.png! > Obviously we see
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Attachment: image-2023-11-06-20-26-07-406.png > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Labels: pull-request-available > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png, image-2023-11-06-19-30-14-120.png, > image-2023-11-06-19-40-49-488.png, image-2023-11-06-19-45-03-388.png, > image-2023-11-06-20-26-07-406.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute the addShutdownHook method, but now no > thread to call the close method of driver, so it alway stay in > ShutdownHookManager's hooks, causing a memory leak > !image-2023-11-06-18-11-47-832.png|width=943,height=241! > !image-2023-11-06-19-45-03-388.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver after SQLOperation exexute cancel > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! > it will not trigger everytime, but we could do this to trigger it > 1、sleep 5s before compile > !image-2023-11-06-18-21-19-056.png! > 2、set hive.query.timeout.seconds to 4 > !image-2023-11-06-18-22-45-812.png! > 3、now everytime I submit sql , the shutdownHookManager$HookEntry will add one > !image-2023-11-06-18-24-41-982.png|width=842,height=134! > i solve it in hive 3.1.2 by adding this code : > !image-2023-11-06-19-30-14-120.png! > Obviously we see no more memory leak, and shutdownHookManager$HookEntry num > will not increase every time i submit sql > !image-2023-11-06-19-40-49-488.png|width=956,height=140! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Description: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute the addShutdownHook method, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png|width=943,height=241! !image-2023-11-06-19-45-03-388.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver after SQLOperation exexute cancel !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the shutdownHookManager$HookEntry will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! i solve it in hive 3.1.2 by adding this code : !image-2023-11-06-19-30-14-120.png! Obviously we see no more memory leak, and shutdownHookManager$HookEntry num will not increase every time i submit sql !image-2023-11-06-19-40-49-488.png|width=956,height=140! was: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute the addShutdownHook method, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png|width=943,height=241! !image-2023-11-06-19-45-03-388.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver after SQLOperation exexute cancel !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! i solve it in hive 3.1.2 by adding this code : !image-2023-11-06-19-30-14-120.png! Obviously we see no more memory leak, and shutdownHookManager$HookEntry num will not increase every time i submit sql !image-2023-11-06-19-40-49-488.png|width=956,height=140! > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Labels: pull-request-available > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png, image-2023-11-06-19-30-14-120.png, > image-2023-11-06-19-40-49-488.png, image-2023-11-06-19-45-03-388.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute the addShutdownHook method, but now no > thread to call the close method of driver, so it alway stay in > ShutdownHookManager's hooks, causing a memory leak > !image-2023-11-06-18-11-47-832.png|width=943,height=241! > !image-2023-11-06-19-45-03-388.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver after SQLOperation exexute cancel > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! > it will not trigger everytime, but we could do this to trigger it > 1、sleep 5s before compile > !image-2023-11-06-18-21-19-056.png! > 2、set hive.query.timeout.seconds to 4 > !image-2023-11-06-18-22-45-812.png! > 3、now everytime I submit sql , the shutdownHookManager$HookEntry will add one > !image-2023-11-06-18-24-41-982.png|width=842,height=134! > i solve it in hive 3.1.2 by adding this code : > !image-2023-11-06-19-30-14-120.png! > Obviously we see no more memory leak, and shutdownHookManager$HookEntry num > will not increase every time i submit sql > !image-2023-11-06-19-40-49-488.png|width=956,height=140! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Description: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute the addShutdownHook method, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png|width=943,height=241! !image-2023-11-06-19-45-03-388.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver after SQLOperation exexute cancel !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! i solve it in hive 3.1.2 by adding this code : !image-2023-11-06-19-30-14-120.png! Obviously we see no more memory leak, and shutdownHookManager$HookEntry num will not increase every time i submit sql !image-2023-11-06-19-40-49-488.png|width=956,height=140! was: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute the addShutdownHook method, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png|width=943,height=241! !image-2023-11-06-19-45-03-388.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver after SQLOperation exexute cancel !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! i solve it in hive 3.1.2 by add this code : !image-2023-11-06-19-30-14-120.png! and obviously we see no more memory leak, ShutdownHookManager$HookEntry num will not increase every time i submit sql !image-2023-11-06-19-40-49-488.png|width=956,height=140! > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Labels: pull-request-available > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png, image-2023-11-06-19-30-14-120.png, > image-2023-11-06-19-40-49-488.png, image-2023-11-06-19-45-03-388.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute the addShutdownHook method, but now no > thread to call the close method of driver, so it alway stay in > ShutdownHookManager's hooks, causing a memory leak > !image-2023-11-06-18-11-47-832.png|width=943,height=241! > !image-2023-11-06-19-45-03-388.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver after SQLOperation exexute cancel > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! > it will not trigger everytime, but we could do this to trigger it > 1、sleep 5s before compile > !image-2023-11-06-18-21-19-056.png! > 2、set hive.query.timeout.seconds to 4 > !image-2023-11-06-18-22-45-812.png! > 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one > !image-2023-11-06-18-24-41-982.png|width=842,height=134! > i solve it in hive 3.1.2 by adding this code : > !image-2023-11-06-19-30-14-120.png! > Obviously we see no more memory leak, and shutdownHookManager$HookEntry num > will not increase every time i submit sql > !image-2023-11-06-19-40-49-488.png|width=956,height=140! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Description: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute the addShutdownHook method, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png|width=943,height=241! !image-2023-11-06-19-45-03-388.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver after SQLOperation exexute cancel !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! i solve it in hive 3.1.2 by add this code : !image-2023-11-06-19-30-14-120.png! and obviously we see no more memory leak, ShutdownHookManager$HookEntry num will not increase every time i submit sql !image-2023-11-06-19-40-49-488.png|width=956,height=140! was: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute the addShutdownHook method, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png|width=943,height=241! !image-2023-11-06-19-45-03-388.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! i solve it in hive 3.1.2 by add this code : !image-2023-11-06-19-30-14-120.png! and obviously we see no more memory leak, ShutdownHookManager$HookEntry num will not increase every time i submit sql !image-2023-11-06-19-40-49-488.png|width=956,height=140! > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Labels: pull-request-available > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png, image-2023-11-06-19-30-14-120.png, > image-2023-11-06-19-40-49-488.png, image-2023-11-06-19-45-03-388.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute the addShutdownHook method, but now no > thread to call the close method of driver, so it alway stay in > ShutdownHookManager's hooks, causing a memory leak > !image-2023-11-06-18-11-47-832.png|width=943,height=241! > !image-2023-11-06-19-45-03-388.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver after SQLOperation exexute cancel > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! > it will not trigger everytime, but we could do this to trigger it > 1、sleep 5s before compile > !image-2023-11-06-18-21-19-056.png! > 2、set hive.query.timeout.seconds to 4 > !image-2023-11-06-18-22-45-812.png! > 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one > !image-2023-11-06-18-24-41-982.png|width=842,height=134! > i solve it in hive 3.1.2 by add this code : > !image-2023-11-06-19-30-14-120.png! > and obviously we see no more memory leak, ShutdownHookManager$HookEntry num > will not increase every time i submit sql > !image-2023-11-06-19-40-49-488.png|width=956,height=140! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Attachment: image-2023-11-06-19-45-03-388.png > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Labels: pull-request-available > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png, image-2023-11-06-19-30-14-120.png, > image-2023-11-06-19-40-49-488.png, image-2023-11-06-19-45-03-388.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute addShutdownHook, but now no thread to call > the close method of driver, so it alway stay in ShutdownHookManager's hooks, > causing a memory leak > !image-2023-11-06-18-11-47-832.png|width=943,height=241! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! > it will not trigger everytime, but we could do this to trigger it > 1、sleep 5s before compile > !image-2023-11-06-18-21-19-056.png! > 2、set hive.query.timeout.seconds to 4 > !image-2023-11-06-18-22-45-812.png! > 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one > !image-2023-11-06-18-24-41-982.png|width=842,height=134! > i solve it in hive 3.1.2 by add this code : > !image-2023-11-06-19-30-14-120.png! > and obviously we see no more memory leak, ShutdownHookManager$HookEntry num > will not increase every time i submit sql > !image-2023-11-06-19-40-49-488.png|width=956,height=140! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Description: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute the addShutdownHook method, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png|width=943,height=241! !image-2023-11-06-19-45-03-388.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! i solve it in hive 3.1.2 by add this code : !image-2023-11-06-19-30-14-120.png! and obviously we see no more memory leak, ShutdownHookManager$HookEntry num will not increase every time i submit sql !image-2023-11-06-19-40-49-488.png|width=956,height=140! was: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute addShutdownHook, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png|width=943,height=241! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! i solve it in hive 3.1.2 by add this code : !image-2023-11-06-19-30-14-120.png! and obviously we see no more memory leak, ShutdownHookManager$HookEntry num will not increase every time i submit sql !image-2023-11-06-19-40-49-488.png|width=956,height=140! > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Labels: pull-request-available > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png, image-2023-11-06-19-30-14-120.png, > image-2023-11-06-19-40-49-488.png, image-2023-11-06-19-45-03-388.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute the addShutdownHook method, but now no > thread to call the close method of driver, so it alway stay in > ShutdownHookManager's hooks, causing a memory leak > !image-2023-11-06-18-11-47-832.png|width=943,height=241! > !image-2023-11-06-19-45-03-388.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! > it will not trigger everytime, but we could do this to trigger it > 1、sleep 5s before compile > !image-2023-11-06-18-21-19-056.png! > 2、set hive.query.timeout.seconds to 4 > !image-2023-11-06-18-22-45-812.png! > 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one > !image-2023-11-06-18-24-41-982.png|width=842,height=134! > i solve it in hive 3.1.2 by add this code : > !image-2023-11-06-19-30-14-120.png! > and obviously we see no more memory leak, ShutdownHookManager$HookEntry num > will not increase every time i submit sql > !image-2023-11-06-19-40-49-488.png|width=956,height=140! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Description: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute addShutdownHook, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png|width=943,height=241! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! i solve it in hive 3.1.2 by add this code : !image-2023-11-06-19-30-14-120.png! and obviously we see no more memory leak, ShutdownHookManager$HookEntry num will not increase every time i submit sql !image-2023-11-06-19-40-49-488.png|width=956,height=140! was: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute addShutdownHook, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! i solve it in hive 3.1.2 by add this code : !image-2023-11-06-19-30-14-120.png! and obviously we see no more memory leak, ShutdownHookManager$HookEntry num will not increase every time i submit sql !image-2023-11-06-19-40-49-488.png! > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Labels: pull-request-available > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png, image-2023-11-06-19-30-14-120.png, > image-2023-11-06-19-40-49-488.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute addShutdownHook, but now no thread to call > the close method of driver, so it alway stay in ShutdownHookManager's hooks, > causing a memory leak > !image-2023-11-06-18-11-47-832.png|width=943,height=241! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! > it will not trigger everytime, but we could do this to trigger it > 1、sleep 5s before compile > !image-2023-11-06-18-21-19-056.png! > 2、set hive.query.timeout.seconds to 4 > !image-2023-11-06-18-22-45-812.png! > 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one > !image-2023-11-06-18-24-41-982.png|width=842,height=134! > i solve it in hive 3.1.2 by add this code : > !image-2023-11-06-19-30-14-120.png! > and obviously we see no more memory leak, ShutdownHookManager$HookEntry num > will not increase every time i submit sql > !image-2023-11-06-19-40-49-488.png|width=956,height=140! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Description: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute addShutdownHook, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! i solve it in hive 3.1.2 by add this code : !image-2023-11-06-19-30-14-120.png! and obviously we see no more memory leak, ShutdownHookManager$HookEntry num will not increase every time i submit sql !image-2023-11-06-19-40-49-488.png! was: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute addShutdownHook, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! i solve it in hive 3.1.2 by add this code : !image-2023-11-06-19-30-14-120.png! and obviously we see no more memory leak > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Labels: pull-request-available > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png, image-2023-11-06-19-30-14-120.png, > image-2023-11-06-19-40-49-488.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute addShutdownHook, but now no thread to call > the close method of driver, so it alway stay in ShutdownHookManager's hooks, > causing a memory leak > !image-2023-11-06-18-11-47-832.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! > it will not trigger everytime, but we could do this to trigger it > 1、sleep 5s before compile > !image-2023-11-06-18-21-19-056.png! > 2、set hive.query.timeout.seconds to 4 > !image-2023-11-06-18-22-45-812.png! > 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one > !image-2023-11-06-18-24-41-982.png|width=842,height=134! > i solve it in hive 3.1.2 by add this code : > !image-2023-11-06-19-30-14-120.png! > and obviously we see no more memory leak, ShutdownHookManager$HookEntry num > will not increase every time i submit sql > !image-2023-11-06-19-40-49-488.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Attachment: image-2023-11-06-19-40-49-488.png > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Labels: pull-request-available > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png, image-2023-11-06-19-30-14-120.png, > image-2023-11-06-19-40-49-488.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute addShutdownHook, but now no thread to call > the close method of driver, so it alway stay in ShutdownHookManager's hooks, > causing a memory leak > !image-2023-11-06-18-11-47-832.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! > it will not trigger everytime, but we could do this to trigger it > 1、sleep 5s before compile > !image-2023-11-06-18-21-19-056.png! > 2、set hive.query.timeout.seconds to 4 > !image-2023-11-06-18-22-45-812.png! > 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one > !image-2023-11-06-18-24-41-982.png|width=842,height=134! > i solve it in hive 3.1.2 by add this code : > !image-2023-11-06-19-30-14-120.png! > and obviously we see no more memory leak -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-27854: -- Labels: pull-request-available (was: ) > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Labels: pull-request-available > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png, image-2023-11-06-19-30-14-120.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute addShutdownHook, but now no thread to call > the close method of driver, so it alway stay in ShutdownHookManager's hooks, > causing a memory leak > !image-2023-11-06-18-11-47-832.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! > it will not trigger everytime, but we could do this to trigger it > 1、sleep 5s before compile > !image-2023-11-06-18-21-19-056.png! > 2、set hive.query.timeout.seconds to 4 > !image-2023-11-06-18-22-45-812.png! > 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one > !image-2023-11-06-18-24-41-982.png|width=842,height=134! > i solve it in hive 3.1.2 by add this code : > !image-2023-11-06-19-30-14-120.png! > and obviously we see no more memory leak -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Description: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute addShutdownHook, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! i solve it in hive 3.1.2 by add this code : !image-2023-11-06-19-30-14-120.png! and obviously we see no more memory leak was: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute addShutdownHook, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png, image-2023-11-06-19-30-14-120.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute addShutdownHook, but now no thread to call > the close method of driver, so it alway stay in ShutdownHookManager's hooks, > causing a memory leak > !image-2023-11-06-18-11-47-832.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! > it will not trigger everytime, but we could do this to trigger it > 1、sleep 5s before compile > !image-2023-11-06-18-21-19-056.png! > 2、set hive.query.timeout.seconds to 4 > !image-2023-11-06-18-22-45-812.png! > 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one > !image-2023-11-06-18-24-41-982.png|width=842,height=134! > i solve it in hive 3.1.2 by add this code : > !image-2023-11-06-19-30-14-120.png! > and obviously we see no more memory leak -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Attachment: image-2023-11-06-19-30-14-120.png > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png, image-2023-11-06-19-30-14-120.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute addShutdownHook, but now no thread to call > the close method of driver, so it alway stay in ShutdownHookManager's hooks, > causing a memory leak > !image-2023-11-06-18-11-47-832.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! > it will not trigger everytime, but we could do this to trigger it > 1、sleep 5s before compile > !image-2023-11-06-18-21-19-056.png! > 2、set hive.query.timeout.seconds to 4 > !image-2023-11-06-18-22-45-812.png! > 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one > !image-2023-11-06-18-24-41-982.png|width=842,height=134! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Description: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute addShutdownHook, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! it will not trigger everytime, but we could do this to trigger it 1、sleep 5s before compile !image-2023-11-06-18-21-19-056.png! 2、set hive.query.timeout.seconds to 4 !image-2023-11-06-18-22-45-812.png! 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one !image-2023-11-06-18-24-41-982.png|width=842,height=134! was: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute addShutdownHook, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute addShutdownHook, but now no thread to call > the close method of driver, so it alway stay in ShutdownHookManager's hooks, > causing a memory leak > !image-2023-11-06-18-11-47-832.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! > it will not trigger everytime, but we could do this to trigger it > 1、sleep 5s before compile > !image-2023-11-06-18-21-19-056.png! > 2、set hive.query.timeout.seconds to 4 > !image-2023-11-06-18-22-45-812.png! > 3、now everytime I submit sql , the ShutdownHookManager's hooks will add one > !image-2023-11-06-18-24-41-982.png|width=842,height=134! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Attachment: image-2023-11-06-18-24-41-982.png > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png, > image-2023-11-06-18-24-41-982.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute addShutdownHook, but now no thread to call > the close method of driver, so it alway stay in ShutdownHookManager's hooks, > causing a memory leak > !image-2023-11-06-18-11-47-832.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Attachment: image-2023-11-06-18-22-45-812.png > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png, image-2023-11-06-18-22-45-812.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute addShutdownHook, but now no thread to call > the close method of driver, so it alway stay in ShutdownHookManager's hooks, > causing a memory leak > !image-2023-11-06-18-11-47-832.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Attachment: image-2023-11-06-18-21-19-056.png > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png, > image-2023-11-06-18-21-19-056.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute addShutdownHook, but now no thread to call > the close method of driver, so it alway stay in ShutdownHookManager's hooks, > causing a memory leak > !image-2023-11-06-18-11-47-832.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HIVE-27854) Driver compile may causing memory leak
[ https://issues.apache.org/jira/browse/HIVE-27854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] chenruotao updated HIVE-27854: -- Attachment: image-2023-11-06-18-13-43-182.png image-2023-11-06-18-13-00-842.png image-2023-11-06-18-11-47-832.png Description: when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute addShutdownHook, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak !image-2023-11-06-18-11-47-832.png! the cancel from SQLOperation will set it's driver = null, so it can not call the close method of driver !image-2023-11-06-18-13-43-182.png! !image-2023-11-06-18-13-00-842.png! was:when submit a sql to hiveserver, if the sql query timeout before compile, it will continue compile and execute addShutdownHook, but now no thread to call the close method of driver, so it alway stay in ShutdownHookManager's hooks, causing a memory leak > Driver compile may causing memory leak > -- > > Key: HIVE-27854 > URL: https://issues.apache.org/jira/browse/HIVE-27854 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 3.1.2, 4.0.0-beta-1 >Reporter: chenruotao >Assignee: chenruotao >Priority: Major > Attachments: image-2023-11-06-18-11-47-832.png, > image-2023-11-06-18-13-00-842.png, image-2023-11-06-18-13-43-182.png > > > when submit a sql to hiveserver, if the sql query timeout before compile, it > will continue compile and execute addShutdownHook, but now no thread to call > the close method of driver, so it alway stay in ShutdownHookManager's hooks, > causing a memory leak > !image-2023-11-06-18-11-47-832.png! > the cancel from SQLOperation will set it's driver = null, so it can not call > the close method of driver > !image-2023-11-06-18-13-43-182.png! > !image-2023-11-06-18-13-00-842.png! -- This message was sent by Atlassian Jira (v8.20.10#820010)
