Jing-Long Wu created HIVE-27318:
-----------------------------------
Summary: Hive CLI creates 2 Tez sessions
Key: HIVE-27318
URL: https://issues.apache.org/jira/browse/HIVE-27318
Project: Hive
Issue Type: Bug
Components: Hive, Tez
Affects Versions: 3.2.0
Environment: Bigtop 3.2.0 on Debian 10
h4.
Reporter: Jing-Long Wu
Running Hive CLI I noticed in the new version of Hive 3.1.3 and Tez 0.10.1 in
Bigtop 3.2.0 (as opposed to Bigtop 1.5.0), there are now 2 Tez sessions created
with "hive.execution.engine=tez" enabled as default:
{code:java}
2023-05-04T13:46:19,189 INFO [main] SessionState: Hive Session ID =
dbd5b26f-9904-44a1-b70b-9477287e0c03
2023-05-04T13:46:22,275 INFO [pool-6-thread-1] SessionState: Hive Session ID =
58d15cbd-f815-4c55-b323-02559dcacd61{code}
When compared to previous behavior, there was only 1 session created with the
main thread. Worst thing is that the session created by the pool thread is not
even utilized at all, and will just linger and take up resources for the
ApplicationMaster container, until it dies from timeout defined in
"tez.session.am.dag.submit.timeout.secs".
Running queries also only utilizes the session created by the main thread ie.
{code:java}
Status: Running (Executing on YARN cluster with App id
application_1682588680048_0135)
2023-05-04T13:50:18,804 INFO [dbd5b26f-9904-44a1-b70b-9477287e0c03 main]
SessionState: Status: Running (Executing on YARN cluster with App id
application_1682588680048_0135)
{code}
This is using identical configs in Hive for both Hive 3.1.3 and Hive 2.3.6.
This behavior is not present in Hiveserver2 though, and could only be observed
when using the Hive CLI.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
