[jira] [Commented] (HIVE-14397) Queries launched after reopening of tez session launches additional sessions
[ https://issues.apache.org/jira/browse/HIVE-14397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15402615#comment-15402615 ] Sergey Shelukhin commented on HIVE-14397: - +1 > Queries launched after reopening of tez session launches additional sessions > > > Key: HIVE-14397 > URL: https://issues.apache.org/jira/browse/HIVE-14397 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 2.1.0, 2.2.0 >Reporter: Takahiko Saito >Assignee: Prasanth Jayachandran >Priority: Critical > Attachments: HIVE-14397.1.patch > > > Say we have configured hive.server2.tez.default.queues with 2 queues q1 and > q2 with default expiry interval of 5 mins. > After 5 mins of non-usage the sessions corresponding to queues q1 and q2 will > be expired. When new set of queries are issue after this expiry, the default > sessions backed by q1 and q2 and reopened again. Now when we run more queries > the reopened sessions are not used instead new session is opened. > At this point there will be 4 sessions running (2 abandoned sessions and 2 > current sessions). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14397) Queries launched after reopening of tez session launches additional sessions
[
https://issues.apache.org/jira/browse/HIVE-14397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15402582#comment-15402582
]
Prasanth Jayachandran commented on HIVE-14397:
--
[~sershe] The root cause of this bug is this code
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L275-L287
If the queue name is set in conf then it always creates a new session when
getSession is invoked.
The part that fixes this is conf.unset("tez.queue.name"). This makes sure we
use the sessions from the pool when available.
The other part of the patch
{code}
conf.set(TezConfiguration.TEZ_QUEUE_NAME, sessionState.getQueueName());
{code}
is required to reopen the initial sessions when user specified queue name is
not available. If we don't set this then all the reopened sessions will use
"default" queue.
> Queries launched after reopening of tez session launches additional sessions
>
>
> Key: HIVE-14397
> URL: https://issues.apache.org/jira/browse/HIVE-14397
> Project: Hive
> Issue Type: Bug
> Components: Tez
>Affects Versions: 2.1.0, 2.2.0
>Reporter: Takahiko Saito
>Assignee: Prasanth Jayachandran
>Priority: Critical
> Attachments: HIVE-14397.1.patch
>
>
> Say we have configured hive.server2.tez.default.queues with 2 queues q1 and
> q2 with default expiry interval of 5 mins.
> After 5 mins of non-usage the sessions corresponding to queues q1 and q2 will
> be expired. When new set of queries are issue after this expiry, the default
> sessions backed by q1 and q2 and reopened again. Now when we run more queries
> the reopened sessions are not used instead new session is opened.
> At this point there will be 4 sessions running (2 abandoned sessions and 2
> current sessions).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14397) Queries launched after reopening of tez session launches additional sessions
[ https://issues.apache.org/jira/browse/HIVE-14397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15402503#comment-15402503 ] Sergey Shelukhin commented on HIVE-14397: - Which part of the patch fixes the problem where "when we run more queries the reopened sessions are not used instead new session is opened"? Queue name logic makes sense to me > Queries launched after reopening of tez session launches additional sessions > > > Key: HIVE-14397 > URL: https://issues.apache.org/jira/browse/HIVE-14397 > Project: Hive > Issue Type: Bug > Components: Tez >Affects Versions: 2.1.0, 2.2.0 >Reporter: Takahiko Saito >Assignee: Prasanth Jayachandran >Priority: Critical > Attachments: HIVE-14397.1.patch > > > Say we have configured hive.server2.tez.default.queues with 2 queues q1 and > q2 with default expiry interval of 5 mins. > After 5 mins of non-usage the sessions corresponding to queues q1 and q2 will > be expired. When new set of queries are issue after this expiry, the default > sessions backed by q1 and q2 and reopened again. Now when we run more queries > the reopened sessions are not used instead new session is opened. > At this point there will be 4 sessions running (2 abandoned sessions and 2 > current sessions). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
