[jira] [Commented] (HIVE-15645) Tez session pool may restart sessions in a wrong queue
[
https://issues.apache.org/jira/browse/HIVE-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15828225#comment-15828225
]
Hive QA commented on HIVE-15645:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12847969/HIVE-15645.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10942 tests
executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out)
(batchId=234)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_llap_counters]
(batchId=136)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part]
(batchId=148)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
(batchId=139)
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerCheckInvocation.org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerCheckInvocation
(batchId=208)
org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerShowFilters.org.apache.hadoop.hive.ql.security.authorization.plugin.TestHiveAuthorizerShowFilters
(batchId=208)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3009/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3009/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3009/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12847969 - PreCommit-HIVE-Build
> Tez session pool may restart sessions in a wrong queue
> --
>
> Key: HIVE-15645
> URL: https://issues.apache.org/jira/browse/HIVE-15645
> Project: Hive
> Issue Type: Bug
>Reporter: Carter Shanklin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-15645.01.patch, HIVE-15645.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15645) Tez session pool may restart sessions in a wrong queue
[ https://issues.apache.org/jira/browse/HIVE-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15827428#comment-15827428 ] Gunther Hagleitner commented on HIVE-15645: --- +1 > Tez session pool may restart sessions in a wrong queue > -- > > Key: HIVE-15645 > URL: https://issues.apache.org/jira/browse/HIVE-15645 > Project: Hive > Issue Type: Bug >Reporter: Carter Shanklin >Assignee: Sergey Shelukhin > Attachments: HIVE-15645.01.patch, HIVE-15645.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15645) Tez session pool may restart sessions in a wrong queue
[ https://issues.apache.org/jira/browse/HIVE-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826708#comment-15826708 ] Sergey Shelukhin commented on HIVE-15645: - The unsetting of the config is done for sessions with queue specified by the user, which is not something supported by pool sessions. The exception case is to manifest future bugs (if any) better - it was not supposed to work, and thus worked improperly... > Tez session pool may restart sessions in a wrong queue > -- > > Key: HIVE-15645 > URL: https://issues.apache.org/jira/browse/HIVE-15645 > Project: Hive > Issue Type: Bug >Reporter: Carter Shanklin >Assignee: Sergey Shelukhin > Attachments: HIVE-15645.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15645) Tez session pool may restart sessions in a wrong queue
[ https://issues.apache.org/jira/browse/HIVE-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826692#comment-15826692 ] Gunther Hagleitner commented on HIVE-15645: --- This still might be one step forward two steps back. How do you know that clearing the conf wasn't done for some specific reason? Also, the exception nukes a case that used to work - what if that was done intentionally? > Tez session pool may restart sessions in a wrong queue > -- > > Key: HIVE-15645 > URL: https://issues.apache.org/jira/browse/HIVE-15645 > Project: Hive > Issue Type: Bug >Reporter: Carter Shanklin >Assignee: Sergey Shelukhin > Attachments: HIVE-15645.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15645) Tez session pool may restart sessions in a wrong queue
[ https://issues.apache.org/jira/browse/HIVE-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826628#comment-15826628 ] Sergey Shelukhin commented on HIVE-15645: - We had a repro on some cluster that indicates that the patch will fix the problem. It has to do with config being out of sync with the property. First session gets config and property correct, but something (I am pretty sure it's the unset in open path) resets the config. Then the 2nd session (after expiration) gets the property correct but the config is not set, so it logs as if it is going to correct queue but goes to wrong queue, which is what we have observed for a specific session. The field is also reset to null from conf (in a place where I added the warn log), after the log statement about the queue. The 3rd session (after the 2nd expiration) logs null queue (because the field is also null now), and goes to the wrong queue, as does every one after that. So, for pool sessions we set the session into conf every time now. I also added a warn log for the future, and a null check cause we never expect null queue for pool sessions. To fix this properly the separation of pool and non-pool sessions that was started at some point needs to be completed, but that's a major refactoring, not a bugfix. > Tez session pool may restart sessions in a wrong queue > -- > > Key: HIVE-15645 > URL: https://issues.apache.org/jira/browse/HIVE-15645 > Project: Hive > Issue Type: Bug >Reporter: Carter Shanklin >Assignee: Sergey Shelukhin > Attachments: HIVE-15645.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15645) Tez session pool may restart sessions in a wrong queue
[ https://issues.apache.org/jira/browse/HIVE-15645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15826617#comment-15826617 ] Gunther Hagleitner commented on HIVE-15645: --- Your comment says: // There's some bogus code that can modify the queue name. Force-set it for pool sessions. This fix sounds like a guess. Is there an actual test case to show what's being fixed? If there is bogus code, can we fix that? > Tez session pool may restart sessions in a wrong queue > -- > > Key: HIVE-15645 > URL: https://issues.apache.org/jira/browse/HIVE-15645 > Project: Hive > Issue Type: Bug >Reporter: Carter Shanklin >Assignee: Sergey Shelukhin > Attachments: HIVE-15645.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
