[jira] [Commented] (HIVE-9625) Delegation tokens for HMS are not renewed
[
https://issues.apache.org/jira/browse/HIVE-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14330449#comment-14330449
]
Brock Noland commented on HIVE-9625:
I think that getting a new token on failure is going to be pretty difficult.
The only places I can see retrying are in the metastore package in {{HMSC}} or
{{RetryingMetastore}} but there is no way to get a new token there.
Additionally I believe the token needs to be acquired outside of a
{{doas(user)}} call.
Looks like a non-trivial change.
Delegation tokens for HMS are not renewed
-
Key: HIVE-9625
URL: https://issues.apache.org/jira/browse/HIVE-9625
Project: Hive
Issue Type: Bug
Components: HiveServer2
Reporter: Brock Noland
Assignee: Brock Noland
Attachments: HIVE-9625.1.patch
AFAICT the delegation tokens stored in [HiveSessionImplwithUGI
|https://github.com/apache/hive/blob/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java#L45]
for HMS + Impersonation are never renewed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9625) Delegation tokens for HMS are not renewed
[
https://issues.apache.org/jira/browse/HIVE-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14329278#comment-14329278
]
Brock Noland commented on HIVE-9625:
Before calling {{getDelegationToken}} we call {{Hive.closeCurrent}} for this
reason. I'll test it and see what happens.
Delegation tokens for HMS are not renewed
-
Key: HIVE-9625
URL: https://issues.apache.org/jira/browse/HIVE-9625
Project: Hive
Issue Type: Bug
Components: HiveServer2
Reporter: Brock Noland
Assignee: Brock Noland
Attachments: HIVE-9625.1.patch
AFAICT the delegation tokens stored in [HiveSessionImplwithUGI
|https://github.com/apache/hive/blob/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java#L45]
for HMS + Impersonation are never renewed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9625) Delegation tokens for HMS are not renewed
[
https://issues.apache.org/jira/browse/HIVE-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14314028#comment-14314028
]
Hive QA commented on HIVE-9625:
---
{color:green}Overall{color}: +1 all checks pass
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12697589/HIVE-9625.1.patch
{color:green}SUCCESS:{color} +1 7540 tests passed
Test results:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2736/testReport
Console output:
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/2736/console
Test logs:
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-2736/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12697589 - PreCommit-HIVE-TRUNK-Build
Delegation tokens for HMS are not renewed
-
Key: HIVE-9625
URL: https://issues.apache.org/jira/browse/HIVE-9625
Project: Hive
Issue Type: Bug
Components: HiveServer2
Reporter: Brock Noland
Assignee: Brock Noland
Attachments: HIVE-9625.1.patch
AFAICT the delegation tokens stored in [HiveSessionImplwithUGI
|https://github.com/apache/hive/blob/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java#L45]
for HMS + Impersonation are never renewed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9625) Delegation tokens for HMS are not renewed
[ https://issues.apache.org/jira/browse/HIVE-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14315237#comment-14315237 ] Prasad Mujumdar commented on HIVE-9625: --- [~brocknoland] Are you able to verify the patch on with secure HS2, HMS ? I suspect that just calling getDelegationToken() on error won't help. AFAIR the expired tokens are not auto renewed. There's a GC thread that runs every hour to remove the expired token. You might have to renew or drop the token before re acquiring it. Delegation tokens for HMS are not renewed - Key: HIVE-9625 URL: https://issues.apache.org/jira/browse/HIVE-9625 Project: Hive Issue Type: Bug Components: HiveServer2 Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-9625.1.patch AFAICT the delegation tokens stored in [HiveSessionImplwithUGI |https://github.com/apache/hive/blob/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java#L45] for HMS + Impersonation are never renewed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9625) Delegation tokens for HMS are not renewed
[ https://issues.apache.org/jira/browse/HIVE-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14312777#comment-14312777 ] Brock Noland commented on HIVE-9625: [~thejas] [~prasadm] - do agree that these tokens do not have a renew process at present? If so, I think we can add a background thread to renew them in [SessionManager|https://github.com/apache/hive/blob/trunk/service/src/java/org/apache/hive/service/cli/session/SessionManager.java#L150]. Delegation tokens for HMS are not renewed - Key: HIVE-9625 URL: https://issues.apache.org/jira/browse/HIVE-9625 Project: Hive Issue Type: Bug Components: HiveServer2 Reporter: Brock Noland AFAICT the delegation tokens stored in [HiveSessionImplwithUGI |https://github.com/apache/hive/blob/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java#L45] for HMS + Impersonation are never renewed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9625) Delegation tokens for HMS are not renewed
[ https://issues.apache.org/jira/browse/HIVE-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14312907#comment-14312907 ] Thejas M Nair commented on HIVE-9625: - The renewal of credentials done in HIVE-4233 is similar to how I believe its done in HDFS/MR/Yarn clients. Delegation tokens for HMS are not renewed - Key: HIVE-9625 URL: https://issues.apache.org/jira/browse/HIVE-9625 Project: Hive Issue Type: Bug Components: HiveServer2 Reporter: Brock Noland AFAICT the delegation tokens stored in [HiveSessionImplwithUGI |https://github.com/apache/hive/blob/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java#L45] for HMS + Impersonation are never renewed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9625) Delegation tokens for HMS are not renewed
[ https://issues.apache.org/jira/browse/HIVE-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14312902#comment-14312902 ] Thejas M Nair commented on HIVE-9625: - Yes, this does look like a problem. Regarding the way to renew it, I think it would be better if we can renew it from a HMS client implementation on a failure-retry, similar to how reloginFromKeyTab was added to the client in HIVE-4233. This way any client of HMS could potentially benefit from this change. I haven't explored this fully, so I am not sure if there are any impediments for this, this is just something to consider. Delegation tokens for HMS are not renewed - Key: HIVE-9625 URL: https://issues.apache.org/jira/browse/HIVE-9625 Project: Hive Issue Type: Bug Components: HiveServer2 Reporter: Brock Noland AFAICT the delegation tokens stored in [HiveSessionImplwithUGI |https://github.com/apache/hive/blob/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java#L45] for HMS + Impersonation are never renewed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9625) Delegation tokens for HMS are not renewed
[ https://issues.apache.org/jira/browse/HIVE-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14313065#comment-14313065 ] Brock Noland commented on HIVE-9625: I think the attached might work. Delegation tokens for HMS are not renewed - Key: HIVE-9625 URL: https://issues.apache.org/jira/browse/HIVE-9625 Project: Hive Issue Type: Bug Components: HiveServer2 Reporter: Brock Noland Assignee: Brock Noland Attachments: HIVE-9625.1.patch AFAICT the delegation tokens stored in [HiveSessionImplwithUGI |https://github.com/apache/hive/blob/trunk/service/src/java/org/apache/hive/service/cli/session/HiveSessionImplwithUGI.java#L45] for HMS + Impersonation are never renewed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
