[jira] [Commented] (OOZIE-2797) Add a framework for cancellation of Delegation Tokens in Oozie AM
[ https://issues.apache.org/jira/browse/OOZIE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870498#comment-15870498 ] Rohini Palaniswamy commented on OOZIE-2797: --- bq. Will file a follow-up on OOZIE-1770 to create a framework similar to CredentialsProvider in the new LauncherAM. Please do not create another jira. This jira is for that. > Add a framework for cancellation of Delegation Tokens in Oozie AM > - > > Key: OOZIE-2797 > URL: https://issues.apache.org/jira/browse/OOZIE-2797 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Fix For: 5.0.0 > > Attachments: OOZIE-2797-reference.patch > > > Pig and hive clients when run from command line take care of canceling > HCatalog delegation tokens. But they do not get cancelled when run via Oozie. > RM only supports cancellation of hdfs and yarn tokens on job completion. So > when running with Oozie, it is left to the HCatalog server to remove the > token after the expiry. When there are too many jobs, the number of unexpired > tokens can overwhelm the hcat server and cause outages. Oozie should take > care of canceling these tokens after the job is done to avoid that. > We should add a token cancellation framework in the new Oozie AM to support > implementations for token cancellation for different credential types similar > to CredentialsProvider. >Attaching a reference patch that we used internally for directly canceling > hcat tokens for just pig and hive actions that could be used by anyone who > needs it till OOZIE-1770 is done and the new token cancellation framework is > added by this jira. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (OOZIE-2797) Add a framework for cancellation of Delegation Tokens in Oozie AM
[
https://issues.apache.org/jira/browse/OOZIE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15870493#comment-15870493
]
Andras Piros commented on OOZIE-2797:
-
[~rohini] thanks for clarifying this! Will file a follow-up on OOZIE-1770 to
create a framework similar to {{CredentialsProvider}} in the new {{LauncherAM}}.
[~satishsaley] +1 (non-binding)
> Add a framework for cancellation of Delegation Tokens in Oozie AM
> -
>
> Key: OOZIE-2797
> URL: https://issues.apache.org/jira/browse/OOZIE-2797
> Project: Oozie
> Issue Type: Bug
>Reporter: Satish Subhashrao Saley
>Assignee: Satish Subhashrao Saley
> Fix For: 5.0.0
>
> Attachments: OOZIE-2797-reference.patch
>
>
> Pig and hive clients when run from command line take care of canceling
> HCatalog delegation tokens. But they do not get cancelled when run via Oozie.
> RM only supports cancellation of hdfs and yarn tokens on job completion. So
> when running with Oozie, it is left to the HCatalog server to remove the
> token after the expiry. When there are too many jobs, the number of unexpired
> tokens can overwhelm the hcat server and cause outages. Oozie should take
> care of canceling these tokens after the job is done to avoid that.
> We should add a token cancellation framework in the new Oozie AM to support
> implementations for token cancellation for different credential types similar
> to CredentialsProvider.
>Attaching a reference patch that we used internally for directly canceling
> hcat tokens for just pig and hive actions that could be used by anyone who
> needs it till OOZIE-1770 is done and the new token cancellation framework is
> added by this jira.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2797) Add a framework for cancellation of Delegation Tokens in Oozie AM
[ https://issues.apache.org/jira/browse/OOZIE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15868325#comment-15868325 ] Rohini Palaniswamy commented on OOZIE-2797: --- bq. Maybe a follow-up JIRA should be linked to this one to be clear about what is the purpose of OOZIE-2797 and what will be handled separately. Updated title and description of this jira to be more clear. > Add a framework for cancellation of Delegation Tokens in Oozie AM > - > > Key: OOZIE-2797 > URL: https://issues.apache.org/jira/browse/OOZIE-2797 > Project: Oozie > Issue Type: Bug >Reporter: Satish Subhashrao Saley >Assignee: Satish Subhashrao Saley > Fix For: 5.0.0 > > Attachments: OOZIE-2797-1.patch > > > Pig and hive clients when run from command line take care of canceling > HCatalog delegation tokens. But they do not get cancelled when run via Oozie. > RM only supports cancellation of hdfs and yarn tokens on job completion. So > when running with Oozie, it is left to the HCatalog server to remove the > token after the expiry. When there are too many jobs, the number of unexpired > tokens can overwhelm the hcat server and cause outages. Oozie should take > care of canceling these tokens after the job is done to avoid that. > We should add a token cancellation framework in the new Oozie AM to support > implementations for token cancellation for different credential types similar > to CredentialsProvider. >Attaching a reference patch that we used internally for directly canceling > hcat tokens for just pig and hive actions that could be used by anyone who > needs it till OOZIE-1770 is done and the new token cancellation framework is > added by this jira. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
