[
https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651657#comment-16651657
]
Thomas Graves commented on SPARK-25732:
---------------------------------------
So like Marcelo mentioned can't you re-use the keytab/principal option already
there? It might need slightly modified to pull from HDFS but that is really
what this is doing, its just livy is submitting the job for you. Really the
user could specify it when submitting the job as a conf (? I guess depends on
who is calling livy, jupyter for instance definitely could as user can pass
configs). I would prefer that over adding more configs.
There are lots of cases things are in the middle of job submission, livy,
oozie, other workflow managers. I don't see that as a reason not to do tokens.
User should know they are submitting jobs (especially one that runs for 2
weeks) and until we have a good automated solution, they would have to setup
cron or something else to push tokens before they expire. I know the YARN
folks were looking at options to help with this but haven't synced with them
lately as ideally there would be a way to push the tokens to the RM for it to
continue to renew so you would only have to do it before max lifetime. Its
easy enough to write a script that runs and does a list of applications running
for the user and push tokens to each of those, assuming we had spark-submit
option to push tokens.
> Allow specifying a keytab/principal for proxy user for token renewal
> ---------------------------------------------------------------------
>
> Key: SPARK-25732
> URL: https://issues.apache.org/jira/browse/SPARK-25732
> Project: Spark
> Issue Type: Improvement
> Components: Deploy
> Affects Versions: 2.4.0
> Reporter: Marco Gaido
> Priority: Major
>
> As of now, application submitted with proxy-user fail after 2 week due to the
> lack of token renewal. In order to enable it, we need the the
> keytab/principal of the impersonated user to be specified, in order to have
> them available for the token renewal.
> This JIRA proposes to add two parameters {{--proxy-user-principal}} and
> {{--proxy-user-keytab}}, and the last letting a keytab being specified also
> in a distributed FS, so that applications can be submitted by servers (eg.
> Livy, Zeppelin) without needing all users' principals being on that machine.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
