[
https://issues.apache.org/jira/browse/FLINK-23169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17376979#comment-17376979
]
Yang Wang commented on FLINK-23169:
-----------------------------------
Thanks [~hackergin] for proposing this ticket. Let me try to understand your
use case.
By default, Flink will create the staging directory in the user home
directory(e.g. /user/admin/.flink). I remember some users want to customize the
staging directory, which is shared by all users, so we introduce the config
option {{yarn.staging-directory}}.
I am not sure why you are not using the default user home directory. Moreover,
the user home directory could also be configured via {{dfs.user.home.base.dir}}
in hdfs-site.xml.
> Support user-level app staging directory when yarn.staging-directory is
> specified
> ---------------------------------------------------------------------------------
>
> Key: FLINK-23169
> URL: https://issues.apache.org/jira/browse/FLINK-23169
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
> Reporter: jinfeng
> Priority: Major
>
> When yarn.staging-directory is specified, different users will use the same
> directory as the staging directory. It may not friendly for a job platform
> to submit job for different users. I propose to use the user-level directory
> by default when yarn.staging-directory is specified. We only need to make
> small changes for `getStagingDir` function in
> YarnClusterDescriptor
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
