[
https://issues.apache.org/jira/browse/FLINK-23169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17431102#comment-17431102
]
Xintong Song commented on FLINK-23169:
--------------------------------------
[~hackergin],
I'm still not convinced that this is a common requirement. IMHO, grouping the
staging directories by user is just one option. There could be others, e.g.,
grouping by user-group, yarn queue, submission time, etc.
My concern on this is that, we may make it convenient for some use cases, at
the cost of complicating things for others. Leaving it completely to users or
external platforms is probably the most flexible approach.
WDYT?
> Support user-level app staging directory when yarn.staging-directory is
> specified
> ---------------------------------------------------------------------------------
>
> Key: FLINK-23169
> URL: https://issues.apache.org/jira/browse/FLINK-23169
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
> Reporter: jinfeng
> Priority: Major
>
> When yarn.staging-directory is specified, different users will use the same
> directory as the staging directory. It may not friendly for a job platform
> to submit job for different users. I propose to use the user-level directory
> by default when yarn.staging-directory is specified. We only need to make
> small changes for `getStagingDir` function in
> YarnClusterDescriptor
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
