[
https://issues.apache.org/jira/browse/FLINK-23169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17431022#comment-17431022
]
jinfeng commented on FLINK-23169:
---------------------------------
Hi [~xtsong]
1. Putting them together doesn't seem to be a big problem. But I think it is
more convenient to manage the quota size and permissions of the directory
separately by user.
2. Yes, we can configure different staging directories when submiting the
job, just like configue the checkpoint directory.
For a single user, it may not be a big problem. But for a flink job platform,
it should consider to manage the config for different users. So I think this
may be a general requirement. And here is the spark related jira .
https://issues.apache.org/jira/browse/SPARK-26877
> Support user-level app staging directory when yarn.staging-directory is
> specified
> ---------------------------------------------------------------------------------
>
> Key: FLINK-23169
> URL: https://issues.apache.org/jira/browse/FLINK-23169
> Project: Flink
> Issue Type: Improvement
> Components: Deployment / YARN
> Reporter: jinfeng
> Priority: Major
>
> When yarn.staging-directory is specified, different users will use the same
> directory as the staging directory. It may not friendly for a job platform
> to submit job for different users. I propose to use the user-level directory
> by default when yarn.staging-directory is specified. We only need to make
> small changes for `getStagingDir` function in
> YarnClusterDescriptor
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
