[ 
https://issues.apache.org/jira/browse/SPARK-24053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marcelo Vanzin resolved SPARK-24053.
------------------------------------
    Resolution: Not A Bug

Copy & paste of PR comments:

{quote}
> When we have multiple users on the same cluster

Then the staging directory would be created under the respective user already 
(it's created under the user's home directory). I have no idea why you need 
this.

Moreover, you can set spark.yarn.stagingDir to whatever you want, and may even 
reference env variables or system properties. e.g.

spark.yarn.stagingDir=/tmp/${system:user.name}

And the staging directory will be created under that location. So really you 
don't need to write any code for this, even if you have a weird deployment 
where users don't have their own home directory.
{quote}

>  Support add subdirectory named as user name on staging directory
> -----------------------------------------------------------------
>
>                 Key: SPARK-24053
>                 URL: https://issues.apache.org/jira/browse/SPARK-24053
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 2.3.0
>            Reporter: zhoukang
>            Priority: Minor
>
> When we have multiple users on the same cluster,we can support add 
> subdirectory which named as login user



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to