Github user caneGuy commented on the issue:
https://github.com/apache/spark/pull/21128
Got it,thanks @vanzin
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail:
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21128
See my last comment. You don't need code in Spark to achieve what you want
to do.
---
-
To unsubscribe, e-mail:
Github user caneGuy commented on the issue:
https://github.com/apache/spark/pull/21128
@vanzin Thanks!
Since we set the common staging dir as
```
hdfs://xxx/spark/xxx/staging
```
This patch will give us a directory for given user like:
```
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21128
Moreover, you can set `spark.yarn.stagingDir` to whatever you want, and may
even reference env variables or system properties. e.g.
```
spark.yarn.stagingDir=/tmp/${system:user.name}
Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21128
> When we have multiple users on the same cluster
Then the staging directory would be created under the respective user
already (it's created under the user's home directory). I have no idea
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21128
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/89715/
Test PASSed.
---
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21128
Merged build finished. Test PASSed.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21128
**[Test build #89715 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89715/testReport)**
for PR 21128 at commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/21128
**[Test build #89715 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/89715/testReport)**
for PR 21128 at commit
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/21128
Can one of the admins verify this patch?
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional
10 matches
Mail list logo
