GitHub user tejasapatil opened a pull request:
https://github.com/apache/spark/pull/14205
[SPARK-16547] [CORE] EventLoggingListener to auto create log base dir if it
does not exist
## What changes were proposed in this pull request?
When the HDFS namenode gets changed or replaced, one has to manually create
the `spark.eventLog.dir` dir. This PR would change Spark to auto-create that
directory if its not present thus saving one manual step.
## How was this patch tested?
Changed the unit tests to not have the `spark.eventLog.dir` created as a
pre-step. I have ensured that the modified version of tests fail without the
fix.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tejasapatil/spark hdfs_auto_dir_creation
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14205.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14205
----
commit b5835525c64575516f46408d9eb1da7e4b768bf8
Author: Tejas Patil <[email protected]>
Date: 2016-07-14T15:44:42Z
EventLoggingListener to auto create log base dir if it does not exist
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
