Github user andreweduffy commented on a diff in the pull request:
https://github.com/apache/spark/pull/14201#discussion_r71031747
--- Diff:
launcher/src/main/java/org/apache/spark/launcher/SparkLauncher.java ---
@@ -359,6 +369,83 @@ public SparkLauncher setVerbose(boolean verbose) {
}
/**
+ * Sets the working directory of spark-submit.
+ *
+ * @param dir The directory to set as spark-submit's working directory.
+ * @return This launcher.
+ */
+ public SparkLauncher directory(File dir) {
+ workingDir = dir;
+ return this;
+ }
+
+ /**
+ * Specifies that stderr in spark-submit should be redirected to stdout.
+ *
+ * @return This launcher.
+ */
+ public SparkLauncher redirectError() {
+ redirectErrorStream = true;
+ return this;
+ }
+
+ /**
+ * Redirects error output to the specified Redirect.
+ *
+ * @param to The method of redirection.
+ * @return This launcher.
+ */
+ public SparkLauncher redirectError(ProcessBuilder.Redirect to) {
+ errorStream = to;
+ return this;
+ }
+
+ /**
+ * Redirects standard output to the specified Redirect.
+ *
+ * @param to The method of redirection.
+ * @return This launcher.
+ */
+ public SparkLauncher redirectOutput(ProcessBuilder.Redirect to) {
+ outputStream = to;
+ return this;
+ }
+
+ /**
+ * Redirects error output to the specified File.
+ *
+ * @param errFile The file to which stderr is written.
+ * @return This launcher.
+ */
+ public SparkLauncher redirectError(File errFile) {
+ errorStream = ProcessBuilder.Redirect.to(errFile);
+ return this;
+ }
+
+ /**
+ * Redirects error output to the specified File.
+ *
+ * @param outFile The file to which stdout is written.
+ * @return This launcher.
+ */
+ public SparkLauncher redirectOutput(File outFile) {
+ outputStream = ProcessBuilder.Redirect.to(outFile);
+ return this;
+ }
+
+ /**
+ * Sets all output to be logged and redirected to a logger with the
specified name.
+ *
+ * @param loggerName The name of the logger to log stdout and stderr.
+ * @return This launcher.
+ */
+ public SparkLauncher redirectToLog(String loggerName) {
--- End diff --
So I had an extra check in the last commit, thought there was a comment
that it should be removed though, maybe I should just clarify here. I see 2
possible checks that we should be doing here:
* Ensure that only one of `redirectError(...)` and `redirectError()` can be
used (done)
* Ensure that `redirectToLog()` is never used with any other `redirect*`
method (removed in this diff, can be added back)
Are there others that I'm missing?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
