Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/14185
I tried playing a little bit with what this API would look like, and I'm
starting to question my previous idea that exposing the ProcessBuilder is the
way to go here...
The above issue with logging redirection is just one source of issues.
There are other problems, such as the API becoming a little bit convoluted:
```
SparkLauncher launcher = ...;
ProcessBuilder pb = launcher.createProcessBuilder();
launcher.startApplication(pb);
```
And all the different ways to start the Spark app (3 different methods in
SparkLauncher + `ProcessBuilder.start()`). At this point I'm starting to think
it might be better to mirror parts of the ProcessBuilder API that are
interesting. e.g., have:
```
SparkLauncher directory(File directory)
SparkLauncher redirectErrorStream(boolean redirectErrorStream)
SparkLauncher redirectError(ProcessBuilder.Redirect destination)
SparkLauncher redirectOutput(ProcessBuilder.Redirect destination)
```
Optionally these (since you can use `Redirect.to(File)`):
```
SparkLauncher redirectError(File destination)
SparkLauncher redirectOutput(File destination)
```
And add this one which implements the current logger redirection:
```
SparkLauncher redirectToLog(String loggerName)
```
By default logging redirection would be done when using `startApplication`,
using the current semantics, unless the user has overridden that by calling one
of the new methods (which would also apply to `launch`).
This adds more methods and is a bit more work, but it avoids certains
oddities in the API, avoids overloading `startApplication`, and hides
ProcessBuilder APIs we don't want to expose (like `command()`). What do you
think?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
