Re: SparkSubmit.scala and stderr

2015-02-03 Thread Evan Chan
Why not just use SLF4J? On Tue, Feb 3, 2015 at 2:22 PM, Reynold Xin wrote: > We can use ScalaTest's privateMethodTester also instead of exposing that. > > On Tue, Feb 3, 2015 at 2:18 PM, Marcelo Vanzin wrote: > >> Hi Jay, >> >> On Tue, Feb 3, 2015 at 6:28 AM, jayhutfles wrote: >> > // Expos

Re: SparkSubmit.scala and stderr

2015-02-03 Thread Reynold Xin
We can use ScalaTest's privateMethodTester also instead of exposing that. On Tue, Feb 3, 2015 at 2:18 PM, Marcelo Vanzin wrote: > Hi Jay, > > On Tue, Feb 3, 2015 at 6:28 AM, jayhutfles wrote: > > // Exposed for testing > > private[spark] var printStream: PrintStream = System.err > > > B

Re: SparkSubmit.scala and stderr

2015-02-03 Thread Marcelo Vanzin
Hi Jay, On Tue, Feb 3, 2015 at 6:28 AM, jayhutfles wrote: > // Exposed for testing > private[spark] var printStream: PrintStream = System.err > But as the comment states that it's for testing, maybe I'm > misunderstanding its intent... The comment is there to tell someone reading the co

Re: SparkSubmit.scala and stderr

2015-02-03 Thread Sean Owen
Despite its name, stderr is frequently used as the destination for anything that's not the output of the program, which includes log messages. That way, for example, you can redirect the output of such a program to capture its result without also capturing log or error messages, which will still ju

SparkSubmit.scala and stderr

2015-02-03 Thread jayhutfles
Hi all, I just saw that the SparkSubmit.scala class has the following lines: object SparkSubmit { ... // Exposed for testing private[spark] var printStream: PrintStream = System.err ... } This causes all verbose logging messages elsewhere in SparkSubmit to go to stderr, not s