Re: Spark streaming on YARN?

[Neal Sidhwaney]

On Fri, Jan 10, 2014 at 6:04 AM, Mike Percy <mpe...@apache.org> wrote:

>
> While it was difficult to get the logs from YARN before it deleted them
> during job cleanup, I finally did and all I got was this from stderr
> (stdout file was empty):
>
>
> Are you setting the SPARK_JAVA_OPTS environment variable? Under 0.8.1, we
had to set that to -Dlog4j.configuration=<path to logging properties file>
to get STDOUT for workers.  This was in standalone mode, though not with
YARN, so YMMV.
Neal