2017-05-31 10:48 GMT+02:00 Paolo Patierno <ppatie...@live.com>: > No it's running in standalone mode as Docker image on Kubernetes. > > > The only way I found was to access "stderr" file created under the "work" > directory in the SPARK_HOME but ... is it the right way ? >
I think that is the right way. I haven't looked in the documentation, but I think in a stand-alone cluster you have a master node, that manages your worker node, each node running one "management" process. When you submit a job, these management processes spawn "executor" processes which have stdout/-err going to $SPARK_HOME/work/…, but are not piped through the management processes. The logs should also be available through the web UI on port 8081 of the worker. Best Eike
