I run scala code from zeppelin notebook, when I look at spark log from spark/work/app-20150901193944-0015/0/stderr Log file looks like: SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/zhaoxq/Downloads/zeppelin2/incubator-zeppelin/interpreter/spark/zeppelin-spark-0.6.0-incubating-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/zhaoxq/Downloads/zeppelin2/incubator-zeppelin/zeppelin-interpreter/target/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/zhaoxq/spark/assembly/target/scala-2.10/spark-assembly-1.5.0-SNAPSHOT-hadoop2.2.0.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
But if I run same code from spark-shell, the log file containts correct spark log info: 15/09/01 19:31:58 INFO CoarseGrainedExecutorBackend: Registered signal handlers for [TERM, HUP, INT] 15/09/01 19:31:58 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 15/09/01 19:31:59 INFO SecurityManager: Changing view acls to: zhaoxq 15/09/01 19:31:59 INFO SecurityManager: Changing modify acls to: zhaoxq 15/09/01 19:31:59 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(zhaoxq); users with modify permissions: Set(zhaoxq) 15/09/01 19:31:59 INFO Slf4jLogger: Slf4jLogger started ................. ................ Also, if I rebuild zeppelin with remote false mode, which is: change setting.getOption().setRemote(false); in org.apache.zeppelin.interpreter.InterpreterFactory, spark worker log is same as running from spark-shell, which is what I want. So, my question is how can I get useful spark worker log with zeppelin remote mode? Thanks.
