Are you running the code with yarn? if so, figure out the applicationID through the web ui, then run the next command:
yarn logs your_application_id Alonso Isidoro Roman [image: https://]about.me/alonso.isidoro.roman <https://about.me/alonso.isidoro.roman?promo=email_sig&utm_source=email_sig&utm_medium=email_sig&utm_campaign=external_links> 2017-05-31 9:42 GMT+02:00 Paolo Patierno <ppatie...@live.com>: > Hi all, > > > I have a simple cluster with one master and one worker. On another machine > I launch the driver where at some point I have following line of codes : > > > max.foreachRDD(rdd -> { > > LOG.info("*** max.foreachRDD"); > > rdd.foreach(value -> { > > LOG.info("*** rdd.foreach"); > > }); > }); > > > The message "*** max.foreachRDD" is visible in the console of the driver > machine ... and it's ok. > I can't see the "*** rdd.foreach" message that should be executed on the > worker node right ? Btw on the worker node console I can't see it. Why ? > > My need is to log what happens in the code executed on worker node > (because it works if I execute it on master local[*] but not when submitted > to a worker node). > > Following the log4j.properties file I put in the /conf dir : > > # Set everything to be logged to the console > log4j.rootCategory=INFO, console > log4j.appender.console=org.apache.log4j.ConsoleAppender > log4j.appender.console.target=System.err > log4j.appender.console.layout=org.apache.log4j.PatternLayout > log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p > %c{1}: %m%n > > # Settings to quiet third party logs that are too verbose > log4j.logger.org.spark-project.jetty=WARN > log4j.logger.org.spark-project.jetty.util.component.AbstractLifeCycle=ERROR > log4j.logger.org.apache.spark.repl.SparkIMain$exprTyper=INFO > log4j.logger.org.apache.spark.repl.SparkILoop$SparkILoopInterpreter=INFO > > > > Thanks > Paolo. > > > *Paolo Patierno* > > *Senior Software Engineer (IoT) @ Red Hat **Microsoft MVP on **Windows > Embedded & IoT* > *Microsoft Azure Advisor* > > Twitter : @ppatierno <http://twitter.com/ppatierno> > Linkedin : paolopatierno <http://it.linkedin.com/in/paolopatierno> > Blog : DevExperience <http://paolopatierno.wordpress.com/> >
