Hi Jean, How about using sc.setLogLevel("WARN") ? You may add this statement after initializing the Spark Context.
>From the Spark API - "Valid log levels include: ALL, DEBUG, ERROR, FATAL, INFO, OFF, TRACE, WARN". Here's the link in the Spark API. http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.SparkContext Hope this helps, Anupam On Mon, Jul 4, 2016 at 2:18 PM, Jean Georges Perrin <j...@jgp.net> wrote: > Thanks Mich, but what is SPARK_HOME when you run everything through Maven? > > On Jul 4, 2016, at 5:12 PM, Mich Talebzadeh <mich.talebza...@gmail.com> > wrote: > > check %SPARK_HOME/conf > > copy file log4j.properties.template to log4j.properties > > edit log4j.properties and set the log levels to your needs > > cat log4j.properties > > # Set everything to be logged to the console > log4j.rootCategory=ERROR, console > log4j.appender.console=org.apache.log4j.ConsoleAppender > log4j.appender.console.target=System.err > log4j.appender.console.layout=org.apache.log4j.PatternLayout > log4j.appender.console.layout.ConversionPattern=%d{yy/MM/dd HH:mm:ss} %p > %c{1}: %m%n > # Settings to quiet third party logs that are too verbose > log4j.logger.org.spark-project.jetty=WARN > log4j.logger.org.spark-project.jetty.util.component.AbstractLifeCycle=ERROR > log4j.logger.org.apache.spark.repl.SparkIMain$exprTyper=INFO > log4j.logger.org.apache.spark.repl.SparkILoop$SparkILoopInterpreter=INFO > log4j.logger.org.apache.parquet=ERROR > log4j.logger.parquet=ERROR > # SPARK-9183: Settings to avoid annoying messages when looking up > nonexistent UDFs in SparkSQL with Hive support > log4j.logger.org.apache.hadoop.hive.metastore.RetryingHMSHandler=FATAL > log4j.logger.org.apache.hadoop.hive.ql.exec.FunctionRegistry=ERROR > > HTH > > > > Dr Mich Talebzadeh > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > http://talebzadehmich.wordpress.com > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > On 4 July 2016 at 21:56, Jean Georges Perrin <j...@jgp.net> wrote: > >> Hi, >> >> I have installed Apache Spark via Maven. >> >> How can I control the volume of log it displays on my system? I tried >> different location for a log4j.properties, but none seems to work for me. >> >> Thanks for help... >> --------------------------------------------------------------------- >> To unsubscribe e-mail: user-unsubscr...@spark.apache.org >> >> > >