Khanderao, I think Shay is talking about the details of how to deploy / where to put the file, not the directives to put inside the log4j file.
I also believe that the docs are wrong w.r.t. log4j, particularly for the YARN standalone instructions @ https://spark.incubator.apache.org/docs/latest/running-on-yarn.html where it says: # Configure logging $ cp conf/log4j.properties.template conf/log4j.properties Maybe this works / worked for some versions of Spark and/or YARN, just not the ones I am using. I'd guess there is some classpath / set-up script related bug somewhere, or maybe some issue with deploying the file to the cluster via the distributed cache mechanism. Or maybe I just missed a step somewhere, but I tried a bunch of things. Mike On Fri, Jan 10, 2014 at 1:36 PM, Khanderao kand <[email protected]>wrote: > Shay, > > This is much more log4j question than a Spark question. > > > Anyway, Copying the templates would enable root logger at INFO level. > > Are you interested in suppressing logs? "However this did not suppress > any logging. " If you want to suppress logging, trying setting to OFF > > try > > log4j.rootCategory=OFF > > Similarly make jetty logger off too. > > > > Khanderao > > > On Thu, Jan 9, 2014 at 10:49 AM, Shay Seng <[email protected]> wrote: > >> Hey, >> >> In " >> https://spark.incubator.apache.org/docs/0.8.1/configuration.html#configuring-logging >> " >> It states: >> *"Spark uses log4j <http://logging.apache.org/log4j/> for logging. You >> can configure it by adding a log4j.properties file in the conf directory. >> One way to start is to copy the existing log4j.properties.template located >> there."* >> >> I did as specified and added a log4j.properties file in the spark conf >> directory. However this did not suppress any logging. >> However if I launch spark from a directory that contains a >> log4j.properties file, it does get read in. >> >> Does the docs need re-writing, or is this a bug? >> >> tks, >> Shay >> >> >> >
