one more bit of info i also seem to have tracked down that spark is using slf4j and StaticLoggerBinder, so it would seem it is bypassing log4j factory for slf4j; but then still, log4j.properties are being picked up.
On the other when i force and hack slf4j to use log4j with mahout shell, the log4j.properties still are not being picked up. I have very little experience with slf4j, so this is all purplexing to no end. On Fri, Aug 15, 2014 at 4:59 PM, Ted Dunning <[email protected]> wrote: > One of our guys just had a very similar problem. I will step over and ask. > > > > On Fri, Aug 15, 2014 at 4:51 PM, Dmitriy Lyubimov <[email protected]> > wrote: > > > with spark shell. Trying to clamp down on the INFO logging to console. > > Can't seem to solve it. > > > > Since spark itself seems to add $SPARK_HOME/conf classpath (to verify > what > > mahout ends up with in this case, run bin/mahout -spark classpath | sed > > 's/:/\n/g'), one obvious solution is to modify spark's > > conf/log4j.properties itself. When i do that, debug output is clamped > down > > with spark shell, but not with Mahout's. > > > > I also did a hack and added log4j.properties into mahout's own classpath, > > to no avail either. > > > > So I can't seem to be able to track down this damn logger initialization > > issue, no matter what i do. it would seem Spark's elements of mahout > shell > > start identically, but for some reason log4j just refuses to initialize > > differently. > > > > this INFO stuff coming from spark is quite annoying in the shell, we need > > to be able to regulate console output of the log4j. any help with the > issue > > is appreciated. > > > > d > > >
