[ https://issues.apache.org/jira/browse/SPARK-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tony Stevenson updated SPARK-1071: ---------------------------------- Assignee: Sean Owen (was: Sean Owen) > Tidy logging strategy and use of log4j > -------------------------------------- > > Key: SPARK-1071 > URL: https://issues.apache.org/jira/browse/SPARK-1071 > Project: Spark > Issue Type: Improvement > Components: Build, Input/Output > Affects Versions: 0.9.0 > Reporter: Sean Owen > Assignee: Sean Owen > Priority: Minor > Fix For: 1.0.0 > > > Prompted by a recent thread on the mailing list, I tried and failed to see if > Spark can be made independent of log4j. There are a few cases where control > of the underlying logging is pretty useful, and to do that, you have to bind > to a specific logger. > Instead I propose some tidying that leaves Spark's use of log4j, but gets rid > of warnings and should still enable downstream users to switch. The idea is > to pipe everything (except log4j) through SLF4J, and have Spark use SLF4J > directly when logging, and where Spark needs to output info (REPL and tests), > bind from SLF4J to log4j. > This leaves the same behavior in Spark. It means that downstream users who > want to use something except log4j should: > - Exclude dependencies on log4j, slf4j-log4j12 from Spark > - Include dependency on log4j-over-slf4j > - Include dependency on another logger X, and another slf4j-X > - Recreate any log config that Spark does, that is needed, in the other > logger's config > That sounds about right. > Here are the key changes: > - Include the jcl-over-slf4j shim everywhere by depending on it in core. > - Exclude dependencies on commons-logging from third-party libraries. > - Include the jul-to-slf4j shim everywhere by depending on it in core. > - Exclude slf4j-* dependencies from third-party libraries to prevent > collision or warnings > - Added missing slf4j-log4j12 binding to GraphX, Bagel module tests > And minor/incidental changes: > - Update to SLF4J 1.7.5, which happily matches Hadoop 2’s version and is a > recommended update over 1.7.2 > - (Remove a duplicate HBase dependency declaration in SparkBuild.scala) > - (Remove a duplicate mockito dependency declaration that was causing > warnings and bugging me) > Pull request coming. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org