[
https://issues.apache.org/jira/browse/SPARK-1071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tony Stevenson updated SPARK-1071:
----------------------------------
Reporter: Sean Owen (was: Sean Owen)
> Tidy logging strategy and use of log4j
> --------------------------------------
>
> Key: SPARK-1071
> URL: https://issues.apache.org/jira/browse/SPARK-1071
> Project: Spark
> Issue Type: Improvement
> Components: Build, Input/Output
> Affects Versions: 0.9.0
> Reporter: Sean Owen
> Assignee: Sean Owen
> Priority: Minor
> Fix For: 1.0.0
>
>
> Prompted by a recent thread on the mailing list, I tried and failed to see if
> Spark can be made independent of log4j. There are a few cases where control
> of the underlying logging is pretty useful, and to do that, you have to bind
> to a specific logger.
> Instead I propose some tidying that leaves Spark's use of log4j, but gets rid
> of warnings and should still enable downstream users to switch. The idea is
> to pipe everything (except log4j) through SLF4J, and have Spark use SLF4J
> directly when logging, and where Spark needs to output info (REPL and tests),
> bind from SLF4J to log4j.
> This leaves the same behavior in Spark. It means that downstream users who
> want to use something except log4j should:
> - Exclude dependencies on log4j, slf4j-log4j12 from Spark
> - Include dependency on log4j-over-slf4j
> - Include dependency on another logger X, and another slf4j-X
> - Recreate any log config that Spark does, that is needed, in the other
> logger's config
> That sounds about right.
> Here are the key changes:
> - Include the jcl-over-slf4j shim everywhere by depending on it in core.
> - Exclude dependencies on commons-logging from third-party libraries.
> - Include the jul-to-slf4j shim everywhere by depending on it in core.
> - Exclude slf4j-* dependencies from third-party libraries to prevent
> collision or warnings
> - Added missing slf4j-log4j12 binding to GraphX, Bagel module tests
> And minor/incidental changes:
> - Update to SLF4J 1.7.5, which happily matches Hadoop 2’s version and is a
> recommended update over 1.7.2
> - (Remove a duplicate HBase dependency declaration in SparkBuild.scala)
> - (Remove a duplicate mockito dependency declaration that was causing
> warnings and bugging me)
> Pull request coming.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
