vanzin commented on a change in pull request #23675: [SPARK-26753][CORE] Fixed
custom log levels for spark-shell by using Filter instead of Threshold
URL: https://github.com/apache/spark/pull/23675#discussion_r252011626
##########
File path: core/src/main/scala/org/apache/spark/internal/Logging.scala
##########
@@ -229,3 +237,33 @@ private[spark] object Logging {
"org.slf4j.impl.Log4jLoggerFactory".equals(binderClass)
}
}
+
+private class SparkShellLoggingFilter() extends Filter {
+
+ /**
+ * If sparkShellThresholdLevel is not defined, this filter is a no-op.
+ * If log level of event is lower than thresholdLevel, then the decision is
made based on
+ * whether the log came from root or some custom configuration
+ * @param loggingEvent
+ * @return decision for accept/deny log event
+ */
+ def decide(loggingEvent: LoggingEvent): Int = {
+ val thresholdLevel = Logging.sparkShellThresholdLevel
+ if (thresholdLevel == null) {
+ return Filter.NEUTRAL
+ }
+ val rootLevel = LogManager.getRootLogger().getLevel()
+ if (loggingEvent.getLevel().isGreaterOrEqual(thresholdLevel) ||
+ !loggingEvent.getLevel().eq(rootLevel)) {
Review comment:
That's my question. Doesn't `!loggingEvent.getLevel().eq(rootLevel)` already
cover that?
The other check is only an optimization if `thresholdLevel` is the same as
`rootLevel`, and in that case a better optimization would be to not set
`thresholdLevel` (so the null check triggers instead).
Using the good old truth table:
```
Level Thresh Root Action
INFO WARN INFO Check for override
WARN WARN INFO Allow
DEBUG WARN INFO Allow
```
So the resulting action doesn't seem dependent on the threshold level at
all, assuming it's always different than the root logger's level.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
