ankuriitg commented on a change in pull request #23675: [SPARK-26753][CORE]
Fixed custom log levels for spark-shell by using Filter instead of Threshold
URL: https://github.com/apache/spark/pull/23675#discussion_r252008596
##########
File path: core/src/main/scala/org/apache/spark/internal/Logging.scala
##########
@@ -229,3 +237,33 @@ private[spark] object Logging {
"org.slf4j.impl.Log4jLoggerFactory".equals(binderClass)
}
}
+
+private class SparkShellLoggingFilter() extends Filter {
+
+ /**
+ * If sparkShellThresholdLevel is not defined, this filter is a no-op.
+ * If log level of event is lower than thresholdLevel, then the decision is
made based on
+ * whether the log came from root or some custom configuration
+ * @param loggingEvent
+ * @return decision for accept/deny log event
+ */
+ def decide(loggingEvent: LoggingEvent): Int = {
+ val thresholdLevel = Logging.sparkShellThresholdLevel
+ if (thresholdLevel == null) {
+ return Filter.NEUTRAL
+ }
+ val rootLevel = LogManager.getRootLogger().getLevel()
+ if (loggingEvent.getLevel().isGreaterOrEqual(thresholdLevel) ||
+ !loggingEvent.getLevel().eq(rootLevel)) {
Review comment:
So, the `loggingEvent.getLevel().isGreaterOrEqual(thresholdLevel)` is just
an optimization. If the log message level is same as root level but it is
greater than threshold level, then we still allow the event to pass through,
irrespective of whether it was customized or not.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
