Github user markgrover commented on a diff in the pull request:
https://github.com/apache/spark/pull/15971#discussion_r89383168
--- Diff:
core/src/main/scala/org/apache/spark/scheduler/EventLoggingListener.scala ---
@@ -231,6 +233,19 @@ private[spark] class EventLoggingListener(
}
}
+
+ private def redactEvent(event: SparkListenerEnvironmentUpdate):
SparkListenerEnvironmentUpdate = {
--- End diff --
>That's the only argument I can buy, and said user has different ways to
get that information.
True, `sparkConf.getAll` is always available.
I still think it's better to put the redaction closer to the "sinks" for
now. The good thing though is that if we see the number of "sinks" increasing,
and everyone wanting redaction, we can take redaction more upstream. For now, 2
sinks seem manageable and it's hard to guess if future sinks are going to want
redaction or not. So, unless you strongly object, I'd like to keep the
redaction closer to the "sinks" now.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
