Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21158
> Not all the Spark end users are allowed to access the command lines and
run ls
Come on. That is just an example. You surely are smart enough to think of
another example where you can see other people's user names outside of Spark.
> it would be better to keep them unchanged, if possible
I disagree. Can you list all the Spark configuration that would be redacted
by the conf in 2.3? I could only find two:
* spark.driver.userClassPathFirsrt
* spark.executor.userClassPathFirst
And those are a great example of why the original change is a regression.
It's wrong to redact those. It makes debugging things harder.
And yet there are a ton of config keys where you can provide URLs, the
thing you seem really worried about, which are not caught by that regex. Let's
see... spark.master, spark.jars, spark.files, spark.yarn.archives,
spark.yarn.jars, spark.jars.repositories... I could go on.
> Since the new behaviors have already been introduced in the last release,
Every change introduces a behavior change (unless you're like changing a
comment string or something like that). Saying a change is not wanted because
it changes behavior is really quite silly.
The question is always whether the change is desired or not. And in this
case *it is*. The change that was introduced in 2.3.0 is a regression in
behavior from 2.2, or, in other words, it makes things *worse*. So we should
fix it.
> please document the behavior changes.
As I've stated in previous comments, I don't think there is a user-visible
behavior change - at least one users would care about. As I mentioned above,
the default value in 2.3.0 only causes problems, and doesn't apply to anything
useful other than the JDBC URL, a behavior that this change keeps.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
