Github user gatorsmile commented on the issue:
https://github.com/apache/spark/pull/21158
@vanzin That really depends on how the users deploy Spark. Not all the
Spark end users are allowed to access the command lines and run `ls`, but they
are allowed to read Spark UI in almost all the cases.
@tgravescs @vanzin This PR tries to not affect the original scenario
explained in the JIRA SPARK-22479. I would say thank you @vanzin for your
efforts.
The PR to fix SPARK-22479 also impacts more cases (e.g., in the environment
page) too. I can understand your arguments. In your scenarios, it would be nice
to expose all these info to the external users by default, as what we did in
the past. Since the new behaviors have already been introduced in the last
release, it would be better to keep them unchanged, if possible, based on the
principle of least surprise.
The lessons we learned is to document the behavior change in the migration
guide. When users try the RC branches, the users could easily catch the
changes. Unfortunately, we did not do it in that PR. Thus, it becomes a
surprise to @tgravescs after we already released Spark 2.3. We need to avoid
similar cases. We need to enforce every PR to document the behavior changes, no
matter whether it is a regression fix or a bug fix.
I am also fine to merge this if the other reviewers of the previous PR
think we should change the behavior again. cc @onursatici @ash211 @hvanhovell
@jiangxb1987
In the PR, please document the behavior changes. Hopefully, we will not see
another PR to change the default value again after we release Spark 2.4. cc
@vanzin @tgravescs WDYT?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
