GitHub user ueshin opened a pull request:
https://github.com/apache/spark/pull/23031
[SPARK-26060][CORE][SQL] Track SparkConf entries and make SET command
reject such entries.
## What changes were proposed in this pull request?
Currently the `SET` command works without any warnings even if the
specified key is for `SparkConf` entries and it has no effect because the
command does not update `SparkConf`, but the behavior might confuse users. We
should track `SparkConf` entries and make the command reject for such entries.
## How was this patch tested?
Added a test and existing tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ueshin/apache-spark
issues/SPARK-26060/set_command
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/23031.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #23031
----
commit e8de7bb6847486ab9416bda8c97e8b0ce6c3bbe2
Author: Takuya UESHIN <ueshin@...>
Date: 2018-11-13T09:59:31Z
Register SparkConfig entries.
commit 33d831e9f69b6a7f5bf965ba7357e73a70ac4239
Author: Takuya UESHIN <ueshin@...>
Date: 2018-11-13T11:20:25Z
Make SET command reject SparkConf entries.
commit e19b1e430e967d0aeb5fe2a112e3a82ca18c5c80
Author: Takuya UESHIN <ueshin@...>
Date: 2018-11-14T08:15:09Z
Update the migration guide.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
