Xi Lyu created SPARK-49798:
------------------------------
Summary: Fix inaccurate documentation of RuntimeConfig.get
Key: SPARK-49798
URL: https://issues.apache.org/jira/browse/SPARK-49798
Project: Spark
Issue Type: Documentation
Components: Spark Core
Affects Versions: 4.0.0
Reporter: Xi Lyu
The existing documentation of `RuntimeConfig.get()` is misleading:
* `get(key: String)` method will not throw any exception if the key is not set
as long as the config entry has a default value, instead, they will just return
the `defaultValue` of the `ConfigEntry`. An `NoSuchElementException` will only
be thrown if there is no default value for the config entry.
* `get(key: String, default: String)` method will ignore the `defaultValue` of
its `ConfigEntry`, and return the given param `default` if unset.
* `getOption(key: String)` method will return the `defaultValue` of its
`ConfigEntry` if the config is not set.
An example:
This line makes sure the config is not set.
```
scala> spark.conf.unset("spark.sql.session.timeZone")
```
The following code returns `Etc/UTC`, which doesn't throw any exception.
```
scala> spark.conf.get("spark.sql.session.timeZone")
res2: String = "Etc/UTC"
```
The following code returns `Europe/Berlin`, ignoring the default value.
However, the documentation only says it returns the value, without mentioning
ignoring the default value of the entry when the config is not explicitly set.
```
scala> spark.conf.get("spark.sql.session.timeZone", "Europe/Berlin")
res3: String = "Europe/Berlin"
```
The incorrect documentation is likely to mislead users to weird behaviours if
they rely on the documentation.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
