GitHub user pfcoperez opened a pull request:
https://github.com/apache/spark/pull/16823
[SPARK] Config methods simplification at SparkSession#Builder
## What changes were proposed in this pull request?
`SparkSession`' companion object `Builder` inner class presents several
implementations for config setter which are exact copies of the same method
just varying the value input type.
There are setters for `Double`, `Long` and `Boolean`. And the three of them
transform the input value into an string via the default `.toString` method.
These are basic types and, therefore, child classes of `AnyVal`. Why
explicitly enumerating them in a set of methods replicating code?
The `config` implementation for `String` values can be invoke by a single
method leveraging Scala's generic types with type bounds:
`def config[T <: AnyVal](key: String, value: T): Builder = config(key,
value.toString)`
This PR tries to:
- Reduce the amount of replicated code. That implies that further code
updates will require modifying different methods with the dangers derived from
possible oversights.
- Increase the code readability.
- Add all Scala basic types as possible inputs for the builder
configuration pseudo-dsl implemented by `SparkSession#Builder`
## How was this patch tested?
This PRs changes the implementation of already tested, by the current test
suites. Hence, no more testing has been added as no actual functionality is
added.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pfcoperez/spark
sparksession_buildersimplification
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/16823.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #16823
----
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
