GitHub user hvanhovell opened a pull request:
https://github.com/apache/spark/pull/17433
[SPARK-20100][SQL] Refactor SessionState initialization
## What changes were proposed in this pull request?
The current SessionState initialization code path is quite complex. A part
of the creation is done in the SessionState companion objects, a part of the
creation is one inside the SessionState class, and a part is done by passing
functions.
This PR refactors this code path, and consolidates SessionState
initialization into a builder class. This SessionState will not do any
initialization and just becomes a place holder for the various Spark SQL
internals. This also lays the ground work for two future improvements:
1. This provides us with a start for removing the `HiveSessionState`.
Removing the `HiveSessionState` would also require us to move resource loading
into a separate class, and to (re)move metadata hive.
2. This makes it easier to customize the Spark Session. Currently you will
need to create a custom version of the builder. I have added hooks to
facilitate this. A future step will be to create a semi stable API on top of
this.
## How was this patch tested?
Existing tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/hvanhovell/spark SPARK-20100
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/17433.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #17433
----
commit 711582f65192ea11c1fc09f7e2453fd9e41e8d3d
Author: Herman van Hovell <[email protected]>
Date: 2017-03-26T09:48:25Z
Refactor SessionState initialization.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
