GitHub user andrewor14 opened a pull request:
https://github.com/apache/spark/pull/12485
[SPARK-14720] Remove HiveContext (step 1)
## What changes were proposed in this pull request?
In Spark 2.0 we will have a new entry point for users known as the
`SparkSession`. This class will handle the lazy initialization of the Hive
metastore if the user runs commands that require interaction with the metastore
(e.g. `CREATE TABLE`). With this, we can remove the `HiveContext`, which is an
odd API to be exposed to Spark users.
This patch doesn't fully remove `HiveContext` but does most of the work. A
follow-up patch will actually delete the file itself.
## How was this patch tested?
Jenkins.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/andrewor14/spark spark-session
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12485.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12485
----
commit f6585f98962403247289c4352954579d14b4e2ed
Author: Andrew Or <[email protected]>
Date: 2016-04-18T17:45:43Z
[SPARK-14647][SQL] Group SQLContext/HiveContext state into SharedState
commit fe89b8ddc44a32949be6b763e180ef6587d1e310
Author: Andrew Or <[email protected]>
Date: 2016-04-18T19:45:33Z
Make HiveSessionState take in SQLContext, not HiveContext
This requires changing all the downstream places that take in
HiveContext and replacing that with (SQLContext, HiveSessionState).
commit 54046d66f68ad06d78bbda73207651c1f29518ab
Author: Andrew Or <[email protected]>
Date: 2016-04-18T20:57:33Z
Move QueryExecution out of HiveContext
commit 5fc8177e832a01f48da1d558c762a5b522f4496c
Author: Andrew Or <[email protected]>
Date: 2016-04-18T20:58:50Z
Merge branch 'master' of github.com:apache/spark into spark-session
commit 83b3f7094290613e22713b96a41b937cf9200217
Author: Andrew Or <[email protected]>
Date: 2016-04-18T21:11:54Z
Minor cleanup
commit b33514c241fb56514cd734caa6741fa8ba1cfb4f
Author: Andrew Or <[email protected]>
Date: 2016-04-18T22:53:17Z
Implement SparkSession and use it to track state
Now both shared state and session state is tracked in SparkSession
and we use reflection to instantiate them. After this commit
SQLContext and HiveContext are just wrappers for SparkSession.
commit 8379143bb8881771771ce2784d447f92df2bfa95
Author: Andrew Or <[email protected]>
Date: 2016-04-18T23:12:05Z
Merge branch 'master' of github.com:apache/spark into spark-session
commit 6b808aa411ec099b8d856b16ead1d7f9d7c342c0
Author: Andrew Or <[email protected]>
Date: 2016-04-18T23:29:20Z
Clean up some TODO's and bad signatures
commit 5198955b26c9c6d070a5ffe7292408ed9db2a525
Author: Andrew Or <[email protected]>
Date: 2016-04-19T00:47:26Z
Move the bulk of HiveContext into SessionCatalog
commit d58c6afc332622dccde92f588327125c56766446
Author: Andrew Or <[email protected]>
Date: 2016-04-19T00:51:21Z
Remove more things from HiveContext
commit edaebe5e5dbf03fe8425b1b4185c1a430b15bece
Author: Andrew Or <[email protected]>
Date: 2016-04-19T00:54:33Z
Fix style
commit ce1214d79a90ff212ad871929ed1031350dd8f68
Author: Andrew Or <[email protected]>
Date: 2016-04-19T00:54:46Z
Merge branch 'master' of github.com:apache/spark into spark-session
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
