GitHub user yhuai opened a pull request:
https://github.com/apache/spark/pull/12447
[SPARK-14647][SQL] Group SQLContext/HiveContext state into SharedState
## What changes were proposed in this pull request?
This patch adds a SharedState that groups state shared across multiple
SQLContexts. This is analogous to the SessionState added in SPARK-13526 that
groups session-specific state. This cleanup makes the constructors of the
contexts simpler and ultimately allows us to remove HiveContext in the near
future.
## How was this patch tested?
Existing tests.
Closes #12405
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/yhuai/spark sharedState
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12447.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12447
----
commit b03ea772e3f3d3f9a410eea5a0303f957ba3f8d2
Author: Andrew Or <[email protected]>
Date: 2016-04-15T00:47:39Z
Group SQLContext constructor args into PersistentState
commit e8c78c308b3e6cbd23843b9acd4f79e594c4835e
Author: Andrew Or <[email protected]>
Date: 2016-04-15T00:54:51Z
Move the HiveClient's into SessionState
commit a968aaf9331df307b9750c12ea69706f121e1a2c
Author: Andrew Or <[email protected]>
Date: 2016-04-15T01:08:19Z
Fix style
commit e0a3c94709fd7ba9f455622a6d72aa9fbf7ae2fd
Author: Andrew Or <[email protected]>
Date: 2016-04-15T01:17:33Z
Always use HiveClient (not Impl)
Previously executionHive was a HiveClientImpl but metadataHive
was a HiveClient. This is inconsistent and somewhat arbitrary.
It's better practice to always use the abstract thing.
commit f83ad8be376ac94dad9a2259fcc54c70e5c5d6ee
Author: Andrew Or <[email protected]>
Date: 2016-04-15T06:11:37Z
Fix tests?
commit b52e773401634d3fba9b81d54e31d594ea3f8640
Author: Andrew Or <[email protected]>
Date: 2016-04-15T06:14:04Z
PersistentState -> SharedState
commit 97ed094bfdd14be2686c7fc70fdc09fc51ead15d
Author: Andrew Or <[email protected]>
Date: 2016-04-15T06:14:28Z
Merge branch 'master' of github.com:apache/spark into persistent-state
commit 5539c1bc36f2e59b14f4836dd5f5b4423dc4ad0a
Author: Andrew Or <[email protected]>
Date: 2016-04-15T06:14:46Z
Fix style
commit c540a9c3a94ed6448bbbe854fe0aaafd1d37fae5
Author: Andrew Or <[email protected]>
Date: 2016-04-15T18:30:30Z
Revert "Always use HiveClient (not Impl)"
This reverts commit e0a3c94709fd7ba9f455622a6d72aa9fbf7ae2fd.
commit 2a4497552b405fb1ef88dee854b61d64c4a19bcf
Author: Andrew Or <[email protected]>
Date: 2016-04-15T21:31:24Z
Merge branch 'master' of github.com:apache/spark into persistent-state
commit 9c73146c2f0a278759401682a18f1ad5e26f81a0
Author: Andrew Or <[email protected]>
Date: 2016-04-15T21:33:24Z
Fix tests?
commit 2a84b8c6bc5e62bd4eaee03c7623f3095c3b9698
Author: Andrew Or <[email protected]>
Date: 2016-04-15T23:00:28Z
Fix tests?
commit 1ba92108b5716ff54d1b4bec619d2b804d1a86e9
Author: Yin Huai <[email protected]>
Date: 2016-04-16T19:09:14Z
Merge remote-tracking branch 'upstream/master' into sharedState
commit 988fb8ca6ca154f94a1f38b524d6d174db34e959
Author: Yin Huai <[email protected]>
Date: 2016-04-16T19:19:41Z
Add some comments.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
