GitHub user andrewor14 opened a pull request:
https://github.com/apache/spark/pull/11938
[SPARK-14014][SQL] Integrate session catalog (attempt #2)
## What changes were proposed in this pull request?
This reopens #11836, which was merged but promptly reverted because it
introduced flaky Hive tests.
## How was this patch tested?
See `CatalogTestCases`, `SessionCatalogSuite` and `HiveContextSuite`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/andrewor14/spark session-catalog-again
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11938.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11938
----
commit 66b1a3ff285f7147774e85738d2904794ad305b5
Author: Andrew Or <[email protected]>
Date: 2016-03-23T20:34:22Z
[SPARK-14014][SQL] Replace existing catalog with SessionCatalog
## What changes were proposed in this pull request?
`SessionCatalog`, introduced in #11750, is a catalog that keeps track of
temporary functions and tables, and delegates metastore operations to
`ExternalCatalog`. This functionality overlaps a lot with the existing
`analysis.Catalog`.
As of this commit, `SessionCatalog` and `ExternalCatalog` will no longer be
dead code. There are still things that need to be done after this patch, namely:
- SPARK-14013: Properly implement temporary functions in `SessionCatalog`
- SPARK-13879: Decide which DDL/DML commands to support natively in Spark
- SPARK-?????: Implement the ones we do want to support through
`SessionCatalog`.
- SPARK-?????: Merge SQL/HiveContext
## How was this patch tested?
This is largely a refactoring task so there are no new tests introduced.
The particularly relevant tests are `SessionCatalogSuite` and
`ExternalCatalogSuite`.
Author: Andrew Or <[email protected]>
Author: Yin Huai <[email protected]>
Closes #11836 from andrewor14/use-session-catalog.
commit 6b4a1246f34f51077f89551cab011036f9a7d43d
Author: Andrew Or <[email protected]>
Date: 2016-03-23T21:09:51Z
Reset all properties after each test
commit 70530cf3cd0a68a3dd0ad6b0a7d546c1cf5698ef
Author: Andrew Or <[email protected]>
Date: 2016-03-23T21:15:16Z
Enable HiveContextSuite
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
