Github user gatorsmile commented on the pull request:
https://github.com/apache/spark/pull/12385#issuecomment-210968627
@yhuai @andrewor14 The latest commit moves DB existence checking from
external catalog to session catalog. I am not sure if this is the right move. I
need to hear your inputs before continuing the work to move the other similar
checking.
So far, Hive external catalog has very limited input checking, like DB
checking. It relies on the Hive. So far, the test cases in
`HiveExternalCatalogSuite` does not check if the message contents are expected.
(I know, it is hard if `HiveExternalCatalogSuite` and `InMemoryCatalogSuite`
share the same test cases) Thus, the current way could miss capturing the bugs.
This is one of the potential issues we are resolving now.
Assuming users might extend and use the other external catalog, we
definitely should do the checking in `SessionCatalog`. It will improve the
quality since the implementation of these external catalog is not predictable.
To avoid duplicate checking, we only do the checking in `SessionCatalog`.
**This assumes `SessionCatalog` is the ONE AND ONLY component that uses
`HiveExternalCatalog` and `InMemoryCatalog`.** I am not sure if this assumption
is valid and we will keep them valid. : )
Let me know if you have any opinion. Thanks!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
