rdblue commented on issue #24768: [SPARK-27919][SQL] Add v2 session catalog URL: https://github.com/apache/spark/pull/24768#issuecomment-504705873 @cloud-fan, this corrects the problems with a default catalog. > Currently we can only change the default catalog for CREATE TABLE, which causes the inconsistency here After this PR, the default catalog -- if set -- it is responsible for all table names without an explicit catalog. Otherwise, the session catalog is used as the default. That's a very simple rule for when to apply the default catalog, and it is applied everywhere. And when necessary, a v2 session catalog is used so that v2 implementations can run but store data in the session catalog. This provides a good long-term path for migrating to v2. In the short term, the session catalog is the default and the v2 session catalog is used when necessary. But as v2 support is more complete, we can consider making the v2 session catalog the default. And, people that have alternative catalogs and only want to use v2 can do that easily. > In the long term, I think every session should have its own session catalog Let's clarify terms because I think that the confusion comes from not using the same ones: * **Session catalog** refers to the `SessionCatalog` interface (v1) and is a singleton catalog for a session * **V2 session catalog** is a v2 catalog that delegates operations to the session catalog * The **default catalog** is an optional v2 catalog that is responsible for identifiers that do not specify a catalog explicitly Your statement sounds like you want every session to have a default catalog. Is that correct? I don't think that you are saying that in the long term, each session should be limited to just one singleton catalog that uses the `SessionCatalog` interface. > forward other requests to the v2 catalogs Using a catalog to federate calls to other catalogs is not the design from the SPIP and, I think, would be far more complicated than the migration plan we already have. If you'd like, we can talk about this in the next sync.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
