Bruce Robbins created SPARK-24814:
-------------------------------------
Summary: Relationship between catalog and datasources
Key: SPARK-24814
URL: https://issues.apache.org/jira/browse/SPARK-24814
Project: Spark
Issue Type: New Feature
Components: SQL
Affects Versions: 2.4.0
Reporter: Bruce Robbins
This is somewhat related, though not identical to, Ryan Blue's SPIP on
datasources and catalogs.
Here are the requirements (IMO) for fully implementing V2 datasources and their
relationships to catalogs:
# The global catalog should be configurable (the default can be HMS, but it
should be overridable).
# The default catalog (or an explicitly specified catalog in a query, once
multiple catalogs are supported) can determine the V2 datasource to use for
reading and writing the data.
# Conversely, a V2 datasource can determine which catalog to use for
resolution (e.g., if the user issues
{{spark.read.format("acmex").table("mytable")}}, the acmex datasource would
decide which catalog to use for resolving “mytable”).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
