[
https://issues.apache.org/jira/browse/SPARK-15777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15593229#comment-15593229
]
Yan commented on SPARK-15777:
-----------------------------
One approach could be first tagging a subtree as specific to a data source, and
then only applying the custom rules from that data source to the subtree so
tagged. There could be other feasible approaches, and it is considered one of
the details left open for future discussions. Thanks.
> Catalog federation
> ------------------
>
> Key: SPARK-15777
> URL: https://issues.apache.org/jira/browse/SPARK-15777
> Project: Spark
> Issue Type: New Feature
> Components: SQL
> Reporter: Reynold Xin
> Attachments: SparkFederationDesign.pdf
>
>
> This is a ticket to track progress to support federating multiple external
> catalogs. This would require establishing an API (similar to the current
> ExternalCatalog API) for getting information about external catalogs, and
> ability to convert a table into a data source table.
> As part of this, we would also need to be able to support more than a
> two-level table identifier (database.table). At the very least we would need
> a three level identifier for tables (catalog.database.table). A possibly
> direction is to support arbitrary level hierarchical namespaces similar to
> file systems.
> Once we have this implemented, we can convert the current Hive catalog
> implementation into an external catalog that is "mounted" into an internal
> catalog.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
