[
https://issues.apache.org/jira/browse/SPARK-53447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kent Yao updated SPARK-53447:
-----------------------------
Priority: Minor (was: Blocker)
> How to configure multiple Hive Metastores in a single Spark session?
> --------------------------------------------------------------------
>
> Key: SPARK-53447
> URL: https://issues.apache.org/jira/browse/SPARK-53447
> Project: Spark
> Issue Type: Question
> Components: Connect, Security, SQL
> Affects Versions: 3.3.1
> Environment: Spark 3.3.1
> Reporter: zane.young
> Priority: Minor
>
> In some of our scenarios, we need to perform *cross-cluster queries* by
> reading from multiple HDFS clusters within the same Spark session.
> To achieve this, we would like to configure *multiple Hive Metastores* in
> Spark, so that tables from different clusters can be queried without
> restarting the session.
> However, when I tried to configure Spark with multiple Hive Metastore URIs,
> it did not take effect.
> In addition, if multiple HMS are supported, how should *Kerberos
> authentication* be configured for each metastore?
> *Question:*
> * How can we configure multiple Hive Metastores in a single Spark session
> (for example, one for Hive tables and another for Hudi tables)? If this is
> possible, how can we query data from these different metastores using Spark
> SQL catalogs？
> * Is there a recommended way to handle *Kerberos authentication* for
> multiple HMS?
> Looking forward to your suggestions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
