kbendick opened a new issue #2607: URL: https://github.com/apache/iceberg/issues/2607
We have users who are trying to use multiple catalogs, but because Spark treats the session catalog differently (and we pull the Hadoop configuration off of the spark context), users cannot pass things like different kerberos principals (even though we do allow for some catalog overrides that users can If users wanted to access data from one HDFS cluster to another that had a different authority, they would not be able to do so because we pull the underlying Hadoop configuration from the spark session. Some example configurations that users cannot currently override include: ``` "spark.hadoop.hive.metastore.connect.retries" "spark.hadoop.hive.metastore.kerberos.principal" "spark.hadoop.hive.metastore.sasl.enabled" ``` This might be a spark specific issue (as the problem is pulling the Hadoop configuration from the spark session), outside of the currently allowed overrides like numClientConnections and uris. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
