RussellSpitzer edited a comment on pull request #31541:
URL: https://github.com/apache/spark/pull/31541#issuecomment-776882137
Our major issue is that it makes this rather special case behavior for
spark_catalog named catalogs. For example say I create a catalog called ice
```bash
./bin/spark-shell --conf
spark.sql.catalog.ice=org.apache.iceberg.spark.SparkCatalog --conf
spark.sql.catalog.ice.type=hive --conf
spark.sql.catalog.ice.uri=thrift://localhost:9083
```
Then there is no problem with me setting this to the default catalog and
accessing a multipart identifier
```sql
USE ice; SELECT * FROM default.table.files
```
But let's say I use the exact same class but name it spark_catalog so that
it is the default session catalog.
```bash
./bin/spark-shell --conf
spark.sql.catalog.spark_catalog=org.apache.iceberg.spark.SparkCatalog --conf
spark.sql.catalog.spark_catalog.type=hive --conf
spark.sql.catalog.spark_catalog.uri=thrift://localhost:9083
```
Then the same request fails
```SQL
SELECT * FROM default.table.files
org.apache.spark.sql.AnalysisException: The namespace in session catalog
must have exactly one name part: default.part.files;
at
org.apache.spark.sql.connector.catalog.LookupCatalog$SessionCatalogAndIdentifier$.unapply(LookupCatalog.scala:65)
```
This seems like a really strange behavior to me since we are allowed to use
whatever catalog we want as "spark_catalog" but there are different rules than
if we had named it anything else.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
