RussellSpitzer commented on a change in pull request #1784:
URL: https://github.com/apache/iceberg/pull/1784#discussion_r526401342
##########
File path: spark/src/main/java/org/apache/iceberg/actions/BaseSparkAction.java
##########
@@ -137,7 +138,19 @@
}
// Try catalog based name based resolution
try {
- return spark.table(tableName + "." + type);
+ if (tableName.startsWith("spark_catalog")) {
+ // Do to the design of Spark, we cannot pass multi element namespaces
to the session catalog
+ // We also don't know whether the Catalog is Hive or Hadoop Based, so
we will try to load it
+ // in the hive manner first, then fall back and try the location if we
have completely run out of options
+ // TODO remove this when we have Spark workaround for multipart
identifiers in SparkSessionCatalog
+ try {
+ return
noCatalogReader.load(tableName.replaceFirst("spark_catalog\\.", "") + "." +
type);
Review comment:
One huge drawback to this is we can't actually use any Catalog Options
that have been specified, so if we have options in the future than alter the
way a Metadata table is read we won't have them here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
