RussellSpitzer commented on a change in pull request #1784:
URL: https://github.com/apache/iceberg/pull/1784#discussion_r526585353
##########
File path: spark/src/main/java/org/apache/iceberg/actions/BaseSparkAction.java
##########
@@ -137,7 +138,19 @@
}
// Try catalog based name based resolution
try {
- return spark.table(tableName + "." + type);
+ if (tableName.startsWith("spark_catalog")) {
+ // Do to the design of Spark, we cannot pass multi element namespaces
to the session catalog
+ // We also don't know whether the Catalog is Hive or Hadoop Based, so
we will try to load it
+ // in the hive manner first, then fall back and try the location if we
have completely run out of options
+ // TODO remove this when we have Spark workaround for multipart
identifiers in SparkSessionCatalog
+ try {
+ return
noCatalogReader.load(tableName.replaceFirst("spark_catalog\\.", "") + "." +
type);
Review comment:
Still to figure out is how we do this, since the current method is
static and it must be because the hierarchy is currently setup such that some
spark actions do not extend from this class
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
