rdblue commented on a change in pull request #1161:
URL: https://github.com/apache/iceberg/pull/1161#discussion_r449655254
##########
File path: spark/src/main/java/org/apache/iceberg/actions/BaseAction.java
##########
@@ -30,8 +31,11 @@ protected String metadataTableName(MetadataTableType type) {
String tableName = table().toString();
if (tableName.contains("/")) {
return tableName + "#" + type;
- } else if (tableName.startsWith("hadoop.") ||
tableName.startsWith("hive.")) {
- // HiveCatalog and HadoopCatalog prepend a logical name which we need to
drop for Spark 2.4
+ } else if (tableName.startsWith("hadoop.")) {
+ // HadoopCatalog tableName style is 'hadoop.ns.tb'
Review comment:
I don't think this comment is correct because the solution is to convert
to a path table reference.
If i understand correctly, the problem in Spark 2.4 is that IcebergSource
uses the Hive catalog or it uses HadoopTables to load a path. The table you're
passing in was loaded by HadoopCatalog, so it doesn't work because that catalog
is not available. I agree that the solution is to convert a Hadoop table to a
path reference, but then this comment should explain what's happening: a
HadoopCatalog was used to load the table, so convert to the path.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
