rdblue commented on a change in pull request #1784:
URL: https://github.com/apache/iceberg/pull/1784#discussion_r527957480
##########
File path: spark/src/main/java/org/apache/iceberg/actions/BaseSparkAction.java
##########
@@ -128,33 +127,48 @@
return manifestDF.union(otherMetadataFileDF).union(manifestListDF);
}
+ // Attempt to use Spark3 Catalog resolution if available on the path
+ private static DynMethods.StaticMethod loadCatalogImpl = null;
+
+ private static Dataset<Row> loadCatalogMetadataTable(SparkSession spark,
String tableName, MetadataTableType type) {
+ if (loadCatalogImpl == null) {
+ try {
+ loadCatalogImpl = DynMethods.builder("loadCatalogMetadataTable")
+ .hiddenImpl("org.apache.iceberg.spark.Spark3Util",
+ SparkSession.class, String.class, MetadataTableType.class)
+ .buildStaticChecked();
Review comment:
This doesn't need to load the method each time it is called. Usually, I
would add `orNoop` and call `build` to construct a static field. Then in this
method, you'd just need to check whether you have the method or noop:
```java
Preconditions.checkArgument(!LOAD_CATALOG.isNoop(), "Cannot find
Spark3Util class ...");
LOAD_CATALOG.invoke(spark, tableName, type);
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
