RussellSpitzer commented on a change in pull request #1784:
URL: https://github.com/apache/iceberg/pull/1784#discussion_r527961284
##########
File path: spark/src/main/java/org/apache/iceberg/actions/BaseSparkAction.java
##########
@@ -128,33 +127,48 @@
return manifestDF.union(otherMetadataFileDF).union(manifestListDF);
}
+ // Attempt to use Spark3 Catalog resolution if available on the path
+ private static DynMethods.StaticMethod loadCatalogImpl = null;
+
+ private static Dataset<Row> loadCatalogMetadataTable(SparkSession spark,
String tableName, MetadataTableType type) {
+ if (loadCatalogImpl == null) {
+ try {
+ loadCatalogImpl = DynMethods.builder("loadCatalogMetadataTable")
+ .hiddenImpl("org.apache.iceberg.spark.Spark3Util",
+ SparkSession.class, String.class, MetadataTableType.class)
+ .buildStaticChecked();
Review comment:
That's nifty, I originally had a second variable to check if it was
failed to load but thought that was a bit of a waste. This seems pretty clean
though
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
