flyrain commented on a change in pull request #2983:
URL: https://github.com/apache/iceberg/pull/2983#discussion_r690664032
##########
File path: spark3/src/main/java/org/apache/iceberg/spark/Spark3Util.java
##########
@@ -755,6 +755,27 @@ private static String
sqlString(org.apache.iceberg.expressions.Literal<?> lit) {
return null;
}
+ /**
+ * Returns an Iceberg Table by its name from a Spark V2 Catalog
+ *
+ * @param spark SparkSession used for looking up catalog references and
tables
+ * @param name The multipart identifier of the Iceberg table
+ * @return an Iceberg table
+ */
+ public static org.apache.iceberg.Table loadIcebergTable(SparkSession spark,
String name)
+ throws ParseException, NoSuchTableException {
+ CatalogAndIdentifier catalogAndIdentifier = catalogAndIdentifier(spark,
name);
+
+ CatalogPlugin catalog = catalogAndIdentifier.catalog;
+ Preconditions.checkArgument(catalog instanceof BaseCatalog, "Catalog
%s(%s) is not an Iceberg Catalog",
+ catalog.name(), catalog.getClass().toString());
+ BaseCatalog baseCatalog = (BaseCatalog) catalogAndIdentifier.catalog;
+ Table sparkTable = baseCatalog.loadTable(catalogAndIdentifier.identifier);
+
+ Preconditions.checkArgument(sparkTable instanceof SparkTable, "%s isn't a
valid Iceberg table", name);
Review comment:
Reused the method `toIcebergTable` in the new commit.
##########
File path: spark3/src/main/java/org/apache/iceberg/spark/Spark3Util.java
##########
@@ -755,6 +755,26 @@ private static String
sqlString(org.apache.iceberg.expressions.Literal<?> lit) {
return null;
}
+ /**
+ * Returns an Iceberg Table by its name from a Spark V2 Catalog
+ *
+ * @param spark SparkSession used for looking up catalog references and
tables
+ * @param name The multipart identifier of the Iceberg table
+ * @return an Iceberg table
+ */
+ public static org.apache.iceberg.Table loadIcebergTable(SparkSession spark,
String name)
+ throws ParseException, NoSuchTableException {
+ CatalogAndIdentifier catalogAndIdentifier = catalogAndIdentifier(spark,
name);
+
+ CatalogPlugin catalog = catalogAndIdentifier.catalog;
+ Preconditions.checkArgument(catalog instanceof BaseCatalog, "Catalog
%s(%s) is not an Iceberg Catalog",
+ catalog.name(), catalog.getClass().toString());
+ BaseCatalog baseCatalog = (BaseCatalog) catalogAndIdentifier.catalog;
+
+ Table sparkTable = baseCatalog.loadTable(catalogAndIdentifier.identifier);
+ return toIcebergTable(sparkTable);
Review comment:
Does it make sense to refresh the iceberg table here to make sure the
table's `ops` up to date? cc @RussellSpitzer, @kbendick, @szehon-ho
##########
File path: spark3/src/main/java/org/apache/iceberg/spark/Spark3Util.java
##########
@@ -755,6 +755,26 @@ private static String
sqlString(org.apache.iceberg.expressions.Literal<?> lit) {
return null;
}
+ /**
+ * Returns an Iceberg Table by its name from a Spark V2 Catalog
+ *
+ * @param spark SparkSession used for looking up catalog references and
tables
+ * @param name The multipart identifier of the Iceberg table
+ * @return an Iceberg table
+ */
+ public static org.apache.iceberg.Table loadIcebergTable(SparkSession spark,
String name)
+ throws ParseException, NoSuchTableException {
+ CatalogAndIdentifier catalogAndIdentifier = catalogAndIdentifier(spark,
name);
+
+ CatalogPlugin catalog = catalogAndIdentifier.catalog;
+ Preconditions.checkArgument(catalog instanceof BaseCatalog, "Catalog
%s(%s) is not an Iceberg Catalog",
+ catalog.name(), catalog.getClass().toString());
+ BaseCatalog baseCatalog = (BaseCatalog) catalogAndIdentifier.catalog;
+
+ Table sparkTable = baseCatalog.loadTable(catalogAndIdentifier.identifier);
+ return toIcebergTable(sparkTable);
Review comment:
Pro is that caller will always get an updated ops of the table.
Con is that `refresh` involves IO, which is not necessary if user is OK with
the stale `ops`.
##########
File path: spark3/src/main/java/org/apache/iceberg/spark/Spark3Util.java
##########
@@ -755,6 +755,26 @@ private static String
sqlString(org.apache.iceberg.expressions.Literal<?> lit) {
return null;
}
+ /**
+ * Returns an Iceberg Table by its name from a Spark V2 Catalog
+ *
+ * @param spark SparkSession used for looking up catalog references and
tables
+ * @param name The multipart identifier of the Iceberg table
+ * @return an Iceberg table
+ */
+ public static org.apache.iceberg.Table loadIcebergTable(SparkSession spark,
String name)
+ throws ParseException, NoSuchTableException {
+ CatalogAndIdentifier catalogAndIdentifier = catalogAndIdentifier(spark,
name);
+
+ CatalogPlugin catalog = catalogAndIdentifier.catalog;
+ Preconditions.checkArgument(catalog instanceof BaseCatalog, "Catalog
%s(%s) is not an Iceberg Catalog",
+ catalog.name(), catalog.getClass().toString());
+ BaseCatalog baseCatalog = (BaseCatalog) catalogAndIdentifier.catalog;
+
+ Table sparkTable = baseCatalog.loadTable(catalogAndIdentifier.identifier);
+ return toIcebergTable(sparkTable);
Review comment:
Pro is that caller will always get an updated ops of the table.
Con is that `refresh` involves IO, which is not necessary if user is OK with
the potentially stale `ops`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
