rymurr commented on a change in pull request #1875:
URL: https://github.com/apache/iceberg/pull/1875#discussion_r548051961
##########
File path: spark/src/main/java/org/apache/iceberg/spark/SparkUtil.java
##########
@@ -61,4 +65,42 @@ public static void validatePartitionTransforms(PartitionSpec
spec) {
String.format("Cannot write using unsupported transforms: %s",
unsupported));
}
}
+
+ /**
+ * A modified version of Spark's LookupCatalog.CatalogAndIdentifier.unapply
+ * Attempts to find the catalog and identifier a multipart identifier
represents
+ * @param nameParts Multipart identifier representing a table
+ * @return The CatalogPlugin and Identifier for the table
+ */
+ public static <C, T> Pair<C, T> catalogAndIdentifier(List<String> nameParts,
+ Function<String, C>
catalogProvider,
+ IdentiferFunction<T>
identiferProvider,
+ C defaultCatalog,
+ String[]
currentNamespace) {
+ Preconditions.checkArgument(!nameParts.isEmpty(),
+ "Cannot determine catalog and Identifier from empty name parts");
+
+ int lastElementIndex = nameParts.size() - 1;
+ String name = nameParts.get(lastElementIndex);
+
+ if (nameParts.size() == 1) {
+ // Only a single element, use current catalog and namespace
+ return Pair.of(defaultCatalog, identiferProvider.of(currentNamespace,
name));
+ } else {
+ C catalog = catalogProvider.apply(nameParts.get(0));
+ if (catalog == null) {
+ // The first element was not a valid catalog, treat it like part of
the namespace
+ String[] namespace = nameParts.subList(0,
lastElementIndex).toArray(new String[0]);
Review comment:
actually, scratch that. It does make sense. It takes [a,b,c] as the
namespace and d as the name for [a,b,c,d]. So I have left it in.
The other one returns a as the catalog [b,c] as the namespace and d as the
table.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
