mxm commented on code in PR #18582:
URL: https://github.com/apache/iceberg/pull/18582#discussion_r4323063158
##########
flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/TableUpdater.java:
##########
@@ -78,14 +78,28 @@ private void findOrCreateTable(
TableIdentifier identifier, Schema schema, PartitionSpec spec,
TableCreator tableCreator) {
Tuple2<Boolean, Exception> exists = cache.exists(identifier);
if (Boolean.FALSE.equals(exists.f0)) {
- if (exists.f1 instanceof NoSuchNamespaceException) {
+ // The lookup cannot tell a missing table from a missing namespace: the
Glue catalog, for
+ // instance, reports a table in a namespace that does not exist as
NoSuchTableException. Ask
+ // the catalog directly.
+ if (catalog instanceof SupportsNamespaces) {
Review Comment:
This comment is misleading because it only describes an edge case. The
normal case would be a race condition with other tasks which try to create the
namespace at the same time.
##########
flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/dynamic/TableUpdater.java:
##########
@@ -78,14 +78,28 @@ private void findOrCreateTable(
TableIdentifier identifier, Schema schema, PartitionSpec spec,
TableCreator tableCreator) {
Tuple2<Boolean, Exception> exists = cache.exists(identifier);
if (Boolean.FALSE.equals(exists.f0)) {
- if (exists.f1 instanceof NoSuchNamespaceException) {
+ // The lookup cannot tell a missing table from a missing namespace: the
Glue catalog, for
+ // instance, reports a table in a namespace that does not exist as
NoSuchTableException. Ask
+ // the catalog directly.
+ if (catalog instanceof SupportsNamespaces) {
SupportsNamespaces catalogWithNameSpace = (SupportsNamespaces) catalog;
- LOG.info("Namespace {} not found during table search. Creating
namespace", identifier);
- try {
- catalogWithNameSpace.createNamespace(identifier.namespace());
- } catch (AlreadyExistsException e) {
- LOG.debug("Namespace {} created concurrently",
identifier.namespace(), e);
+ // A table without a namespace has none to create, and some catalogs,
like the Hadoop
+ // catalog, do not accept the empty namespace in the call.
+ if (!identifier.namespace().isEmpty()
+ && !catalogWithNameSpace.namespaceExists(identifier.namespace())) {
+ LOG.info("Namespace {} not found during table search. Creating
namespace", identifier);
+ try {
+ catalogWithNameSpace.createNamespace(identifier.namespace());
+ } catch (AlreadyExistsException e) {
+ LOG.debug("Namespace {} created concurrently",
identifier.namespace(), e);
+ }
}
+ } else if (exists.f1 instanceof NoSuchNamespaceException) {
+ throw new UnsupportedOperationException(
+ String.format(
+ "Cannot create the namespace of table %s: catalog %s does not
support namespaces",
+ identifier, catalog.name()),
+ exists.f1);
}
Review Comment:
We should also throw in case of other exceptions in `exists.f1`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
