jackye1995 commented on a change in pull request #1823:
URL: https://github.com/apache/iceberg/pull/1823#discussion_r533086881
##########
File path: aws/src/main/java/org/apache/iceberg/aws/AwsProperties.java
##########
@@ -114,6 +115,33 @@
*/
public static final String S3FILEIO_ACL = "s3fileio.acl";
+ /**
+ * If {@link org.apache.iceberg.aws.glue.GlueCatalog} should use external
lock or not, default to false.
+ * If set to true, it will use DynamoDB to enforce locking during commits.
+ */
+ public static final String GLUE_CATALOG_LOCK_ENABLED =
"gluecatalog.lock.enabled";
+ public static final boolean GLUE_CATALOG_LOCK_ENABLED_DEFAULT = false;
+
+ /**
+ * The DynamoDB table used for locking.
+ * One lock table is designed to be used for only one catalog.
+ * It is recommended to use a different table name for each Glue catalog.
Review comment:
I think there will always be a confusion of Glue catalog ID versus the
catalog name concept in Iceberg. The reason I don't want to do what you propose
is that:
1. we will be forcing users to use the same catalog name for the same Glue
catalog, which might not be the case, so I don't want to make catalog name the
identifier for uniqueness.
2. Getting the actual Glue catalog ID is not straightforward, there is no
API for it and there might be proxies, different endpoints involved pointing to
different catalogs, so it is hard to get a consistent id.
I don't see any particular benefit for sharing the same dynamo table across
all catalogs, since Dynamo itself is serverless and dividing them seems to have
more flexibility, so I think the current design should be fine.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
