jackye1995 commented on a change in pull request #1823:
URL: https://github.com/apache/iceberg/pull/1823#discussion_r530633057
##########
File path: aws/src/main/java/org/apache/iceberg/aws/AwsProperties.java
##########
@@ -114,6 +115,33 @@
*/
public static final String S3FILEIO_ACL = "s3fileio.acl";
+ /**
+ * If {@link org.apache.iceberg.aws.glue.GlueCatalog} should use external
lock or not, default to false.
+ * If set to true, it will use DynamoDB to enforce locking during commits.
+ */
+ public static final String GLUE_CATALOG_LOCK_ENABLED =
"gluecatalog.lock.enabled";
+ public static final boolean GLUE_CATALOG_LOCK_ENABLED_DEFAULT = false;
Review comment:
Yes I agree, but after a second thought, there are a few considerations
for making this false by default:
1. this is an additional AWS resource that charges the user if enabled, so
it is better to let people explicitly do that
2. for data warehouse ETLM use cases, the current Glue concurrent
modification exception is good enough as a safe guard. I think it makes more
sense for user to turn this on when they know there is a potential for conflict
commits in use cases like streaming.
@yyanyy and @giovannifumarola, what do you think?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
