xiaoxuandev commented on code in PR #4423:
URL: https://github.com/apache/iceberg/pull/4423#discussion_r928508536
##########
aws/src/main/java/org/apache/iceberg/aws/glue/GlueCatalog.java:
##########
@@ -273,13 +278,24 @@ private boolean isGlueIcebergTable(Table table) {
public boolean dropTable(TableIdentifier identifier, boolean purge) {
try {
TableOperations ops = newTableOps(identifier);
- TableMetadata lastMetadata = ops.current();
+
+ GlueTableOperations glueOps = (GlueTableOperations) ops;
+ S3FileIO s3FileIO = (S3FileIO) glueOps.io();
+ TableMetadata lastMetadata = null;
+ boolean isTablePurged = isTablePurged(identifier, s3FileIO.client());
+ if (!isTablePurged) {
+ lastMetadata = ops.current();
+ }
+
glue.deleteTable(DeleteTableRequest.builder()
.catalogId(awsProperties.glueCatalogId())
.databaseName(IcebergToGlueConverter.getDatabaseName(identifier))
.name(identifier.name())
.build());
LOG.info("Successfully dropped table {} from Glue", identifier);
+ ValidationException.check(!purge ||
!awsProperties.glueLakeFormationEnabled() || isTablePurged,
+ "Cannot purge table with LakeFormation enabled because S3 access is
lost after table is dropped. " +
Review Comment:
Discussed offline, we decide to go with approach 1, a few reasons:
1. we don't want the data file to be deleted if the users don't have the
drop permission, so this rules out the option 2
2. we can use the pre-fetched credentials because LF always gives us the
credentials for each individual service, and it is always our own
responsibility to use them in a disciplined way. Even after the Glue table is
dropped, we know those S3 files belong to this table, so using the pre-fetched
S3 credentials to delete them is OK. The credential can expire, but the file
deletion is best effort anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
