JingsongLi commented on code in PR #10978:
URL: https://github.com/apache/paimon/pull/10978#discussion_r4625010638
##########
paimon-core/src/main/java/org/apache/paimon/catalog/AbstractCatalog.java:
##########
@@ -384,22 +387,46 @@ public void dropTable(Identifier identifier, boolean
ignoreIfNotExists)
}
Set<Path> externalPaths = new HashSet<>();
+ List<EntropyInjectTemplateFilesDeletion> templateDeletions = new
ArrayList<>();
+ List<Path> retainedBranchFiles = Collections.emptyList();
Path tablePath = getTableLocation(identifier);
if (tableExistsInFileSystem(tablePath, DEFAULT_MAIN_BRANCH)) {
TableType tableType =
CoreOptions.fromMap(loadTableSchema(identifier).options()).type();
if (tableType == TableType.TABLE || tableType ==
TableType.MATERIALIZED_TABLE) {
// Cleanup only needs persisted metadata, even if runtime
options are invalid.
FileIO tableFileIO = fileIO(tablePath);
SchemaManager schemaManager = new
FileSystemSchemaManager(tableFileIO, tablePath);
-
externalPaths.addAll(getSchemaExternalPaths(schemaManager.listAll()));
+ boolean deletesData =
!loadTableMetadata(identifier).isExternal();
+ List<TableSchema> mainSchemas = schemaManager.listAll();
+ externalPaths.addAll(getSchemaExternalPaths(mainSchemas));
+ if (deletesData) {
+ EntropyInjectTemplateFilesDeletion.collectIfTemplate(
+ tableFileIO, tablePath,
DEFAULT_MAIN_BRANCH, mainSchemas)
+ .ifPresent(templateDeletions::add);
+ }
for (String branch :
FileSystemBranchManager.branches(tableFileIO, tablePath)) {
- externalPaths.addAll(
-
getSchemaExternalPaths(schemaManager.copyWithBranch(branch).listAll()));
+ List<TableSchema> branchSchemas =
+ schemaManager.copyWithBranch(branch).listAll();
+
externalPaths.addAll(getSchemaExternalPaths(branchSchemas));
+ if (deletesData) {
+ EntropyInjectTemplateFilesDeletion.collectIfTemplate(
+ tableFileIO, tablePath, branch,
branchSchemas)
+ .ifPresent(templateDeletions::add);
+ }
+ }
+ if (deletesData) {
+ retainedBranchFiles =
+
EntropyInjectTemplateFilesDeletion.retainedBranchFiles(
+ tableFileIO, tablePath);
}
}
}
dropTableImpl(identifier, new ArrayList<>(externalPaths));
+
templateDeletions.forEach(EntropyInjectTemplateFilesDeletion::deleteCollectedFiles);
Review Comment:
**[P2] Keep a retry path for failed external-file cleanup**
I wrote a real file to an expanded template path, then injected a one-time
IOException for deletion of that external file. DROP returned successfully and
removed the table metadata. Repeating DROP IF EXISTS after storage recovered
left the file behind. The orphan cleaner intentionally rejects template-managed
tables, so the ordinary recovery workflow has no way to find the abandoned
file. Retain a durable cleanup plan across DROP, or provide a retryable cleanup
path with a clear status when file deletion fails.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]