JingsongLi commented on code in PR #10978:
URL: https://github.com/apache/paimon/pull/10978#discussion_r4456027098
##########
paimon-core/src/main/java/org/apache/paimon/catalog/AbstractCatalog.java:
##########
@@ -392,9 +393,24 @@ public void dropTable(Identifier identifier, boolean
ignoreIfNotExists)
FileIO tableFileIO = fileIO(tablePath);
SchemaManager schemaManager = new
FileSystemSchemaManager(tableFileIO, tablePath);
externalPaths.addAll(getSchemaExternalPaths(schemaManager.listAll()));
+ FileStoreTable table = (FileStoreTable) getTable(identifier);
+ boolean isExternal =
loadTableMetadata(identifier).isExternal();
+ Optional<TableSchema> latestMainSchema =
schemaManager.latest();
+ if (!isExternal
+ && latestMainSchema.isPresent()
+ &&
EntropyInjectTemplateFilesDeletion.isTemplate(latestMainSchema.get())) {
+ EntropyInjectTemplateFilesDeletion.delete(table);
Review Comment:
[P1] Delete data only after the catalog accepts DROP
This deletes live files before `dropTableImpl` has accepted the drop. Hive's
implementation first calls HMS `dropTable`, and JDBC first deletes its catalog
record; either can fail with authorization/transport/SQL errors while leaving
the table registered. I reproduced the ordering with a real template append
table and a catalog whose `dropTableImpl` throws: `getTable` still succeeds
after the failed DROP, but the previously present data file is gone. Baseline
defers file deletion until the catalog operation succeeds. Please collect the
referenced paths first and perform physical deletion only after successful
catalog deletion, with the same ordering for branch files.
##########
paimon-core/src/main/java/org/apache/paimon/catalog/AbstractCatalog.java:
##########
@@ -392,9 +393,24 @@ public void dropTable(Identifier identifier, boolean
ignoreIfNotExists)
FileIO tableFileIO = fileIO(tablePath);
SchemaManager schemaManager = new
FileSystemSchemaManager(tableFileIO, tablePath);
externalPaths.addAll(getSchemaExternalPaths(schemaManager.listAll()));
+ FileStoreTable table = (FileStoreTable) getTable(identifier);
+ boolean isExternal =
loadTableMetadata(identifier).isExternal();
+ Optional<TableSchema> latestMainSchema =
schemaManager.latest();
+ if (!isExternal
+ && latestMainSchema.isPresent()
+ &&
EntropyInjectTemplateFilesDeletion.isTemplate(latestMainSchema.get())) {
Review Comment:
[P2] Include historical template strategies in DROP cleanup
External paths and strategies are mutable, but cleanup is selected only from
the latest schema. With a real table, I wrote one file using the template,
changed both options to a new ordinary path with `round-robin`, then dropped
the table. DROP succeeds and removes its metadata while the old expanded
template file remains. `getSchemaExternalPaths(listAll())` only supplies the
literal `{hash}` path, so it cannot reclaim that file, and the surviving
manifest references have been lost. Please inspect the schema history of each
branch when deciding whether to collect manifest-referenced files, rather than
requiring its latest strategy to remain the template.
##########
paimon-core/src/main/java/org/apache/paimon/catalog/AbstractCatalog.java:
##########
@@ -392,9 +393,24 @@ public void dropTable(Identifier identifier, boolean
ignoreIfNotExists)
FileIO tableFileIO = fileIO(tablePath);
SchemaManager schemaManager = new
FileSystemSchemaManager(tableFileIO, tablePath);
externalPaths.addAll(getSchemaExternalPaths(schemaManager.listAll()));
+ FileStoreTable table = (FileStoreTable) getTable(identifier);
Review Comment:
[P2] Keep DROP metadata-only for existing non-template tables
The unconditional `getTable` constructs read wrappers even when no branch
uses the new strategy. A valid existing chain table can have its snapshot
branch removed through `branchManager().dropBranch("snapshot")`; DROP now
throws `NoSuchElementException` in `createChainTable` at this new call instead
of deleting the table. I reproduced this with both chain branch options and no
external paths, after first confirming the intact chain loads. The parent
implementation only reads persisted schemas and can drop it. Please create a
raw branch-local table only when template cleanup actually requires it,
preserving the previous metadata-only cleanup path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]