JingsongLi commented on code in PR #10978:
URL: https://github.com/apache/paimon/pull/10978#discussion_r4478098274


##########
paimon-core/src/main/java/org/apache/paimon/schema/SchemaValidation.java:
##########
@@ -437,6 +439,52 @@ public static void validateTableSchema(TableSchema schema, 
Set<String> dynamicOp
         validatePkClusteringOverride(options);
 
         validateManifestSort(schema, options);
+
+        validateExternalPaths(options);
+    }
+
+    private static void validateExternalPaths(CoreOptions options) {
+        String externalPaths = options.dataFileExternalPaths();
+        if (externalPaths == null || externalPaths.isEmpty()) {
+            return;
+        }
+
+        boolean templateStrategy =
+                options.externalPathStrategy() == 
ExternalPathStrategy.ENTROPY_INJECT_TEMPLATE;
+        String[] paths = externalPaths.split(",");
+
+        if (templateStrategy && paths.length > 1) {
+            throw new IllegalArgumentException(
+                    String.format(
+                            "'%s' only accepts a single path when '%s' is 
'%s', but got %s.",
+                            CoreOptions.DATA_FILE_EXTERNAL_PATHS.key(),
+                            
CoreOptions.DATA_FILE_EXTERNAL_PATHS_STRATEGY.key(),
+                            ExternalPathStrategy.ENTROPY_INJECT_TEMPLATE,
+                            paths.length));
+        }
+
+        for (String externalPath : paths) {
+            boolean isTemplate = 
TemplateEntropyInjectExternalPathProvider.isTemplate(externalPath);
+            if (templateStrategy && !isTemplate) {
+                throw new IllegalArgumentException(
+                        String.format(
+                                "External path '%s' must contain the '%s' 
placeholder when '%s' is '%s'.",
+                                externalPath.trim(),
+                                
TemplateEntropyInjectExternalPathProvider.HASH_PLACEHOLDER,
+                                
CoreOptions.DATA_FILE_EXTERNAL_PATHS_STRATEGY.key(),
+                                ExternalPathStrategy.ENTROPY_INJECT_TEMPLATE));
+            }
+            if (!templateStrategy && isTemplate) {

Review Comment:
   [P2] Preserve literal `{hash}` directories for existing strategies
   
   An existing `round-robin` table can use a directory literally named 
`{hash}`. I created and wrote such a table with the baseline validation, then 
reopened it on this head. Both scan planning and writer creation fail at this 
check because runtime validation also applies to persisted schemas. Please 
interpret `{hash}` only when `entropy-inject-template` is selected.



##########
paimon-core/src/main/java/org/apache/paimon/schema/SchemaValidation.java:
##########
@@ -437,6 +439,52 @@ public static void validateTableSchema(TableSchema schema, 
Set<String> dynamicOp
         validatePkClusteringOverride(options);
 
         validateManifestSort(schema, options);
+
+        validateExternalPaths(options);
+    }
+
+    private static void validateExternalPaths(CoreOptions options) {
+        String externalPaths = options.dataFileExternalPaths();
+        if (externalPaths == null || externalPaths.isEmpty()) {
+            return;
+        }
+
+        boolean templateStrategy =
+                options.externalPathStrategy() == 
ExternalPathStrategy.ENTROPY_INJECT_TEMPLATE;
+        String[] paths = externalPaths.split(",");
+
+        if (templateStrategy && paths.length > 1) {
+            throw new IllegalArgumentException(
+                    String.format(
+                            "'%s' only accepts a single path when '%s' is 
'%s', but got %s.",
+                            CoreOptions.DATA_FILE_EXTERNAL_PATHS.key(),
+                            
CoreOptions.DATA_FILE_EXTERNAL_PATHS_STRATEGY.key(),
+                            ExternalPathStrategy.ENTROPY_INJECT_TEMPLATE,
+                            paths.length));
+        }
+
+        for (String externalPath : paths) {
+            boolean isTemplate = 
TemplateEntropyInjectExternalPathProvider.isTemplate(externalPath);

Review Comment:
   [P0] Keep template paths out of recursive DROP cleanup
   
   An empty table can be created with `file:///shared/{hash}/..` because this 
check sees `{hash}` in the raw option. `Path` normalizes it to 
`file:///shared`, and `getSchemaExternalPaths` passes that directory to 
recursive cleanup. I reproduced this with `FileSystemCatalog`. Dropping the 
empty table deleted a neighboring table's committed data file while leaving 
that table registered. Please validate that the placeholder survives URI path 
normalization and exclude template data paths from recursive deletion.



##########
paimon-core/src/main/java/org/apache/paimon/catalog/AbstractCatalog.java:
##########
@@ -392,9 +393,24 @@ public void dropTable(Identifier identifier, boolean 
ignoreIfNotExists)
                 FileIO tableFileIO = fileIO(tablePath);
                 SchemaManager schemaManager = new 
FileSystemSchemaManager(tableFileIO, tablePath);
                 
externalPaths.addAll(getSchemaExternalPaths(schemaManager.listAll()));
+                FileStoreTable table = (FileStoreTable) getTable(identifier);
+                boolean isExternal = 
loadTableMetadata(identifier).isExternal();
+                Optional<TableSchema> latestMainSchema = 
schemaManager.latest();
+                if (!isExternal
+                        && latestMainSchema.isPresent()
+                        && 
EntropyInjectTemplateFilesDeletion.isTemplate(latestMainSchema.get())) {
+                    EntropyInjectTemplateFilesDeletion.delete(table);
+                }
                 for (String branch : 
FileSystemBranchManager.branches(tableFileIO, tablePath)) {

Review Comment:
   [P2] Retain cleanup references when dropping a template branch
   
   I wrote one file only to `dev` on a template append table, dropped `dev`, 
and then dropped the table. The file remained in the shared prefix. 
`dropBranch` removes the metadata that points to the branch's manifests, so 
this loop can no longer find the file. The orphan cleaner also rejects template 
tables. Please retain the branch's cleanup references until its files can be 
reclaimed, while protecting files still referenced by surviving branches.



##########
paimon-core/src/main/java/org/apache/paimon/schema/SchemaValidation.java:
##########
@@ -437,6 +439,52 @@ public static void validateTableSchema(TableSchema schema, 
Set<String> dynamicOp
         validatePkClusteringOverride(options);
 
         validateManifestSort(schema, options);
+
+        validateExternalPaths(options);
+    }
+
+    private static void validateExternalPaths(CoreOptions options) {
+        String externalPaths = options.dataFileExternalPaths();
+        if (externalPaths == null || externalPaths.isEmpty()) {

Review Comment:
   [P2] Require a non-empty template when the strategy is selected
   
   `CREATE TABLE` accepts `entropy-inject-template` with no 
`data-file.external-paths`. Writes then go to the table directory, while the 
orphan cleaner still refuses the table. An explicitly empty path also makes 
`DROP TABLE` fail when it constructs a `Path`. Please require exactly one 
non-empty template for this strategy during CREATE and ALTER.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to