This is an automated email from the ASF dual-hosted git repository.

yuqi1129 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/gravitino.git


The following commit(s) were added to refs/heads/main by this push:
     new 61d5c84753 [#14640] fix(core): preserve native comments and isolate 
metadata operations (#14641)
61d5c84753 is described below

commit 61d5c847538adc240dac008162815401e9568fff
Author: Qi Yu <[email protected]>
AuthorDate: Fri Nov 6 17:04:42 2026 +0800

    [#14640] fix(core): preserve native comments and isolate metadata 
operations (#14641)
    
    ### What changes were proposed in this pull request?
    
    - Preserve complete source column comments in responses, stable column
    IDs, and a bounded, NUL-free entity-store projection. Validate user
    comments consistently before source DDL or managed-table writes;
    accepted requests are stored without truncation. Compare and log the
    same stored projection.
    - Lock the target catalog during creation, initialize and validate it
    before persistence, and coordinate local store/cache publication with
    catalog loading. Lock the target table during DROP; after releasing that
    lock, probe schema existence under a read lock and take a schema write
    lock only for missing schemas, with a recheck before cleanup.
    - Apply provider existing-name rules to existing column references while
    retaining creation rules for ADD COLUMN and rename targets. Enable
    PostgreSQL's existing column-name specification.
    - Wait for asynchronous JDBC staging-table cleanup in marked Trino SHOW
    assertions. Golden regeneration uses the existing expected result and
    preserves the old file on missing expectations or timeout.
    
    ### Why are the changes needed?
    
    Native comments containing NUL or exceeding entity-store capacity make
    valid external tables unloadable. User writes can also execute source
    DDL before registration fails. Catalog verification and table DROP hold
    ancestor write locks and block unrelated objects. Existing native column
    references incorrectly use creation-name rules. Narrowing catalog
    creation's lock additionally requires preventing listings from seeing
    catalogs whose verification has not succeeded.
    
    Fixes #14640
    
    ### Does this PR introduce _any_ user-facing change?
    
    External column comments remain complete in source responses; their
    stored projection is capped at 4096 UTF-16 units. CREATE, ADD COLUMN,
    and comment-update requests reject NUL and comments exceeding that
    bound, including managed tables. Supplementary characters count as two
    units: 2048 emoji fit, while 4096 do not. This portable limit is
    required by H2's VARCHAR storage; accepted requests are never truncated.
    No storage migration is required.
    
    Catalog creation and table DROP release the former ancestor write-lock
    bottlenecks. Table PURGE, view DROP, and cold table import retain their
    existing locking behavior; PURGE and view DROP can still block sibling
    operations during source DDL. This adds no SQL timeout/cancellation or
    cross-node source/store atomicity guarantee. Providers other than
    PostgreSQL retain their existing-name defaults until they opt in.
    
    ### How was this patch tested?
    
    190 targeted tests passed with zero failures/errors/skips: 166 core
    tests (`TestCatalogManager`, `TestCatalogWrapperLease`,
    `TestManagedTableOperations`, `TestTableOperationDispatcher`,
    `TestColumnEntity`, `TestCapabilityHelpers`, `TestTreeLock`), 12
    PostgreSQL capability tests, and 12 Trino SQL-harness tests.
    Catalog-manager tests were also rerun after the final
    exception-cause/comment cleanup.
    
    Coverage includes full source-comment responses with stable IDs,
    unchanged-projection write avoidance, pre-DDL Unicode/NUL rejection,
    managed CREATE/ADD/comment-update round trips, catalog invisibility
    during successful/failed verification, serialized local store/cache
    publication, initialized-wrapper cleanup after insert failure, sibling
    access during blocked DROP, schema cleanup failure/recreation, and
    existing-name validation. Eight regression cases were confirmed to fail
    before the review fixes. Golden-generation tests cover transient
    cleanup, permanent mismatch without file changes, missing expectations,
    statement alignment, repeated generation with 0/1/2 trailing newlines,
    unmarked queries, and rejecting write retries.
    
    A direct probe against the pinned H2 2.2.224 VARCHAR(4096) accepted 4096
    ASCII characters, 4094 ASCII characters plus one emoji, and 2048 emoji;
    it rejected 4095 ASCII characters plus one emoji and 4096 emoji
    (SQLState 22001). Scoped Spotless and `git diff --check` passed. The
    full core suite and Docker Trino SQL suite were not run locally;
    catalog/table regression tests use the in-memory store. No SQL or
    migration scripts changed.
---
 .../postgresql/PostgreSqlCatalogCapability.java    |   2 +-
 .../TestPostgreSqlCatalogCapability.java           |   7 +-
 .../gravitino/catalog/CapabilityHelpers.java       |  21 +-
 .../apache/gravitino/catalog/CatalogManager.java   |  95 +++++---
 .../gravitino/catalog/ManagedTableOperations.java  |   9 +-
 .../catalog/TableOperationDispatcher.java          | 120 ++++++----
 .../gravitino/connector/capability/Capability.java |   8 +-
 .../java/org/apache/gravitino/lock/TreeLock.java   |  12 +-
 .../org/apache/gravitino/meta/ColumnEntity.java    |  46 +++-
 .../gravitino/catalog/TestCapabilityHelpers.java   |  54 +++++
 .../gravitino/catalog/TestCatalogManager.java      | 203 +++++++++++++++-
 .../catalog/TestManagedTableOperations.java        |  61 +++++
 .../catalog/TestTableOperationDispatcher.java      | 254 +++++++++++++++++++++
 .../apache/gravitino/meta/TestColumnEntity.java    |  15 ++
 docs/manage-relational-metadata-using-gravitino.md |  29 +++
 .../integration/test/TestTrinoQueryIT.java         | 194 ++++++++++++++++
 .../connector/integration/test/TrinoQueryIT.java   | 128 ++++++++---
 .../testsets/jdbc-postgresql/00006_datatype.sql    |   5 +-
 18 files changed, 1129 insertions(+), 134 deletions(-)

diff --git 
a/catalogs/catalog-jdbc-postgresql/src/main/java/org/apache/gravitino/catalog/postgresql/PostgreSqlCatalogCapability.java
 
b/catalogs/catalog-jdbc-postgresql/src/main/java/org/apache/gravitino/catalog/postgresql/PostgreSqlCatalogCapability.java
index 07f5c1e144..218642d51a 100644
--- 
a/catalogs/catalog-jdbc-postgresql/src/main/java/org/apache/gravitino/catalog/postgresql/PostgreSqlCatalogCapability.java
+++ 
b/catalogs/catalog-jdbc-postgresql/src/main/java/org/apache/gravitino/catalog/postgresql/PostgreSqlCatalogCapability.java
@@ -46,7 +46,7 @@ public class PostgreSqlCatalogCapability implements 
Capability {
   /** {@inheritDoc} */
   @Override
   public CapabilityResult specificationOnExistingName(Scope scope, String 
name) {
-    if (scope != Scope.TABLE && scope != Scope.SCHEMA) {
+    if (scope != Scope.TABLE && scope != Scope.SCHEMA && scope != 
Scope.COLUMN) {
       return specificationOnName(scope, name);
     }
     if (MetadataObjects.METADATA_OBJECT_RESERVED_NAME.equals(name)) {
diff --git 
a/catalogs/catalog-jdbc-postgresql/src/test/java/org/apache/gravitino/catalog/postgresql/TestPostgreSqlCatalogCapability.java
 
b/catalogs/catalog-jdbc-postgresql/src/test/java/org/apache/gravitino/catalog/postgresql/TestPostgreSqlCatalogCapability.java
index 6f33006344..6f77cb9e10 100644
--- 
a/catalogs/catalog-jdbc-postgresql/src/test/java/org/apache/gravitino/catalog/postgresql/TestPostgreSqlCatalogCapability.java
+++ 
b/catalogs/catalog-jdbc-postgresql/src/test/java/org/apache/gravitino/catalog/postgresql/TestPostgreSqlCatalogCapability.java
@@ -28,15 +28,18 @@ public class TestPostgreSqlCatalogCapability {
   private final PostgreSqlCatalogCapability capability = new 
PostgreSqlCatalogCapability();
 
   @Test
-  void testNativeDeletionNamesRemainDistinctFromCreationNames() {
+  void testNativeNamesRemainDistinctFromCreationNames() {
     for (Capability.Scope scope :
-        new Capability.Scope[] {Capability.Scope.TABLE, 
Capability.Scope.SCHEMA}) {
+        new Capability.Scope[] {
+          Capability.Scope.TABLE, Capability.Scope.SCHEMA, 
Capability.Scope.COLUMN
+        }) {
       Assertions.assertTrue(
           capability.specificationOnExistingName(scope, 
"native\"name").supported());
       Assertions.assertFalse(capability.specificationOnName(scope, 
"native\"name").supported());
       Assertions.assertFalse(
           capability.specificationOnExistingName(scope, 
"bad\0name").supported());
       Assertions.assertFalse(capability.specificationOnExistingName(scope, 
"").supported());
+      Assertions.assertFalse(capability.specificationOnExistingName(scope, 
"*").supported());
       Assertions.assertFalse(
           capability.specificationOnExistingName(scope, 
"่กจ".repeat(22)).supported());
     }
diff --git 
a/core/src/main/java/org/apache/gravitino/catalog/CapabilityHelpers.java 
b/core/src/main/java/org/apache/gravitino/catalog/CapabilityHelpers.java
index b6e784259b..51f898344e 100644
--- a/core/src/main/java/org/apache/gravitino/catalog/CapabilityHelpers.java
+++ b/core/src/main/java/org/apache/gravitino/catalog/CapabilityHelpers.java
@@ -29,6 +29,7 @@ import org.apache.gravitino.Namespace;
 import org.apache.gravitino.connector.CatalogOperations;
 import org.apache.gravitino.connector.SupportsTableNameResolution;
 import org.apache.gravitino.connector.capability.Capability;
+import org.apache.gravitino.connector.capability.CapabilityResult;
 import org.apache.gravitino.exceptions.NoSuchCatalogException;
 import org.apache.gravitino.file.FilesetChange;
 import org.apache.gravitino.model.ModelChange;
@@ -450,15 +451,23 @@ public class CapabilityHelpers {
     return TableChange.rename(newName, newSchemaName);
   }
 
+  private static void applyExistingColumnName(String name, Capability 
capabilities) {
+    CapabilityResult result =
+        capabilities.specificationOnExistingName(Capability.Scope.COLUMN, 
name);
+    Preconditions.checkArgument(
+        result.supported(), result.unsupportedMessage() + " Illegal name: " + 
name);
+  }
+
   private static TableChange applyCapabilities(
       TableChange.ColumnChange change, Capability capabilities) {
-    applyNameSpecification(Capability.Scope.COLUMN, change.fieldName()[0], 
capabilities);
-    String[] fieldName = applyCaseSensitiveOnColumnName(change.fieldName(), 
capabilities);
-
     if (change instanceof TableChange.AddColumn) {
       return applyCapabilities((TableChange.AddColumn) change, capabilities);
+    }
+
+    applyExistingColumnName(change.fieldName()[0], capabilities);
+    String[] fieldName = applyCaseSensitiveOnColumnName(change.fieldName(), 
capabilities);
 
-    } else if (change instanceof TableChange.UpdateColumnNullability) {
+    if (change instanceof TableChange.UpdateColumnNullability) {
       return applyCapabilities((TableChange.UpdateColumnNullability) change, 
capabilities);
 
     } else if (change instanceof TableChange.UpdateColumnDefaultValue) {
@@ -483,7 +492,7 @@ public class CapabilityHelpers {
           (TableChange.UpdateColumnPosition) change;
       if (updateColumnPosition.getPosition() instanceof TableChange.After) {
         TableChange.After afterPosition = (TableChange.After) 
updateColumnPosition.getPosition();
-        applyNameSpecification(Capability.Scope.COLUMN, 
afterPosition.getColumn(), capabilities);
+        applyExistingColumnName(afterPosition.getColumn(), capabilities);
         String afterFieldName =
             applyCaseSensitiveOnName(
                 Capability.Scope.COLUMN, afterPosition.getColumn(), 
capabilities);
@@ -554,7 +563,7 @@ public class CapabilityHelpers {
 
   private static TableChange applyCapabilities(
       TableChange.RenameColumn renameColumn, Capability capabilities) {
-    String[] fieldName = applyCapabilities(renameColumn.fieldName(), 
capabilities);
+    String[] fieldName = 
applyCaseSensitiveOnColumnName(renameColumn.fieldName(), capabilities);
     String newName = renameColumn.getNewName();
     if (fieldName.length == 1) {
       newName = applyCapabilitiesOnName(Capability.Scope.COLUMN, newName, 
capabilities);
diff --git 
a/core/src/main/java/org/apache/gravitino/catalog/CatalogManager.java 
b/core/src/main/java/org/apache/gravitino/catalog/CatalogManager.java
index 5bf613b92d..e8118d646b 100644
--- a/core/src/main/java/org/apache/gravitino/catalog/CatalogManager.java
+++ b/core/src/main/java/org/apache/gravitino/catalog/CatalogManager.java
@@ -55,6 +55,7 @@ import java.util.Set;
 import java.util.concurrent.CopyOnWriteArrayList;
 import java.util.concurrent.ScheduledThreadPoolExecutor;
 import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
 import java.util.concurrent.atomic.AtomicReference;
 import java.util.concurrent.locks.ReentrantReadWriteLock;
 import java.util.function.Consumer;
@@ -861,42 +862,72 @@ public class CatalogManager implements CatalogDispatcher, 
Closeable {
             .build();
 
     return TreeLockUtils.doWithTreeLock(
-        metalakeIdent,
+        ident,
         LockType.WRITE,
         () -> {
-          checkMetalake(metalakeIdent, store);
-          // Write secrets before store.put / init: createBaseCatalog resolves 
URNs via
-          // toPlaintextProperties. Roll back on any create failure (same 
pattern as
-          // schema/fileset).
-          secretManager.writeSecrets(secretMaterials);
-          boolean entityStored = false;
+          // Keep close() from retiring resources between initialization and 
publication. The
+          // lock order remains tree lock, lifecycle lock, then the per-key 
cache operation.
+          lifecycleLock.readLock().lock();
           try {
+            checkOpen();
+            checkMetalake(metalakeIdent, store);
+            // Initialization resolves secret URNs, but must not publish the 
entity yet.
+            secretManager.writeSecrets(secretMaterials);
+            AtomicBoolean entityStored = new AtomicBoolean(false);
             try {
-              store.put(e, false /* overwrite */);
-            } catch (NoSuchEntityException e1) {
-              // The relational store locks and rechecks the parent metalake 
while inserting the
-              // catalog. A concurrent drop or rename can therefore make the 
metalake disappear
-              // after checkMetalake() succeeds but before this insert starts.
-              LOG.warn("Metalake {} does not exist", metalakeIdent, e1);
-              throw new NoSuchMetalakeException(e1, "Metalake %s does not 
exist", metalakeIdent);
-            } catch (EntityAlreadyExistsException e1) {
-              LOG.warn("Catalog {} already exists", ident, e1);
-              throw new CatalogAlreadyExistsException("Catalog %s already 
exists", ident);
-            }
-            entityStored = true;
-            try (CatalogLease lease =
-                createAndCacheCatalogLease(ident, () -> 
createCatalogWrapper(e, mergedConfig))) {
-              return toCatalogInfo(lease.wrapper());
+              // Reject known duplicates before contacting the backend; 
store.put still arbitrates
+              // concurrent creates from other server instances.
+              if (store.exists(ident, EntityType.CATALOG)) {
+                throw new CatalogAlreadyExistsException("Catalog %s already 
exists", ident);
+              }
+              CatalogWrapper wrapper = createCatalogWrapper(e, mergedConfig);
+              boolean published = false;
+              try {
+                Preconditions.checkState(
+                    wrapper.tryAcquire(), "A new catalog wrapper cannot be 
retired");
+                try (CatalogLease lease = new CatalogLease(wrapper)) {
+                  // Snapshot construction can also fail; finish it before 
making the row visible.
+                  Catalog info = toCatalogInfo(lease.wrapper());
+                  // Initializing outside compute keeps slow backend 
verification out of cache
+                  // locks. Serialize the short store/cache publication with 
acquireCatalogLease
+                  // so a listCatalogsInfo reader cannot build a second 
wrapper in between them.
+                  catalogCache
+                      .asMap()
+                      .compute(
+                          ident,
+                          (key, previous) -> {
+                            try {
+                              store.put(e, false /* overwrite */);
+                            } catch (NoSuchEntityException missing) {
+                              throw new NoSuchMetalakeException(
+                                  missing, "Metalake %s does not exist", 
metalakeIdent);
+                            } catch (EntityAlreadyExistsException duplicate) {
+                              throw new CatalogAlreadyExistsException(
+                                  duplicate, "Catalog %s already exists", 
ident);
+                            } catch (IOException ioe) {
+                              throw new RuntimeException("Failed to store 
catalog " + ident, ioe);
+                            }
+                            entityStored.set(true);
+                            return wrapper;
+                          });
+                  published = true;
+                  return info;
+                }
+              } finally {
+                if (!published) {
+                  wrapper.retire();
+                }
+              }
+            } catch (RuntimeException re) {
+              rollbackFailedCatalogCreate(ident, secretMaterials, 
entityStored.get());
+              throw re;
+            } catch (Exception ex) {
+              rollbackFailedCatalogCreate(ident, secretMaterials, 
entityStored.get());
+              LOG.error("Failed to create catalog {}", ident, ex);
+              throw new RuntimeException(ex);
             }
-
-          } catch (RuntimeException re) {
-            rollbackFailedCatalogCreate(ident, secretMaterials, entityStored);
-            throw re;
-
-          } catch (Exception ex) {
-            rollbackFailedCatalogCreate(ident, secretMaterials, entityStored);
-            LOG.error("Failed to create catalog {}", ident, ex);
-            throw new RuntimeException(ex);
+          } finally {
+            lifecycleLock.readLock().unlock();
           }
         });
   }
@@ -2300,7 +2331,7 @@ public class CatalogManager implements CatalogDispatcher, 
Closeable {
 
   /**
    * Rolls back secrets written for a catalog create attempt and, when {@code 
store.put} succeeded,
-   * removes the catalog entity left behind by a failed init.
+   * removes the catalog entity left behind by a failed publication.
    */
   private void rollbackFailedCatalogCreate(
       NameIdentifier ident, List<SecretMaterial> secretMaterials, boolean 
entityStored) {
diff --git 
a/core/src/main/java/org/apache/gravitino/catalog/ManagedTableOperations.java 
b/core/src/main/java/org/apache/gravitino/catalog/ManagedTableOperations.java
index 63bf737e81..e1f9fa35ce 100644
--- 
a/core/src/main/java/org/apache/gravitino/catalog/ManagedTableOperations.java
+++ 
b/core/src/main/java/org/apache/gravitino/catalog/ManagedTableOperations.java
@@ -312,6 +312,7 @@ public abstract class ManagedTableOperations implements 
TableCatalog {
           DOT.join(change.fieldName()));
       if (change instanceof TableChange.AddColumn addColumn) {
         String columnName = DOT.join(addColumn.getFieldName());
+        ColumnEntity.validateCommentForWrite(columnName, 
addColumn.getComment());
         boolean exists = newColumns.stream().anyMatch(col -> 
col.name().equals(columnName));
         if (exists) {
           throw new IllegalArgumentException(String.format("Column %s already 
exists", columnName));
@@ -456,6 +457,8 @@ public abstract class ManagedTableOperations implements 
TableCatalog {
       Optional<Integer> newPosition,
       Optional<Boolean> newNullable,
       Optional<Boolean> newAutoIncrement) {
+    newComment.ifPresent(
+        comment -> ColumnEntity.validateCommentForWrite(oldColumn.name(), 
comment));
     return ColumnEntity.builder()
         .withId(oldColumn.id())
         .withName(newName.orElse(oldColumn.name()))
@@ -511,7 +514,11 @@ public abstract class ManagedTableOperations implements 
TableCatalog {
     return columns == null
         ? Collections.emptyList()
         : IntStream.range(0, columns.length)
-            .mapToObj(i -> ColumnEntity.toColumnEntity(columns[i], i, 
idGenerator.nextId(), audit))
+            .mapToObj(
+                i -> {
+                  ColumnEntity.validateCommentForWrite(columns[i].name(), 
columns[i].comment());
+                  return ColumnEntity.toColumnEntity(columns[i], i, 
idGenerator.nextId(), audit);
+                })
             .collect(Collectors.toList());
   }
 
diff --git 
a/core/src/main/java/org/apache/gravitino/catalog/TableOperationDispatcher.java 
b/core/src/main/java/org/apache/gravitino/catalog/TableOperationDispatcher.java
index 31a747c34e..091170ab09 100644
--- 
a/core/src/main/java/org/apache/gravitino/catalog/TableOperationDispatcher.java
+++ 
b/core/src/main/java/org/apache/gravitino/catalog/TableOperationDispatcher.java
@@ -481,42 +481,46 @@ public class TableOperationDispatcher extends 
OperationDispatcher implements Tab
   @Override
   public boolean dropTable(NameIdentifier ident) {
     NameIdentifier schemaIdentifier = getSchemaIdentifier(ident);
-    return TreeLockUtils.doWithTreeLock(
-        schemaIdentifier,
-        LockType.WRITE,
-        () -> {
-          NameIdentifier catalogIdent = getCatalogIdentifier(ident);
-          boolean droppedFromCatalog =
-              doWithCatalog(
-                  catalogIdent,
-                  c -> c.doWithTableOps(t -> t.dropTable(ident)),
-                  RuntimeException.class);
-
-          boolean isManagedTable = isManagedEntity(catalogIdent, 
Capability.Scope.TABLE);
-          if (isManagedTable) {
-            return droppedFromCatalog;
-          }
+    Pair<Boolean, Boolean> result =
+        TreeLockUtils.doWithTreeLock(
+            ident,
+            LockType.WRITE,
+            () -> {
+              NameIdentifier catalogIdent = getCatalogIdentifier(ident);
+              boolean droppedFromCatalog =
+                  doWithCatalog(
+                      catalogIdent,
+                      c -> c.doWithTableOps(t -> t.dropTable(ident)),
+                      RuntimeException.class);
+
+              boolean isManagedTable = isManagedEntity(catalogIdent, 
Capability.Scope.TABLE);
+              if (isManagedTable) {
+                return Pair.of(droppedFromCatalog, isManagedTable);
+              }
 
-          // A false result is ambiguous: the external table may have been 
renamed or dropped out of
-          // band. Preserve the registration because deleting it after a 
rename would lose
-          // Gravitino-only metadata. A true out-of-band drop can therefore 
leave a stale
-          // registration that requires separate cleanup.
-          if (droppedFromCatalog) {
-            try {
-              store.delete(ident, TABLE);
-            } catch (OptimisticLockException e) {
-              throw e;
-            } catch (NoSuchEntityException e) {
-              LOG.warn("The table to be dropped does not exist in the store: 
{}", ident, e);
-            } catch (Exception e) {
-              throw new RuntimeException(e);
-            }
-          }
-          // Run unconditionally: an out-of-band drop may have left orphaned 
schema entities. The
-          // cleanup is best-effort and stops as soon as a schema still exists.
-          OrphanedSchemaCleanup.cleanUp(this, catalogIdent, schemaIdentifier);
-          return droppedFromCatalog;
-        });
+              // A false result is ambiguous: the external table may have been 
renamed or dropped
+              // out of band. Preserve the registration because deleting it 
after a rename would
+              // lose Gravitino-only metadata. A true out-of-band drop can 
therefore leave a stale
+              // registration that requires separate cleanup.
+              if (droppedFromCatalog) {
+                try {
+                  store.delete(ident, TABLE);
+                } catch (OptimisticLockException e) {
+                  throw e;
+                } catch (NoSuchEntityException e) {
+                  LOG.warn("The table to be dropped does not exist in the 
store: {}", ident, e);
+                } catch (Exception e) {
+                  throw new RuntimeException(e);
+                }
+              }
+              return Pair.of(droppedFromCatalog, isManagedTable);
+            });
+    if (!result.getRight()) {
+      // Run even after a false drop result: out-of-band drops can leave 
orphaned schemas.
+      // Release the table lock before cleanup to avoid a lock upgrade.
+      cleanUpOrphanedSchema(getCatalogIdentifier(ident), schemaIdentifier);
+    }
+    return result.getLeft();
   }
 
   /**
@@ -917,7 +921,7 @@ public class TableOperationDispatcher extends 
OperationDispatcher implements Tab
     return Objects.equal(left.name(), right.name())
         && columnPosition == right.position()
         && Objects.equal(left.dataType(), right.dataType())
-        && Objects.equal(left.comment(), right.comment())
+        && Objects.equal(ColumnEntity.commentForStorage(left.comment()), 
right.comment())
         && left.nullable() == right.nullable()
         && left.autoIncrement() == right.autoIncrement()
         && isSameDefaultValue(left.defaultValue(), right.defaultValue());
@@ -965,10 +969,10 @@ public class TableOperationDispatcher extends 
OperationDispatcher implements Tab
           String.format(
               "type[catalog=%s, store=%s]", catalogColumn.dataType(), 
entityColumn.dataType()));
     }
-    if (!Objects.equal(catalogColumn.comment(), entityColumn.comment())) {
+    String storedComment = 
ColumnEntity.commentForStorage(catalogColumn.comment());
+    if (!Objects.equal(storedComment, entityColumn.comment())) {
       differences.add(
-          String.format(
-              "comment[catalog=%s, store=%s]", catalogColumn.comment(), 
entityColumn.comment()));
+          String.format("comment[catalog=%s, store=%s]", storedComment, 
entityColumn.comment()));
     }
     if (catalogColumn.nullable() != entityColumn.nullable()) {
       differences.add(
@@ -1224,7 +1228,7 @@ public class TableOperationDispatcher extends 
OperationDispatcher implements Tab
                 .withName(column.name())
                 .withPosition(columnPair.getLeft())
                 .withDataType(column.dataType())
-                .withComment(column.comment())
+                .withComment(ColumnEntity.commentForStorage(column.comment()))
                 .withNullable(column.nullable())
                 .withAutoIncrement(column.autoIncrement())
                 .withDefaultValue(column.defaultValue())
@@ -1339,10 +1343,39 @@ public class TableOperationDispatcher extends 
OperationDispatcher implements Tab
                 combinedTable.tableFromGravitino().id()));
   }
 
+  private void cleanUpOrphanedSchema(NameIdentifier catalogIdent, 
NameIdentifier schemaIdent) {
+    try {
+      // A queued schema writer also blocks new readers. Avoid queuing one 
behind another table's
+      // source DDL when the schema still exists and there is nothing to clean 
up.
+      boolean schemaExists =
+          TreeLockUtils.doWithTreeLock(
+              schemaIdent,
+              LockType.READ,
+              () ->
+                  doWithCatalog(
+                      catalogIdent,
+                      c -> c.doWithSchemaOps(s -> s.schemaExists(schemaIdent)),
+                      RuntimeException.class));
+      if (schemaExists) {
+        return;
+      }
+      TreeLockUtils.doWithTreeLock(
+          schemaIdent,
+          LockType.WRITE,
+          () -> {
+            // Recheck under the write lock before deleting any registration.
+            OrphanedSchemaCleanup.cleanUp(this, catalogIdent, schemaIdent);
+            return null;
+          });
+    } catch (Exception e) {
+      LOG.warn("Failed to clean up orphaned schema entities starting from {}", 
schemaIdent, e);
+    }
+  }
+
   private static void validateColumns(Column[] columns) {
     for (Column column : columns) {
       ColumnEntity.NAME.validate(column.name(), COLUMN);
-      ColumnEntity.COMMENT.validate(column.comment(), COLUMN);
+      ColumnEntity.validateCommentForWrite(column.name(), column.comment());
     }
   }
 
@@ -1351,7 +1384,7 @@ public class TableOperationDispatcher extends 
OperationDispatcher implements Tab
       TableChange.AddColumn addColumn = (TableChange.AddColumn) change;
       if (addColumn.getFieldName().length == 1) {
         ColumnEntity.NAME.validate(addColumn.getFieldName()[0], COLUMN);
-        ColumnEntity.COMMENT.validate(addColumn.getComment(), COLUMN);
+        ColumnEntity.validateCommentForWrite(addColumn.getFieldName()[0], 
addColumn.getComment());
       }
     } else if (change instanceof TableChange.RenameColumn) {
       TableChange.RenameColumn renameColumn = (TableChange.RenameColumn) 
change;
@@ -1361,7 +1394,8 @@ public class TableOperationDispatcher extends 
OperationDispatcher implements Tab
     } else if (change instanceof TableChange.UpdateColumnComment) {
       TableChange.UpdateColumnComment updateComment = 
(TableChange.UpdateColumnComment) change;
       if (updateComment.getFieldName().length == 1) {
-        ColumnEntity.COMMENT.validate(updateComment.getNewComment(), COLUMN);
+        ColumnEntity.validateCommentForWrite(
+            updateComment.getFieldName()[0], updateComment.getNewComment());
       }
     }
   }
diff --git 
a/core/src/main/java/org/apache/gravitino/connector/capability/Capability.java 
b/core/src/main/java/org/apache/gravitino/connector/capability/Capability.java
index ff6cd062f1..359ad24ee2 100644
--- 
a/core/src/main/java/org/apache/gravitino/connector/capability/Capability.java
+++ 
b/core/src/main/java/org/apache/gravitino/connector/capability/Capability.java
@@ -106,13 +106,13 @@ public interface Capability {
   }
 
   /**
-   * Checks a table or schema name on an existing-table deletion path.
+   * Checks an existing table or schema name for deletion, or an existing 
column reference.
    *
    * <p>The default preserves the create-time restrictions. A catalog may 
accept additional native
-   * names only when its deletion operations safely quote them. This does not 
relax creation or
-   * rename validation.
+   * names only when the corresponding operations safely quote them. This does 
not relax creation or
+   * rename-target validation.
    *
-   * @param scope the table or schema scope
+   * @param scope the table, schema, or column scope
    * @param name the existing name to check
    * @return whether the existing name is supported
    */
diff --git a/core/src/main/java/org/apache/gravitino/lock/TreeLock.java 
b/core/src/main/java/org/apache/gravitino/lock/TreeLock.java
index a9c2d7013a..3186979307 100644
--- a/core/src/main/java/org/apache/gravitino/lock/TreeLock.java
+++ b/core/src/main/java/org/apache/gravitino/lock/TreeLock.java
@@ -39,22 +39,22 @@ import org.slf4j.LoggerFactory;
  *   /metalake                            readLock
  *   /metalake/catalog                    readLock
  *   /metalake/catalog/db1                readLock
- *   /metalake/catalog/db/table1          readLock
+ *   /metalake/catalog/db1/table1         readLock
  * </pre>
  *
- * If we need to alter a table `metalake.catalog.db1.table1` (without changing 
the name of it), the
- * lock manager will lock the following:
+ * If we need to drop a table `metalake.catalog.db1.table1`, the lock manager 
will lock the
+ * following:
  *
  * <pre>
  *   /                                    readLock
  *   /metalake                            readLock
  *   /metalake/catalog                    readLock
  *   /metalake/catalog/db1                readLock
- *   /metalake/catalog/db/table1          writeLock
+ *   /metalake/catalog/db1/table1         writeLock
  * </pre>
  *
- * When we need to rename a table or drop a table 
`metalake.catalog.db1.table1`, the lock manager
- * will lock the following:
+ * When we need to rename a table `metalake.catalog.db1.table1`, the lock 
manager will lock the
+ * following:
  *
  * <pre>
  *   /                                    readLock
diff --git a/core/src/main/java/org/apache/gravitino/meta/ColumnEntity.java 
b/core/src/main/java/org/apache/gravitino/meta/ColumnEntity.java
index 451d18854a..8082f3d8cd 100644
--- a/core/src/main/java/org/apache/gravitino/meta/ColumnEntity.java
+++ b/core/src/main/java/org/apache/gravitino/meta/ColumnEntity.java
@@ -20,9 +20,11 @@
 package org.apache.gravitino.meta;
 
 import com.google.common.base.Objects;
+import com.google.common.base.Preconditions;
 import com.google.common.collect.Maps;
 import java.util.Collections;
 import java.util.Map;
+import javax.annotation.Nullable;
 import lombok.ToString;
 import org.apache.gravitino.Audit;
 import org.apache.gravitino.Auditable;
@@ -163,13 +165,55 @@ public class ColumnEntity implements Entity, Auditable {
     return new Builder();
   }
 
+  /**
+   * Validates a user-supplied column comment before writing source or managed 
metadata. H2 counts
+   * UTF-16 units in VARCHAR limits, so accepted comments must fit that 
portable bound without the
+   * lossy projection used for external imports.
+   *
+   * @param columnName the column name used in validation errors
+   * @param comment the requested comment, which may be null
+   * @throws IllegalArgumentException if the comment contains NUL or exceeds 
the storage limit
+   */
+  public static void validateCommentForWrite(String columnName, @Nullable 
String comment) {
+    Preconditions.checkArgument(
+        comment == null || comment.indexOf('\0') < 0,
+        "Column comment for %s must not contain NUL characters",
+        columnName);
+    COMMENT.validate(comment, EntityType.COLUMN);
+    Preconditions.checkArgument(
+        comment == null || comment.length() <= 
EntityFieldLimits.MAX_COLUMN_COMMENT_LENGTH,
+        "Column comment for %s must not exceed %s UTF-16 units",
+        columnName,
+        EntityFieldLimits.MAX_COLUMN_COMMENT_LENGTH);
+  }
+
+  /**
+   * Projects a source column comment into the portable entity-store 
representation. The full source
+   * comment remains available from the connector table.
+   *
+   * @param comment the source comment
+   * @return a NUL-free comment bounded by the storage capacity
+   */
+  @Nullable
+  public static String commentForStorage(@Nullable String comment) {
+    if (comment == null) {
+      return null;
+    }
+    String safe = comment.replace("\0", "");
+    int end = Math.min(safe.length(), 
EntityFieldLimits.MAX_COLUMN_COMMENT_LENGTH);
+    if (end < safe.length() && end > 0 && 
Character.isHighSurrogate(safe.charAt(end - 1))) {
+      end--;
+    }
+    return safe.substring(0, end);
+  }
+
   public static ColumnEntity toColumnEntity(
       Column column, int position, long uid, AuditInfo audit) {
     return builder()
         .withId(uid)
         .withName(column.name())
         .withPosition(position)
-        .withComment(column.comment())
+        .withComment(commentForStorage(column.comment()))
         .withDataType(column.dataType())
         .withNullable(column.nullable())
         .withAutoIncrement(column.autoIncrement())
diff --git 
a/core/src/test/java/org/apache/gravitino/catalog/TestCapabilityHelpers.java 
b/core/src/test/java/org/apache/gravitino/catalog/TestCapabilityHelpers.java
index 7b16498d87..ec5db8b006 100644
--- a/core/src/test/java/org/apache/gravitino/catalog/TestCapabilityHelpers.java
+++ b/core/src/test/java/org/apache/gravitino/catalog/TestCapabilityHelpers.java
@@ -25,11 +25,13 @@ import org.apache.gravitino.connector.BaseCatalog;
 import org.apache.gravitino.connector.capability.Capability;
 import org.apache.gravitino.connector.capability.CapabilityResult;
 import org.apache.gravitino.exceptions.NoSuchCatalogException;
+import org.apache.gravitino.rel.TableChange;
 import org.apache.gravitino.rel.expressions.literals.Literal;
 import org.apache.gravitino.rel.expressions.literals.Literals;
 import org.apache.gravitino.rel.partitions.IdentityPartition;
 import org.apache.gravitino.rel.partitions.Partition;
 import org.apache.gravitino.rel.partitions.Partitions;
+import org.apache.gravitino.rel.types.Types;
 import org.junit.jupiter.api.Assertions;
 import org.junit.jupiter.api.Test;
 import org.mockito.Mockito;
@@ -90,6 +92,58 @@ public class TestCapabilityHelpers {
         }
       };
 
+  @Test
+  void testNewColumnDoesNotUseExistingNameRules() {
+    Capability capability =
+        new Capability() {
+          @Override
+          public CapabilityResult specificationOnExistingName(Scope scope, 
String name) {
+            throw new AssertionError("New columns must use creation rules");
+          }
+        };
+    TableChange[] changes =
+        CapabilityHelpers.applyCapabilities(
+            capability,
+            TableChange.addColumn(new String[] {"new_column"}, 
Types.IntegerType.get()));
+    Assertions.assertEquals(1, changes.length);
+  }
+
+  /** Checks existing references and new names use their respective provider 
rules. */
+  @Test
+  public void testExistingColumnReferencesDoNotUseCreationPattern() {
+    Capability strict =
+        new Capability() {
+          @Override
+          public CapabilityResult specificationOnName(Scope scope, String 
name) {
+            return name.matches("[a-zA-Z_][a-zA-Z0-9_]*")
+                ? CapabilityResult.SUPPORTED
+                : CapabilityResult.unsupported("invalid new name");
+          }
+
+          @Override
+          public CapabilityResult specificationOnExistingName(Scope scope, 
String name) {
+            return CapabilityResult.SUPPORTED;
+          }
+        };
+    TableChange[] changes =
+        CapabilityHelpers.applyCapabilities(
+            strict,
+            TableChange.updateColumnComment(new String[] {"native col"}, 
"comment"),
+            TableChange.renameColumn(new String[] {"native col"}, 
"valid_name"),
+            TableChange.deleteColumn(new String[] {"native col"}, false));
+    Assertions.assertEquals(3, changes.length);
+    Assertions.assertThrows(
+        IllegalArgumentException.class,
+        () ->
+            CapabilityHelpers.applyCapabilities(
+                strict, TableChange.addColumn(new String[] {"new col"}, 
Types.IntegerType.get())));
+    Assertions.assertThrows(
+        IllegalArgumentException.class,
+        () ->
+            CapabilityHelpers.applyCapabilities(
+                strict, TableChange.renameColumn(new String[] {"native col"}, 
"new col")));
+  }
+
   @Test
   void testApplyCaseSensitiveOnNameHonorsCustomNormalizeName() {
     String normalized =
diff --git 
a/core/src/test/java/org/apache/gravitino/catalog/TestCatalogManager.java 
b/core/src/test/java/org/apache/gravitino/catalog/TestCatalogManager.java
index e375280325..1b9f3b7ad5 100644
--- a/core/src/test/java/org/apache/gravitino/catalog/TestCatalogManager.java
+++ b/core/src/test/java/org/apache/gravitino/catalog/TestCatalogManager.java
@@ -52,6 +52,7 @@ import java.util.concurrent.ExecutorService;
 import java.util.concurrent.Executors;
 import java.util.concurrent.Future;
 import java.util.concurrent.TimeUnit;
+import java.util.concurrent.TimeoutException;
 import java.util.concurrent.atomic.AtomicBoolean;
 import java.util.concurrent.atomic.AtomicLong;
 import java.util.concurrent.atomic.AtomicReference;
@@ -235,6 +236,199 @@ public class TestCatalogManager {
     }
   }
 
+  @ParameterizedTest
+  @ValueSource(booleans = {false, true})
+  void testCatalogCreateIsInvisibleUntilValidationCompletes(boolean 
failValidation)
+      throws Exception {
+    NameIdentifier ident = NameIdentifier.of(metalake, "publication_test");
+    CountDownLatch validationStarted = new CountDownLatch(1);
+    CountDownLatch releaseValidation = new CountDownLatch(1);
+    ExecutorService executor = Executors.newFixedThreadPool(2);
+    RuntimeException failure = new IllegalArgumentException("backend 
validation failed");
+    try (SecretManager secrets = memorySecretManager();
+        CatalogManager manager =
+            Mockito.spy(
+                new CatalogManager(config, entityStore, new 
RandomIdGenerator(), secrets))) {
+      Mockito.doAnswer(
+              invocation -> {
+                validationStarted.countDown();
+                Assertions.assertTrue(releaseValidation.await(10, 
TimeUnit.SECONDS));
+                if (failValidation) {
+                  throw failure;
+                }
+                return invocation.callRealMethod();
+              })
+          .when(manager)
+          .createCatalogWrapper(any(CatalogEntity.class), Mockito.anyMap());
+      Future<Catalog> creation =
+          executor.submit(
+              () ->
+                  manager.createCatalog(
+                      ident,
+                      Catalog.Type.RELATIONAL,
+                      provider,
+                      "comment",
+                      catalogProps(),
+                      Map.of(PROPERTY_KEY4, new SecretBinding("memory", 
"password")),
+                      Map.of()));
+      try {
+        Assertions.assertTrue(validationStarted.await(5, TimeUnit.SECONDS));
+        Assertions.assertEquals(
+            0,
+            executor
+                .submit(() -> manager.listCatalogs(Namespace.of(metalake)))
+                .get(3, TimeUnit.SECONDS)
+                .length);
+        Assertions.assertEquals(
+            0,
+            executor
+                .submit(() -> manager.listCatalogsInfo(Namespace.of(metalake)))
+                .get(3, TimeUnit.SECONDS)
+                .length);
+        Assertions.assertNull(manager.getCatalogCache().getIfPresent(ident));
+        releaseValidation.countDown();
+        if (failValidation) {
+          Exception error =
+              Assertions.assertThrows(Exception.class, () -> creation.get(5, 
TimeUnit.SECONDS));
+          Assertions.assertSame(failure, error.getCause());
+          Assertions.assertFalse(entityStore.exists(ident, 
EntityType.CATALOG));
+        } else {
+          Assertions.assertEquals(ident.name(), creation.get(5, 
TimeUnit.SECONDS).name());
+          Assertions.assertEquals(1, 
manager.listCatalogsInfo(Namespace.of(metalake)).length);
+          Mockito.verify(manager, Mockito.never())
+              .createCatalogWrapper(any(CatalogEntity.class), eq(null));
+        }
+      } finally {
+        releaseValidation.countDown();
+        executor.shutdownNow();
+        Assertions.assertTrue(executor.awaitTermination(5, TimeUnit.SECONDS));
+      }
+    }
+  }
+
+  @Test
+  void testCatalogInsertAndCachePublicationDoNotPermitDuplicateLoad() throws 
Exception {
+    InMemoryEntityStore store = Mockito.spy(new InMemoryEntityStore());
+    store.initialize(config);
+    store.put(metalakeEntity, true);
+    NameIdentifier ident = NameIdentifier.of(metalake, "publication_gap");
+    CountDownLatch inserted = new CountDownLatch(1);
+    CountDownLatch releaseInsert = new CountDownLatch(1);
+    CountDownLatch loadStarted = new CountDownLatch(1);
+    ExecutorService executor = Executors.newFixedThreadPool(2);
+    Mockito.doAnswer(
+            invocation -> {
+              Object result = invocation.callRealMethod();
+              inserted.countDown();
+              Assertions.assertTrue(releaseInsert.await(10, TimeUnit.SECONDS));
+              return result;
+            })
+        .when(store)
+        .put(any(CatalogEntity.class), eq(false));
+    try (SecretManager secrets = memorySecretManager();
+        CatalogManager manager =
+            Mockito.spy(new CatalogManager(config, store, new 
RandomIdGenerator(), secrets))) {
+      Mockito.doAnswer(
+              invocation -> {
+                loadStarted.countDown();
+                return invocation.callRealMethod();
+              })
+          .when(manager)
+          .acquireCatalogLease(ident);
+      Future<Catalog> creation =
+          executor.submit(
+              () ->
+                  manager.createCatalog(
+                      ident, Catalog.Type.RELATIONAL, provider, "comment", 
catalogProps()));
+      try {
+        Assertions.assertTrue(inserted.await(5, TimeUnit.SECONDS));
+        Future<Catalog[]> listing =
+            executor.submit(() -> 
manager.listCatalogsInfo(Namespace.of(metalake)));
+        Assertions.assertTrue(loadStarted.await(5, TimeUnit.SECONDS));
+        Assertions.assertThrows(
+            TimeoutException.class, () -> listing.get(200, 
TimeUnit.MILLISECONDS));
+        releaseInsert.countDown();
+        Assertions.assertEquals(ident.name(), creation.get(5, 
TimeUnit.SECONDS).name());
+        Assertions.assertEquals(ident.name(), listing.get(5, 
TimeUnit.SECONDS)[0].name());
+        Mockito.verify(manager, Mockito.times(1))
+            .createCatalogWrapper(any(CatalogEntity.class), Mockito.anyMap());
+        Mockito.verify(manager, Mockito.never())
+            .createCatalogWrapper(any(CatalogEntity.class), eq(null));
+      } finally {
+        releaseInsert.countDown();
+        executor.shutdownNow();
+        Assertions.assertTrue(executor.awaitTermination(5, TimeUnit.SECONDS));
+      }
+    } finally {
+      store.close();
+    }
+  }
+
+  @Test
+  void testFailedCatalogInsertClosesInitializedWrapper() throws Exception {
+    InMemoryEntityStore store = Mockito.spy(new InMemoryEntityStore());
+    store.initialize(config);
+    store.put(metalakeEntity, true);
+    IOException failure = new IOException("catalog insert failed");
+    Mockito.doThrow(failure).when(store).put(any(CatalogEntity.class), 
eq(false));
+    AtomicReference<CatalogManager.CatalogWrapper> initialized = new 
AtomicReference<>();
+    try (SecretManager secrets = memorySecretManager();
+        CatalogManager manager =
+            Mockito.spy(new CatalogManager(config, store, new 
RandomIdGenerator(), secrets))) {
+      Mockito.doAnswer(
+              invocation -> {
+                CatalogManager.CatalogWrapper wrapper =
+                    (CatalogManager.CatalogWrapper) 
invocation.callRealMethod();
+                initialized.set(wrapper);
+                return wrapper;
+              })
+          .when(manager)
+          .createCatalogWrapper(any(CatalogEntity.class), Mockito.anyMap());
+      RuntimeException error =
+          Assertions.assertThrows(
+              RuntimeException.class,
+              () ->
+                  manager.createCatalog(
+                      NameIdentifier.of(metalake, "failed_insert"),
+                      Catalog.Type.RELATIONAL,
+                      provider,
+                      "comment",
+                      catalogProps()));
+      Assertions.assertSame(failure, error.getCause());
+      Assertions.assertNotNull(initialized.get());
+      Assertions.assertTrue(initialized.get().isRetired());
+      Assertions.assertEquals(0, initialized.get().activeOperations());
+      Assertions.assertNull(initialized.get().catalog());
+      Mockito.verify(store, Mockito.never()).delete(any(), 
eq(EntityType.CATALOG), eq(true));
+    } finally {
+      store.close();
+    }
+  }
+
+  /** Checks a catalog creation does not take a sibling catalog lock. */
+  @Test
+  public void testCatalogCreationDoesNotLockSiblingCatalogs() throws Exception 
{
+    try (TreeLockTestSupport.HeldLock sibling =
+        TreeLockTestSupport.HeldLock.acquire(
+            NameIdentifier.of(metalake, "existing_sibling"), LockType.READ)) {
+      TreeLockTestSupport.assertRunsConcurrentlyWith(
+          sibling,
+          () ->
+              catalogManager.createCatalog(
+                  NameIdentifier.of(metalake, "independent_catalog_create"),
+                  Catalog.Type.RELATIONAL,
+                  provider,
+                  "comment",
+                  ImmutableMap.of(
+                      PROPERTY_KEY1,
+                      "value1",
+                      PROPERTY_KEY2,
+                      "value2",
+                      PROPERTY_KEY5_PREFIX + "1",
+                      "value3")));
+    }
+  }
+
   @Test
   void testJdbcDriverDefaultAfterProviderConfigMerge(@TempDir Path 
catalogPackage)
       throws IOException {
@@ -841,7 +1035,7 @@ public class TestCatalogManager {
             catalogManager.createCatalog(
                 failIdent, Catalog.Type.RELATIONAL, provider, "comment", 
failProps));
     
Assertions.assertNull(catalogManager.getCatalogCache().getIfPresent(failIdent));
-    // Store rollback: the just-persisted entity is deleted.
+    // Failed backend validation must not leave a catalog entity.
     Assertions.assertThrows(
         NoSuchEntityException.class,
         () -> entityStore.get(failIdent, EntityType.CATALOG, 
CatalogEntity.class));
@@ -1403,8 +1597,7 @@ public class TestCatalogManager {
         new CatalogManager(config, store, new RandomIdGenerator(), new 
SecretManager(config));
     NameIdentifier ident = NameIdentifier.of("metalake", 
"failed_create_cleanup");
 
-    // A creation that fails validation (key1 is required but missing) stores 
the entity and then
-    // rolls it back via store.delete(), which writes a DROP record to the 
entity change log.
+    // Validation failure must leave no catalog row. A later valid create can 
reuse the name.
     Map<String, String> invalidProps =
         ImmutableMap.of(
             "provider", "test", PROPERTY_KEY2, "value2", PROPERTY_KEY5_PREFIX 
+ "1", "v");
@@ -1429,8 +1622,8 @@ public class TestCatalogManager {
     Assertions.assertNotNull(manager.loadCatalogAndWrap(ident));
     Assertions.assertNotNull(manager.getCatalogCache().getIfPresent(ident));
 
-    // Cleanup DROP and subsequent ALTER both invalidate the recreated entry. 
No local marker
-    // may suppress either record; active operations remain protected by 
CatalogLease.
+    // A replayed DROP from an earlier incarnation and subsequent ALTER must 
both invalidate the
+    // new entry. No local marker may suppress either record; leases protect 
active operations.
     store
         .listener
         .get()
diff --git 
a/core/src/test/java/org/apache/gravitino/catalog/TestManagedTableOperations.java
 
b/core/src/test/java/org/apache/gravitino/catalog/TestManagedTableOperations.java
index 8f8e921d86..394542a9f1 100644
--- 
a/core/src/test/java/org/apache/gravitino/catalog/TestManagedTableOperations.java
+++ 
b/core/src/test/java/org/apache/gravitino/catalog/TestManagedTableOperations.java
@@ -121,6 +121,67 @@ public class TestManagedTableOperations {
     Assertions.assertEquals("schema1", schema.name());
   }
 
+  @Test
+  void testUnicodeCommentValidationAndRoundTrip() {
+    NameIdentifier ident =
+        NameIdentifierUtil.ofTable(METALAKE_NAME, CATALOG_NAME, SCHEMA_NAME, 
"unicode_comments");
+    for (String comment :
+        new String[] {"a".repeat(4095) + "๐Ÿ˜€", "๐Ÿ˜€".repeat(4096), 
"bad\0comment"}) {
+      Assertions.assertThrows(
+          IllegalArgumentException.class,
+          () ->
+              tableOperations.createTable(
+                  ident,
+                  new Column[] {Column.of("c", Types.StringType.get(), 
comment)},
+                  "comment",
+                  StringIdentifier.newPropertiesWithId(
+                      StringIdentifier.fromId(idGenerator.nextId()), Map.of()),
+                  new Transform[0],
+                  Distributions.NONE,
+                  new SortOrder[0],
+                  Indexes.EMPTY_INDEXES));
+    }
+    String comment = "๐Ÿ˜€".repeat(2048);
+    Table created =
+        tableOperations.createTable(
+            ident,
+            new Column[] {Column.of("c", Types.StringType.get(), comment)},
+            "comment",
+            StringIdentifier.newPropertiesWithId(
+                StringIdentifier.fromId(idGenerator.nextId()), Map.of()),
+            new Transform[0],
+            Distributions.NONE,
+            new SortOrder[0],
+            Indexes.EMPTY_INDEXES);
+    Assertions.assertEquals(comment, created.columns()[0].comment());
+    Assertions.assertEquals(comment, 
tableOperations.loadTable(ident).columns()[0].comment());
+    String mixed = "a".repeat(4094) + "๐Ÿ˜€";
+    Table altered =
+        tableOperations.alterTable(
+            ident,
+            TableChange.updateColumnComment(new String[] {"c"}, mixed),
+            TableChange.addColumn(new String[] {"added"}, 
Types.StringType.get(), comment));
+    Assertions.assertEquals(mixed, altered.columns()[0].comment());
+    Assertions.assertEquals(comment, altered.columns()[1].comment());
+    for (String invalid :
+        new String[] {"a".repeat(4095) + "๐Ÿ˜€", "๐Ÿ˜€".repeat(4096), 
"bad\0comment"}) {
+      Assertions.assertThrows(
+          IllegalArgumentException.class,
+          () ->
+              tableOperations.alterTable(
+                  ident, TableChange.updateColumnComment(new String[] {"c"}, 
invalid)));
+      Assertions.assertThrows(
+          IllegalArgumentException.class,
+          () ->
+              tableOperations.alterTable(
+                  ident,
+                  TableChange.addColumn(
+                      new String[] {"rejected"}, Types.StringType.get(), 
invalid)));
+      Assertions.assertEquals(mixed, 
tableOperations.loadTable(ident).columns()[0].comment());
+      Assertions.assertEquals(2, 
tableOperations.loadTable(ident).columns().length);
+    }
+  }
+
   @Test
   public void testCreateAndListTables() {
     // Create a table
diff --git 
a/core/src/test/java/org/apache/gravitino/catalog/TestTableOperationDispatcher.java
 
b/core/src/test/java/org/apache/gravitino/catalog/TestTableOperationDispatcher.java
index eca298fd11..a64b6d3557 100644
--- 
a/core/src/test/java/org/apache/gravitino/catalog/TestTableOperationDispatcher.java
+++ 
b/core/src/test/java/org/apache/gravitino/catalog/TestTableOperationDispatcher.java
@@ -61,6 +61,7 @@ import java.util.stream.Collectors;
 import java.util.stream.IntStream;
 import java.util.stream.Stream;
 import org.apache.commons.lang3.reflect.FieldUtils;
+import org.apache.commons.lang3.reflect.MethodUtils;
 import org.apache.gravitino.Config;
 import org.apache.gravitino.Entity;
 import org.apache.gravitino.EntityAlreadyExistsException;
@@ -139,6 +140,259 @@ public class TestTableOperationDispatcher extends 
TestOperationDispatcher {
         GravitinoEnv.getInstance(), "internalSchemaDispatcher", 
schemaOperationDispatcher, true);
   }
 
+  /** Checks blocked source DDL permits warm sibling metadata reads. */
+  @Test
+  public void testBlockedSourceDropDoesNotBlockSiblingMetadataReads() throws 
Exception {
+    Namespace ns = Namespace.of(metalake, catalog, "drop_source_lock_scope");
+    schemaOperationDispatcher.createSchema(
+        NameIdentifier.of(ns.levels()), "comment", Map.of("k1", "v1", "k2", 
"v2"));
+    NameIdentifier target = NameIdentifier.of(ns, "target");
+    NameIdentifier sibling = NameIdentifier.of(ns, "sibling");
+    createTable(target);
+    createTable(sibling);
+    NameIdentifier completedDrop = NameIdentifier.of(ns, "completed_drop");
+    createTable(completedDrop);
+    CountDownLatch started = new CountDownLatch(1);
+    CountDownLatch release = new CountDownLatch(1);
+    TestCatalog base =
+        catalogManager.doWithCatalog(NameIdentifier.of(metalake, catalog), c 
-> (TestCatalog) c);
+    TestCatalogOperations original = (TestCatalogOperations) base.ops();
+    TestCatalogOperations source = spy(original);
+    FieldUtils.writeField(base, "ops", source, true);
+    doAnswer(
+            invocation -> {
+              started.countDown();
+              Assertions.assertTrue(release.await(5, TimeUnit.SECONDS));
+              return invocation.callRealMethod();
+            })
+        .when(source)
+        .dropTable(target);
+    ExecutorService executor = Executors.newFixedThreadPool(2);
+    Future<Boolean> drop = executor.submit(() -> 
tableOperationDispatcher.dropTable(target));
+    try {
+      Assertions.assertTrue(started.await(5, TimeUnit.SECONDS));
+      // Cleanup for a completed drop must not queue a schema writer behind 
the blocked drop.
+      Assertions.assertTrue(
+          executor
+              .submit(() -> tableOperationDispatcher.dropTable(completedDrop))
+              .get(3, TimeUnit.SECONDS));
+      executor
+          .submit(
+              () -> {
+                Assertions.assertEquals(2, 
tableOperationDispatcher.listTables(ns).length);
+                Assertions.assertEquals(
+                    "sibling", 
tableOperationDispatcher.loadTable(sibling).name());
+              })
+          .get(3, TimeUnit.SECONDS);
+      Assertions.assertFalse(drop.isDone());
+      release.countDown();
+      Assertions.assertTrue(drop.get(5, TimeUnit.SECONDS));
+    } finally {
+      release.countDown();
+      executor.shutdownNow();
+      FieldUtils.writeField(base, "ops", original, true);
+    }
+  }
+
+  @Test
+  void testSchemaCleanupProbeFailureAndRecreatedSchemaPreserveRegistration() 
throws Exception {
+    for (boolean failProbe : new boolean[] {true, false}) {
+      Namespace ns = Namespace.of(metalake, catalog, "cleanup_probe_" + 
failProbe);
+      NameIdentifier schemaIdent = NameIdentifier.of(ns.levels());
+      schemaOperationDispatcher.createSchema(
+          schemaIdent, "comment", Map.of("k1", "v1", "k2", "v2"));
+      NameIdentifier tableIdent = NameIdentifier.of(ns, "t");
+      createTable(tableIdent);
+      TestCatalog base =
+          catalogManager.doWithCatalog(NameIdentifier.of(metalake, catalog), c 
-> (TestCatalog) c);
+      TestCatalogOperations original = (TestCatalogOperations) base.ops();
+      TestCatalogOperations source = spy(original);
+      FieldUtils.writeField(base, "ops", source, true);
+      try {
+        if (failProbe) {
+          doThrow(new IllegalStateException("schema probe failed"))
+              .when(source)
+              .schemaExists(schemaIdent);
+        } else {
+          // The schema reappears between the read probe and the write-locked 
recheck.
+          doReturn(false, true).when(source).schemaExists(schemaIdent);
+        }
+        clearInvocations(entityStore);
+        Assertions.assertTrue(tableOperationDispatcher.dropTable(tableIdent));
+        verify(source, times(failProbe ? 1 : 2)).schemaExists(schemaIdent);
+        verify(entityStore, never()).delete(eq(schemaIdent), eq(SCHEMA), 
anyBoolean());
+      } finally {
+        FieldUtils.writeField(base, "ops", original, true);
+      }
+    }
+  }
+
+  /** Checks external comments preserve complete responses and column 
identities. */
+  @Test
+  public void 
testExternalCommentsAreProjectedForStorageAndPreservedInResponses() throws 
Exception {
+    Namespace ns = Namespace.of(metalake, catalog, 
"external_comment_projection");
+    schemaOperationDispatcher.createSchema(
+        NameIdentifier.of(ns.levels()), "comment", Map.of("k1", "v1", "k2", 
"v2"));
+    NameIdentifier ident = NameIdentifier.of(ns, "native_table");
+    String sourceComment = "a\0" + "๐Ÿ˜€".repeat(3000) + "tail";
+    catalogManager.doWithCatalog(
+        NameIdentifier.of(metalake, catalog),
+        c -> {
+          TestCatalogOperations ops = (TestCatalogOperations) c.ops();
+          ops.createTable(
+              ident,
+              new Column[] {Column.of("c", Types.StringType.get(), 
sourceComment)},
+              "comment",
+              Map.of("k1", "v1", "k2", "v2"),
+              new Transform[0]);
+          return null;
+        });
+    Table imported = tableOperationDispatcher.loadTable(ident);
+    Assertions.assertEquals(sourceComment, imported.columns()[0].comment());
+    TableEntity stored = entityStore.get(ident, TABLE, TableEntity.class);
+    String projection = ColumnEntity.commentForStorage(sourceComment);
+    Assertions.assertEquals(projection, stored.columns().get(0).comment());
+    Assertions.assertFalse(projection.contains("\0"));
+    Assertions.assertTrue(projection.length() <= 
EntityFieldLimits.MAX_COLUMN_COMMENT_LENGTH);
+    long columnId = stored.columns().get(0).id();
+    // A second native comment change exercises the existing-column update 
path.
+    String updated = "b\0" + "x".repeat(5000);
+    catalogManager.doWithCatalog(
+        NameIdentifier.of(metalake, catalog),
+        c -> {
+          ((TestCatalogOperations) c.ops())
+              .alterTable(ident, TableChange.updateColumnComment(new String[] 
{"c"}, updated));
+          return null;
+        });
+    Assertions.assertEquals(
+        updated, 
tableOperationDispatcher.loadTable(ident).columns()[0].comment());
+    TableEntity reloaded = entityStore.get(ident, TABLE, TableEntity.class);
+    Assertions.assertEquals(columnId, reloaded.columns().get(0).id());
+    Assertions.assertEquals(
+        ColumnEntity.commentForStorage(updated), 
reloaded.columns().get(0).comment());
+    clearInvocations(entityStore);
+    Assertions.assertEquals(
+        updated, 
tableOperationDispatcher.loadTable(ident).columns()[0].comment());
+    String changedSuffix = updated + "different suffix";
+    catalogManager.doWithCatalog(
+        NameIdentifier.of(metalake, catalog),
+        c -> {
+          ((TestCatalogOperations) c.ops())
+              .alterTable(
+                  ident, TableChange.updateColumnComment(new String[] {"c"}, 
changedSuffix));
+          return null;
+        });
+    Assertions.assertEquals(
+        changedSuffix, 
tableOperationDispatcher.loadTable(ident).columns()[0].comment());
+    verify(entityStore, never()).update(eq(ident), eq(TableEntity.class), 
eq(TABLE), any());
+  }
+
+  @Test
+  void testColumnDifferenceUsesBoundedCommentProjection() throws Exception {
+    Column source = Column.of("c", Types.StringType.get(), "a".repeat(5000));
+    ColumnEntity stored = ColumnEntity.toColumnEntity(source, 0, 1L, 
AuditInfo.EMPTY);
+    String same =
+        (String)
+            MethodUtils.invokeMethod(
+                tableOperationDispatcher, true, "columnDifferenceContent", 
source, 0, stored);
+    Assertions.assertEquals("", same);
+    Column changed = Column.of("c", Types.StringType.get(), "b\0" + 
"b".repeat(5000));
+    String difference =
+        (String)
+            MethodUtils.invokeMethod(
+                tableOperationDispatcher, true, "columnDifferenceContent", 
changed, 0, stored);
+    Assertions.assertFalse(difference.contains("\0"));
+    Assertions.assertFalse(difference.contains("b".repeat(4097)));
+    Assertions.assertTrue(difference.contains("b".repeat(4096)));
+  }
+
+  @Test
+  void testUnicodeCommentsExceedingStorageFailBeforeSourceDdl() throws 
Exception {
+    Namespace ns = Namespace.of(metalake, catalog, "unicode_comment_request");
+    schemaOperationDispatcher.createSchema(
+        NameIdentifier.of(ns.levels()), "comment", Map.of("k1", "v1", "k2", 
"v2"));
+    NameIdentifier ident = NameIdentifier.of(ns, "t");
+    TestCatalog base =
+        catalogManager.doWithCatalog(NameIdentifier.of(metalake, catalog), c 
-> (TestCatalog) c);
+    TestCatalogOperations original = (TestCatalogOperations) base.ops();
+    TestCatalogOperations source = spy(original);
+    FieldUtils.writeField(base, "ops", source, true);
+    try {
+      for (String comment : new String[] {"a".repeat(4095) + "๐Ÿ˜€", 
"๐Ÿ˜€".repeat(4096)}) {
+        IllegalArgumentException error =
+            Assertions.assertThrows(
+                IllegalArgumentException.class,
+                () ->
+                    tableOperationDispatcher.createTable(
+                        ident,
+                        new Column[] {Column.of("c", Types.StringType.get(), 
comment)},
+                        "comment",
+                        Map.of("k1", "v1", "k2", "v2"),
+                        new Transform[0]));
+        Assertions.assertTrue(error.getMessage().contains("4096 UTF-16"));
+      }
+      verify(source, never()).createTable(any(), any(), any(), any(), 
any(Transform[].class));
+      createTable(ident);
+      clearInvocations(source);
+      for (String comment : new String[] {"a".repeat(4095) + "๐Ÿ˜€", 
"๐Ÿ˜€".repeat(4096)}) {
+        Assertions.assertThrows(
+            IllegalArgumentException.class,
+            () ->
+                tableOperationDispatcher.alterTable(
+                    ident,
+                    TableChange.addColumn(
+                        new String[] {"added"}, Types.StringType.get(), 
comment)));
+        Assertions.assertThrows(
+            IllegalArgumentException.class,
+            () ->
+                tableOperationDispatcher.alterTable(
+                    ident, TableChange.updateColumnComment(new String[] 
{"col1"}, comment)));
+      }
+      verify(source, never()).alterTable(any(), any(TableChange[].class));
+    } finally {
+      FieldUtils.writeField(base, "ops", original, true);
+    }
+  }
+
+  /** Checks invalid request comments cannot partially apply source DDL. */
+  @Test
+  public void testNulColumnCommentsFailBeforeSourceDdl() throws Exception {
+    Namespace ns = Namespace.of(metalake, catalog, "nul_comment_request");
+    schemaOperationDispatcher.createSchema(
+        NameIdentifier.of(ns.levels()), "comment", Map.of("k1", "v1", "k2", 
"v2"));
+    NameIdentifier ident = NameIdentifier.of(ns, "t");
+    IllegalArgumentException createError =
+        Assertions.assertThrows(
+            IllegalArgumentException.class,
+            () ->
+                tableOperationDispatcher.createTable(
+                    ident,
+                    new Column[] {Column.of("c", Types.StringType.get(), 
"bad\0comment")},
+                    "comment",
+                    Map.of("k1", "v1", "k2", "v2"),
+                    new Transform[0]));
+    Assertions.assertTrue(createError.getMessage().contains("Column comment 
for c"));
+    Assertions.assertFalse(tableOperationDispatcher.tableExists(ident));
+    createTable(ident);
+    IllegalArgumentException addError =
+        Assertions.assertThrows(
+            IllegalArgumentException.class,
+            () ->
+                tableOperationDispatcher.alterTable(
+                    ident,
+                    TableChange.addColumn(
+                        new String[] {"added"}, Types.StringType.get(), 
"bad\0comment")));
+    Assertions.assertTrue(addError.getMessage().contains("Column comment for 
added"));
+    IllegalArgumentException alterError =
+        Assertions.assertThrows(
+            IllegalArgumentException.class,
+            () ->
+                tableOperationDispatcher.alterTable(
+                    ident, TableChange.updateColumnComment(new String[] 
{"col1"}, "bad\0comment")));
+    Assertions.assertTrue(alterError.getMessage().contains("Column comment for 
col1"));
+    Assertions.assertEquals(1, 
tableOperationDispatcher.loadTable(ident).columns().length);
+  }
+
   @Test
   public void testCreateAndListTables() throws IOException {
     Namespace tableNs = Namespace.of(metalake, catalog, "schema41");
diff --git a/core/src/test/java/org/apache/gravitino/meta/TestColumnEntity.java 
b/core/src/test/java/org/apache/gravitino/meta/TestColumnEntity.java
index 519bcfca36..6ddaddf0d6 100644
--- a/core/src/test/java/org/apache/gravitino/meta/TestColumnEntity.java
+++ b/core/src/test/java/org/apache/gravitino/meta/TestColumnEntity.java
@@ -30,6 +30,21 @@ import org.junit.jupiter.api.Test;
 
 public class TestColumnEntity {
 
+  @Test
+  void testCommentStorageProjectionBoundaries() {
+    Assertions.assertNull(ColumnEntity.commentForStorage(null));
+    Assertions.assertEquals("", ColumnEntity.commentForStorage(""));
+    Assertions.assertEquals("ab", ColumnEntity.commentForStorage("a\0b"));
+    String limit = "a".repeat(4096);
+    Assertions.assertEquals(limit, ColumnEntity.commentForStorage(limit));
+    Assertions.assertEquals(limit, ColumnEntity.commentForStorage(limit + 
"trailing"));
+    // A supplementary character must either fit completely or be excluded 
completely.
+    String prefix = "a".repeat(4095);
+    Assertions.assertEquals(prefix, ColumnEntity.commentForStorage(prefix + 
"๐Ÿ˜€"));
+    String exactFit = "a".repeat(4094) + "๐Ÿ˜€";
+    Assertions.assertEquals(exactFit, ColumnEntity.commentForStorage(exactFit 
+ "trailing"));
+  }
+
   @Test
   public void testColumnEntityFields() {
     ColumnEntity columnEntity =
diff --git a/docs/manage-relational-metadata-using-gravitino.md 
b/docs/manage-relational-metadata-using-gravitino.md
index 0876445b1a..1a25f67a14 100644
--- a/docs/manage-relational-metadata-using-gravitino.md
+++ b/docs/manage-relational-metadata-using-gravitino.md
@@ -280,3 +280,32 @@ identifiers = 
catalog.as_table_catalog().list_tables(Namespace.of("public"))
 
 </TabItem>
 </Tabs>
+
+### External column comments and existing names
+
+Loading an external table preserves its complete source column comments in the 
returned
+metadata. The entity store keeps a bounded, NUL-free projection of each column 
comment,
+without changing the source comment, column identity, or column governance 
metadata.
+The projection is limited to 4096 UTF-16 units so it fits every supported 
store.
+CREATE and ALTER requests reject NUL column comments and comments exceeding 
4096 UTF-16
+units before source DDL, including managed-table creation, ADD COLUMN and 
comment updates.
+Supplementary Unicode characters count as two units, so 2048 emoji fit while 
4096 emoji do not.
+Accepted request comments are stored without truncation. The UTF-16 bound is 
needed for H2's
+VARCHAR storage limit even though MySQL and PostgreSQL count code points. No 
store migration
+is required.
+
+Existing column references use the provider's existing-name specification; new 
columns
+and rename targets still use creation rules. PostgreSQL accepts quoted native 
column
+names while retaining its UTF-8 name-length, NUL, and wildcard restrictions. 
Other providers
+retain their default validation until they explicitly support native-name 
operations.
+
+Catalog creation takes the target catalog lock, permitting sibling catalogs to 
operate
+while source verification runs. New catalogs are initialized and validated 
before their entity
+is persisted; local cache publication is coordinated with catalog loading. 
Failed verification
+does not expose a transient catalog through listing. Table DROP holds the 
target table lock for source DDL
+and registration removal. After releasing that table lock, best-effort cleanup 
checks schema
+existence under a read lock and only takes a write lock if the schema is 
missing, rechecking
+before deleting orphaned registrations. This does not add SQL statement 
timeouts or
+cancel blocked source DDL. Cold table import retains its existing locking 
behavior. Table PURGE
+and view DROP also retain their schema write lock across source DDL and can 
still block sibling
+operations; this change narrows only table DROP.
diff --git 
a/trino-connector/integration-test/src/test/java/org/apache/gravitino/trino/connector/integration/test/TestTrinoQueryIT.java
 
b/trino-connector/integration-test/src/test/java/org/apache/gravitino/trino/connector/integration/test/TestTrinoQueryIT.java
new file mode 100644
index 0000000000..359ef7bfa9
--- /dev/null
+++ 
b/trino-connector/integration-test/src/test/java/org/apache/gravitino/trino/connector/integration/test/TestTrinoQueryIT.java
@@ -0,0 +1,194 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.gravitino.trino.connector.integration.test;
+
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.time.Duration;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+import org.mockito.Mockito;
+
+class TestTrinoQueryIT {
+  private static final String TABLES = "\"tb01\"\n\"tb02\"";
+  private static final String STAGING_TABLES = TABLES + 
"\n\"tmp_trino_staging\"";
+  private static final Duration TIMEOUT = Duration.ofSeconds(2);
+
+  @Test
+  void testGenerationWaitsForCleanup(@TempDir Path directory) throws Exception 
{
+    Files.writeString(directory.resolve("rollback.sql"), "<RETRY_UNTIL_MATCH> 
SHOW TABLES;\n");
+    Files.writeString(directory.resolve("rollback.txt"), TABLES + "\n\n");
+    TrinoQueryRunner runner = Mockito.mock(TrinoQueryRunner.class);
+    Mockito.when(runner.runQuery("SHOW TABLES")).thenReturn(STAGING_TABLES, 
TABLES);
+    new TrinoQueryIT().executeSqlFileWithGenOutput(directory.toString(), 
"rollback.sql", runner);
+    Assertions.assertEquals(TABLES + "\n", 
Files.readString(directory.resolve("rollback.txt")));
+    Mockito.verify(runner, Mockito.times(2)).runQuery("SHOW TABLES");
+  }
+
+  @Test
+  void testGenerationCanReuseFinalExpectedResult(@TempDir Path directory) 
throws Exception {
+    Files.writeString(directory.resolve("rollback.sql"), "<RETRY_UNTIL_MATCH> 
SHOW TABLES;\n");
+    for (int trailingNewlines = 0; trailingNewlines <= 2; trailingNewlines++) {
+      Files.writeString(directory.resolve("rollback.txt"), TABLES + 
"\n".repeat(trailingNewlines));
+      TrinoQueryRunner runner = Mockito.mock(TrinoQueryRunner.class);
+      Mockito.when(runner.runQuery("SHOW TABLES"))
+          .thenReturn(STAGING_TABLES, TABLES, STAGING_TABLES, TABLES);
+      TrinoQueryIT test = new TrinoQueryIT();
+      for (int generation = 0; generation < 2; generation++) {
+        test.executeSqlFileWithGenOutput(directory.toString(), "rollback.sql", 
runner);
+        Assertions.assertEquals(TABLES + "\n", 
Files.readString(directory.resolve("rollback.txt")));
+      }
+      Mockito.verify(runner, Mockito.times(4)).runQuery("SHOW TABLES");
+    }
+  }
+
+  @Test
+  void testGenerationRequiresExistingRetryExpectation(@TempDir Path directory) 
throws Exception {
+    Files.writeString(
+        directory.resolve("rollback.sql"),
+        "CREATE TABLE t (i INT);\n<RETRY_UNTIL_MATCH> SHOW TABLES;\n");
+    TrinoQueryRunner runner = Mockito.mock(TrinoQueryRunner.class);
+    Mockito.when(runner.runQuery(Mockito.anyString())).thenReturn("ok");
+    Assertions.assertThrows(
+        IllegalArgumentException.class,
+        () ->
+            new TrinoQueryIT()
+                .executeSqlFileWithGenOutput(directory.toString(), 
"rollback.sql", runner));
+    Mockito.verifyNoInteractions(runner);
+    Assertions.assertFalse(Files.exists(directory.resolve("rollback.txt")));
+  }
+
+  @Test
+  void testGenerationTimeoutPreservesGoldenFile(@TempDir Path directory) 
throws Exception {
+    Files.writeString(directory.resolve("rollback.sql"), "<RETRY_UNTIL_MATCH> 
SHOW TABLES;\n");
+    String previous = TABLES + "\n\n";
+    Files.writeString(directory.resolve("rollback.txt"), previous);
+    TrinoQueryRunner runner = Mockito.mock(TrinoQueryRunner.class);
+    Mockito.when(runner.runQuery("SHOW TABLES")).thenReturn(STAGING_TABLES);
+    IllegalStateException error =
+        Assertions.assertThrows(
+            IllegalStateException.class,
+            () ->
+                new TrinoQueryIT()
+                    .executeSqlFileWithGenOutput(
+                        directory.toString(), "rollback.sql", runner, 
Duration.ofMillis(300)));
+    Assertions.assertTrue(error.getMessage().contains("golden file 
unchanged"));
+    Assertions.assertEquals(previous, 
Files.readString(directory.resolve("rollback.txt")));
+    Mockito.verify(runner, Mockito.atLeast(2)).runQuery("SHOW TABLES");
+  }
+
+  @Test
+  void testGenerationKeepsStatementExpectationsAligned(@TempDir Path 
directory) throws Exception {
+    Files.writeString(
+        directory.resolve("rollback.sql"), "SHOW SCHEMAS;\n<RETRY_UNTIL_MATCH> 
SHOW TABLES;\n");
+    Files.writeString(directory.resolve("rollback.txt"), "\"old_schema\"\n\n" 
+ TABLES + "\n\n");
+    TrinoQueryRunner runner = Mockito.mock(TrinoQueryRunner.class);
+    Mockito.when(runner.runQuery("SHOW SCHEMAS")).thenReturn("\"new_schema\"");
+    Mockito.when(runner.runQuery("SHOW TABLES")).thenReturn(STAGING_TABLES, 
TABLES);
+    new TrinoQueryIT().executeSqlFileWithGenOutput(directory.toString(), 
"rollback.sql", runner);
+    Assertions.assertEquals(
+        "\"new_schema\"\n\n" + TABLES + "\n", 
Files.readString(directory.resolve("rollback.txt")));
+    Mockito.verify(runner).runQuery("SHOW SCHEMAS");
+    Mockito.verify(runner, Mockito.times(2)).runQuery("SHOW TABLES");
+  }
+
+  @Test
+  void testUnmarkedGenerationNeedsNoExistingGolden(@TempDir Path directory) 
throws Exception {
+    Files.writeString(directory.resolve("plain.sql"), "SHOW TABLES;\n");
+    TrinoQueryRunner runner = Mockito.mock(TrinoQueryRunner.class);
+    Mockito.when(runner.runQuery("SHOW TABLES")).thenReturn(TABLES);
+    new TrinoQueryIT().executeSqlFileWithGenOutput(directory.toString(), 
"plain.sql", runner);
+    Assertions.assertEquals(TABLES + "\n", 
Files.readString(directory.resolve("plain.txt")));
+    Mockito.verify(runner).runQuery("SHOW TABLES");
+  }
+
+  @Test
+  void testGenerationRejectsMarkedWritesBeforeExecutingSql(@TempDir Path 
directory)
+      throws Exception {
+    Files.writeString(
+        directory.resolve("writes.sql"),
+        "SHOW TABLES;\n<RETRY_UNTIL_MATCH> INSERT INTO t VALUES (1);\n");
+    String previous = TABLES + "\n\n1\n\n";
+    Files.writeString(directory.resolve("writes.txt"), previous);
+    TrinoQueryRunner runner = Mockito.mock(TrinoQueryRunner.class);
+    Assertions.assertThrows(
+        IllegalArgumentException.class,
+        () ->
+            new TrinoQueryIT()
+                .executeSqlFileWithGenOutput(directory.toString(), 
"writes.sql", runner));
+    Mockito.verifyNoInteractions(runner);
+    Assertions.assertEquals(previous, 
Files.readString(directory.resolve("writes.txt")));
+  }
+
+  @Test
+  void testSqlFileWaitsForCleanup(@TempDir Path directory) throws Exception {
+    Files.writeString(
+        directory.resolve("rollback.sql"),
+        "-- Wait for transaction rollback.\n<RETRY_UNTIL_MATCH> SHOW 
TABLES;\n");
+    Files.writeString(directory.resolve("rollback.txt"), TABLES + "\n\n");
+    TrinoQueryIT.getTesterCount(directory.toString(), "pg", "rollback");
+    TrinoQueryRunner runner = Mockito.mock(TrinoQueryRunner.class);
+    Mockito.when(runner.runQuery("SHOW TABLES")).thenReturn(STAGING_TABLES, 
TABLES);
+    new TrinoQueryIT()
+        .executeSqlFileWithCheckResult(directory.toString(), "rollback.sql", 
runner, "pg");
+    Mockito.verify(runner, Mockito.times(2)).runQuery("SHOW TABLES");
+  }
+
+  @Test
+  void testWaitsForStagingTableCleanup() {
+    TrinoQueryRunner runner = Mockito.mock(TrinoQueryRunner.class);
+    Mockito.when(runner.runQuery("SHOW TABLES")).thenReturn(STAGING_TABLES, 
TABLES);
+    Assertions.assertEquals(
+        TABLES,
+        TrinoQueryIT.executeQuery(runner, "<RETRY_UNTIL_MATCH> SHOW TABLES", 
TABLES, TIMEOUT));
+    Mockito.verify(runner, Mockito.times(2)).runQuery("SHOW TABLES");
+  }
+
+  @Test
+  void testPersistentLeakRemainsAMismatch() {
+    TrinoQueryRunner runner = Mockito.mock(TrinoQueryRunner.class);
+    Mockito.when(runner.runQuery("SHOW TABLES")).thenReturn(STAGING_TABLES);
+    String result =
+        TrinoQueryIT.executeQuery(runner, "<RETRY_UNTIL_MATCH> SHOW TABLES", 
TABLES, TIMEOUT);
+    Assertions.assertEquals(STAGING_TABLES, result);
+    Assertions.assertFalse(TrinoQueryIT.match(TABLES, result));
+    Mockito.verify(runner, Mockito.atLeast(2)).runQuery("SHOW TABLES");
+  }
+
+  @Test
+  void testUnmarkedQueryRunsOnce() {
+    TrinoQueryRunner runner = Mockito.mock(TrinoQueryRunner.class);
+    Mockito.when(runner.runQuery("SHOW TABLES")).thenReturn(STAGING_TABLES, 
TABLES);
+    Assertions.assertEquals(
+        STAGING_TABLES, TrinoQueryIT.executeQuery(runner, "SHOW TABLES", 
TABLES, TIMEOUT));
+    Mockito.verify(runner).runQuery("SHOW TABLES");
+  }
+
+  @Test
+  void testRejectsRetryingWrites() {
+    TrinoQueryRunner runner = Mockito.mock(TrinoQueryRunner.class);
+    Assertions.assertThrows(
+        IllegalArgumentException.class,
+        () ->
+            TrinoQueryIT.executeQuery(
+                runner, "<RETRY_UNTIL_MATCH> INSERT INTO t VALUES (1)", "", 
TIMEOUT));
+    Mockito.verifyNoInteractions(runner);
+  }
+}
diff --git 
a/trino-connector/integration-test/src/test/java/org/apache/gravitino/trino/connector/integration/test/TrinoQueryIT.java
 
b/trino-connector/integration-test/src/test/java/org/apache/gravitino/trino/connector/integration/test/TrinoQueryIT.java
index 5ef9e7b532..141badcf9c 100644
--- 
a/trino-connector/integration-test/src/test/java/org/apache/gravitino/trino/connector/integration/test/TrinoQueryIT.java
+++ 
b/trino-connector/integration-test/src/test/java/org/apache/gravitino/trino/connector/integration/test/TrinoQueryIT.java
@@ -19,13 +19,16 @@
 package org.apache.gravitino.trino.connector.integration.test;
 
 import java.io.File;
-import java.io.FileOutputStream;
 import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.time.Duration;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.HashMap;
 import java.util.HashSet;
 import java.util.List;
+import java.util.Locale;
 import java.util.Map;
 import java.util.Set;
 import java.util.TreeMap;
@@ -37,11 +40,13 @@ import java.util.concurrent.Executors;
 import java.util.concurrent.Future;
 import java.util.concurrent.TimeUnit;
 import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
 import java.util.regex.Matcher;
 import java.util.regex.Pattern;
 import org.apache.gravitino.integration.test.util.ITUtils;
 import org.apache.logging.log4j.util.Strings;
 import org.awaitility.Awaitility;
+import org.awaitility.core.ConditionTimeoutException;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -57,6 +62,9 @@ public class TrinoQueryIT extends TrinoQueryITBase {
   private static final Map<String, TestStatus> allTestStatus = new TreeMap<>();
 
   private static final int testParallelism = 2;
+  private static final String RETRY_FLAG = "<RETRY_UNTIL_MATCH>";
+  private static final Pattern RESULT_PATTERN =
+      Pattern.compile("((\".*?\")\\n{2,})|((\\S.*?)\\n{2,})", Pattern.DOTALL);
   private static final Pattern PARAM_PATTERN = 
Pattern.compile("\\$\\{(\\??[\\w_]+)\\}");
   private static final Pattern QUERY_FAILED_PATTERN = Pattern.compile("^Query 
\\w+ failed:");
 
@@ -232,26 +240,16 @@ public class TrinoQueryIT extends TrinoQueryITBase {
 
     Matcher sqlMatcher =
         Pattern.compile("([<\\w].*?);", Pattern.DOTALL | 
Pattern.UNIX_LINES).matcher(sqls);
-    Matcher resultMatcher =
-        Pattern.compile("((\".*?\")\\n{2,})|((\\S.*?)\\n{2,})", Pattern.DOTALL 
| Pattern.UNIX_LINES)
-            .matcher(testResults);
+    Matcher resultMatcher = RESULT_PATTERN.matcher(testResults);
 
     TestStatus testStatus = allTestStatus.get(TestStatus.testName(path, 
catalog));
     testStatus.startTime = System.currentTimeMillis();
     while (sqlMatcher.find()) {
       String sql = sqlMatcher.group(1);
       sql = resolveParameters(sql);
-      String expectResult = "";
-      if (resultMatcher.find()) {
-        if (resultMatcher.group(2) != null) {
-          expectResult = resultMatcher.group(2).trim();
-        } else {
-          expectResult = resultMatcher.group(4).trim();
-        }
-      }
+      String expectResult = nextExpectedResult(resultMatcher);
 
-      String result = queryRunner.runQuery(sql).trim();
-      result = result.replaceAll("WARNING:.*?\\n", "");
+      String result = executeQuery(queryRunner, sql, expectResult, 
Duration.ofSeconds(30));
       boolean match = match(expectResult, result);
 
       if (match) {
@@ -279,6 +277,31 @@ public class TrinoQueryIT extends TrinoQueryITBase {
     LOG.info("Test progress {}/{}", passCount.get(), totalCount.get());
   }
 
+  // Only explicitly marked read queries wait for asynchronous connector 
cleanup. Keep the last
+  // result on timeout so the caller reports the expected/actual mismatch 
instead of hiding a leak.
+  static String executeQuery(
+      TrinoQueryRunner runner, String sql, String expected, Duration timeout) {
+    if (!sql.startsWith(RETRY_FLAG)) {
+      return runner.runQuery(sql).trim().replaceAll("WARNING:.*?\\n", "");
+    }
+    String query = retryQuery(sql);
+    AtomicReference<String> result = new AtomicReference<>("");
+    try {
+      Awaitility.await()
+          .atMost(timeout)
+          .pollDelay(Duration.ZERO)
+          .pollInterval(Duration.ofMillis(100))
+          .until(
+              () -> {
+                
result.set(runner.runQuery(query).trim().replaceAll("WARNING:.*?\\n", ""));
+                return match(expected, result.get());
+              });
+    } catch (ConditionTimeoutException e) {
+      // The existing result check will fail with the final query output.
+    }
+    return result.get();
+  }
+
   /**
    * * This method is used to match the result of the query. There are three 
cases: 1. The expected
    * result is equal to the actual result. 2. The expected result is a query 
failed result, and the
@@ -485,30 +508,58 @@ public class TrinoQueryIT extends TrinoQueryITBase {
 
   void executeSqlFileWithGenOutput(
       String testSetDirName, String filename, TrinoQueryRunner queryRunner) 
throws Exception {
-    String path = ITUtils.joinPath(testSetDirName, filename);
-    String sqls = TrinoQueryITBase.readFileToString(path);
-    sqls = removeSqlComments(sqls);
-    String resultFileName = path.replace(".sql", ".txt");
-    FileOutputStream outputStream = new FileOutputStream(resultFileName);
+    executeSqlFileWithGenOutput(testSetDirName, filename, queryRunner, 
Duration.ofSeconds(30));
+  }
 
+  void executeSqlFileWithGenOutput(
+      String testSetDirName, String filename, TrinoQueryRunner queryRunner, 
Duration timeout)
+      throws Exception {
+    String path = ITUtils.joinPath(testSetDirName, filename);
+    String sqls = removeSqlComments(TrinoQueryITBase.readFileToString(path));
+    Path resultFile = Path.of(path.replace(".sql", ".txt"));
+    // Read the trusted baseline before opening the output. A transient 
staging table must never
+    // become its own expected result, nor may a failed regeneration damage 
the old golden file.
+    String previousResults = Files.exists(resultFile) ? 
Files.readString(resultFile) : "";
+    // Match the check path's EOF normalization, including files written by 
this generator.
+    Matcher resultMatcher = RESULT_PATTERN.matcher(previousResults + "\n\n");
     Matcher sqlMatcher =
-        Pattern.compile("(\\w.*?);", Pattern.DOTALL | 
Pattern.UNIX_LINES).matcher(sqls);
-
-    boolean firstLine = true;
+        Pattern.compile("([<\\w].*?);", Pattern.DOTALL | 
Pattern.UNIX_LINES).matcher(sqls);
+    List<String> queries = new ArrayList<>();
+    List<String> expectations = new ArrayList<>();
     while (sqlMatcher.find()) {
-      if (!firstLine) {
-        outputStream.write("\n".getBytes(StandardCharsets.UTF_8));
+      String sql = resolveParameters(sqlMatcher.group(1));
+      String expected = nextExpectedResult(resultMatcher);
+      if (sql.startsWith(RETRY_FLAG)) {
+        retryQuery(sql);
+        if (expected.isEmpty()) {
+          throw new IllegalArgumentException(
+              "RETRY_UNTIL_MATCH requires an existing expected result in " + 
resultFile);
+        }
       }
-      firstLine = false;
+      queries.add(sql);
+      expectations.add(expected);
+    }
 
-      String sql = resolveParameters(sqlMatcher.group(1));
-      String result = queryRunner.runQuery(sql).trim();
+    StringBuilder output = new StringBuilder();
+    for (int i = 0; i < queries.size(); i++) {
+      String sql = queries.get(i);
+      String result;
+      if (sql.startsWith(RETRY_FLAG)) {
+        result = executeQuery(queryRunner, sql, expectations.get(i), timeout);
+        if (!match(expectations.get(i), result)) {
+          throw new IllegalStateException(
+              "Timed out waiting for " + sql + "; golden file unchanged: " + 
resultFile);
+        }
+      } else {
+        result = queryRunner.runQuery(sql).trim();
+      }
       LOG.info("Execute sql:\n{}\nResult:\n{}", sql, result);
-      outputStream.write(result.getBytes(StandardCharsets.UTF_8));
-      outputStream.write("\n".getBytes(StandardCharsets.UTF_8));
+      if (i > 0) {
+        output.append('\n');
+      }
+      output.append(result).append('\n');
     }
-
-    outputStream.close();
+    Files.writeString(resultFile, output, StandardCharsets.UTF_8);
   }
 
   static int getTesterCount(String testSetDirName, String catalog, String 
testFilterPrefix)
@@ -574,4 +625,19 @@ public class TrinoQueryIT extends TrinoQueryITBase {
       return catalog + ":" + simpleTesterName(filename);
     }
   }
+
+  private static String nextExpectedResult(Matcher matcher) {
+    if (!matcher.find()) {
+      return "";
+    }
+    return (matcher.group(2) != null ? matcher.group(2) : 
matcher.group(4)).trim();
+  }
+
+  private static String retryQuery(String sql) {
+    String query = sql.substring(RETRY_FLAG.length()).trim();
+    if (!query.toUpperCase(Locale.ROOT).startsWith("SHOW ")) {
+      throw new IllegalArgumentException("RETRY_UNTIL_MATCH only supports SHOW 
queries");
+    }
+    return query;
+  }
 }
diff --git 
a/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/jdbc-postgresql/00006_datatype.sql
 
b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/jdbc-postgresql/00006_datatype.sql
index 29b4edeae7..d3c1d0da0f 100644
--- 
a/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/jdbc-postgresql/00006_datatype.sql
+++ 
b/trino-connector/integration-test/src/test/resources/trino-ci-testset/testsets/jdbc-postgresql/00006_datatype.sql
@@ -71,8 +71,9 @@ INSERT INTO tb02 (f1, f2, f3, f4, f5, f6, f7, f9, f10, f11, 
f12, f13, f14, f15,
 VALUES ('Sample text 1', 'same9', x'65', 123.456, 7.89, 12.34, false, 100, 
1000, 1000, 1992382342, DATE '2024-01-01',
         NULL, TIMESTAMP '2024-01-01 08:00:00', TIMESTAMP '2024-01-01 08:00:00 
UTC');
 
--- Failed inserts must roll back their JDBC staging tables.
-SHOW TABLES;
+-- Failed inserts must roll back their JDBC staging tables. Trino aborts 
transactions
+-- asynchronously, so wait for cleanup while still failing if staging tables 
remain.
+<RETRY_UNTIL_MATCH> SHOW TABLES;
 
 CREATE TABLE tb03  (int_array ARRAY(INTEGER));
 

Reply via email to