yuqi1129 opened a new issue, #12237:
URL: https://github.com/apache/gravitino/issues/12237
### Version
main branch (also affects any deployment with two Iceberg JDBC catalogs on
the same backend `uri`)
### Describe what's wrong
`IcebergClassLoaderPoolIT.testDropOneSharedCatalogKeepsSiblingUsable()`
fails intermittently in the MySQL backend integration test.
The test deliberately places two Iceberg JDBC catalogs on the **same**
backend `uri` (DB_A), so they share one `iceberg_tables` control table.
Gravitino defaults `jdbc.schema-version=V1`
(`IcebergCatalogUtil#loadJdbcCatalog`, `putIfAbsent(..., "V1")`), so every
`JdbcCatalog.initialize` runs Iceberg's V1 view-support migration. That
migration probes for the column with `getColumns(null, null, "iceberg_tables",
"iceberg_type")` and, if the probe reports it missing, runs a non-idempotent
`ALTER TABLE iceberg_tables ADD COLUMN iceberg_type` (no `IF NOT EXISTS`). When
the column already exists on the shared/persisted table but the probe misses
it, MySQL rejects the ALTER.
This is not test-only: any real deployment that loads two Iceberg JDBC
catalogs pointing at the same JDBC `uri` shares `iceberg_tables` and can hit
the same non-idempotent migration.
### Error message and/or stacktrace
```
IcebergClassLoaderPoolIT > testDropOneSharedCatalogKeepsSiblingUsable()
FAILED
java.lang.RuntimeException: Failed to operate object operation [LIST]
under [clp_drop_2_...],
reason [Cannot check and eventually update SQL schema]
Caused by: java.sql.SQLSyntaxErrorException: Duplicate column name
'iceberg_type'
at
org.apache.iceberg.jdbc.JdbcCatalog.executeV1CatalogUpdate(JdbcCatalog.java:276)
at
org.apache.iceberg.jdbc.JdbcCatalog.updateSchemaIfRequired(JdbcCatalog.java:235)
at org.apache.iceberg.jdbc.JdbcCatalog.initialize(JdbcCatalog.java:157)
at
org.apache.gravitino.iceberg.common.ClosableJdbcCatalog.initialize(ClosableJdbcCatalog.java:76)
```
### How to reproduce
1. Run the MySQL backend integration test
`:catalogs:catalog-lakehouse-iceberg:test` (deploy mode, `JDK17-deploy-mysql`).
2. `IcebergClassLoaderPoolIT` creates multiple Iceberg JDBC catalogs on the
same MySQL `uri`; the shared `iceberg_tables` in DB_A persists across test
methods/runs (cleanup only drops metalakes, never the control table).
3. On a subsequent init the V1 migration re-fires the `ADD COLUMN
iceberg_type` against a table that already has the column, producing `Duplicate
column name`.
### Additional context
- Iceberg 1.11.0; MySQL backend. Migration precondition is met because
Gravitino defaults `jdbc.schema-version=V1`.
- The migration is Iceberg-side and non-idempotent; the practical fix
belongs in Gravitino's JDBC catalog init (`ClosableJdbcCatalog#initialize`) —
treat "column already exists" during the V1 view-migration as benign (schema
already migrated by another catalog on the same `uri`), rather than failing
init.
- Surfaced by the newly added `IcebergClassLoaderPoolIT` (PR #10480), which
is why it appears now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
