mchades commented on code in PR #13771:
URL: https://github.com/apache/gravitino/pull/13771#discussion_r4254114048
##########
catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/IcebergCatalogCapability.java:
##########
@@ -48,6 +60,21 @@ public CapabilityResult supportsHierarchicalSchema() {
return CapabilityResult.SUPPORTED;
}
+ @Override
+ public CapabilityResult caseSensitiveOnName(Scope scope) {
+ if (hiveBackend) {
+ switch (scope) {
+ case SCHEMA:
+ case TABLE:
Review Comment:
**[P1] Preserve access to existing mixed-case registrations**
With PostgreSQL metadata storage, a Hive-backed Iceberg table created as
`Orders` before this change is stored under that spelling: the old capability
preserves case and `internalCreateTable` saves `ident.name()` directly. This
rule now normalizes every request, including `Orders`, to `orders`, so the
case-sensitive `table_name = #{tableName}` lookup cannot find the existing row.
Import then conflicts with its stable ID, and the reload still misses the row.
The table becomes unloadable regardless of the spelling the user supplies;
schema registrations have the same issue. Migrate or compatibly resolve
existing registrations when enabling normalization, preserving their IDs and
associated relations.
##########
catalogs/catalog-lakehouse-iceberg/src/main/java/org/apache/gravitino/catalog/lakehouse/iceberg/IcebergCatalogCapability.java:
##########
@@ -48,6 +60,21 @@ public CapabilityResult supportsHierarchicalSchema() {
return CapabilityResult.SUPPORTED;
}
+ @Override
+ public CapabilityResult caseSensitiveOnName(Scope scope) {
+ if (hiveBackend) {
+ switch (scope) {
+ case SCHEMA:
+ case TABLE:
+ case COLUMN:
Review Comment:
**[P2] Preserve case-sensitive Iceberg column names**
An existing Iceberg table can contain a column named `UserID`, which
`IcebergTable.fromIcebergTable` preserves. Marking `COLUMN` as case-insensitive
makes `TableNormalizeDispatcher.alterTable` rewrite changes targeting `UserID`
to `userid`. However, `IcebergCatalogWrapperHelper` still uses exact-name
`Schema.findField` and the default case-sensitive `UpdateSchema` behavior in
Iceberg 1.11.0. Previously valid column deletion, renaming, or type changes
therefore fail with a missing-column error. Preserve column spelling and
restrict normalization to the schema/table names that Hive actually treats as
case-insensitive.
##########
core/src/main/java/org/apache/gravitino/catalog/TopicOperationDispatcher.java:
##########
@@ -309,11 +320,24 @@ private void importTopic(NameIdentifier identifier) {
.build();
try {
- store.put(topicEntity, true);
+ // Never overwrite imported topic metadata because the stable ID may be
owned by another
+ // Gravitino path.
+ store.put(topicEntity, false);
Review Comment:
**[P1] Provide a recoverable path for retained Kafka topic IDs**
After a Kafka catalog has imported a topic, removing and re-registering the
catalog soft-deletes the stored row but leaves the external topic intact. This
create-only insert then conflicts with the retained `topic_id`. The new error
asks the operator to remove the external `gravitino.identifier`, but
`KafkaCatalogOperations.loadTopic` derives that value from the Kafka topic UUID
on every load via `convertToGravitinoId`; it is not a removable topic property.
Consequently, the documented recovery cannot work, and loads remain blocked
until physical cleanup removes the old row (seven-day default retention).
Provide an identity recovery or rebinding path that does not require deleting
the external topic and its data.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
