annurahar opened a new issue, #15392:
URL: https://github.com/apache/iceberg/issues/15392
### Apache Iceberg version
1.10.1 (latest release)
### Query engine
None
### Please describe the bug 🐞
When using Kafka Connect with both:
1. Name mapping configured via `schema.name-mapping.default` table property
2. Case-insensitive schema matching enabled via
`iceberg.tables.schema-case-insensitive=true`
The case-insensitive setting is ignored during field lookup. This causes
records with field names that differ only in case (e.g., `"II"` vs `"ii"`) to
fail to match the existing column.
## Root Cause
In `RecordConverter.java`, the `lookupStructField` and `createStructNameMap`
methods do not apply case-insensitive logic when name mapping is present:
**Current behavior in `lookupStructField`:**
```java
if (nameMapping == null) {
return config.schemaCaseInsensitive()
? schema.caseInsensitiveField(fieldName)
: schema.field(fieldName);
}
// Case sensitivity is NOT applied here when nameMapping is present
return structNameMap
.computeIfAbsent(structFieldId, notUsed -> createStructNameMap(schema))
.get(fieldName); // Uses original fieldName without lowercasing
```
**Current behavior in `createStructNameMap`:**
```java
if (mappedField != null && !mappedField.names().isEmpty()) {
mappedField.names().forEach(name -> map.put(name, col)); // No case
handling
} else {
map.put(col.name(), col); // No case handling
}
```
## Expected behavior
When `iceberg.tables.schema-case-insensitive=true` is configured, field
lookups should be case-insensitive regardless of whether name mapping is
present or not.
## Steps to reproduce
1. Create an Iceberg table with a name mapping that has an alias with
different case (e.g., alias `"II"` for column `"ii"`)
2. Configure Kafka Connect with `iceberg.tables.schema-case-insensitive=true`
3. Send a record with the field name in lowercase (e.g., `{"ii": 123}`)
4. **Expected**: Field should be matched to existing column and written to
the table
5. **Actual**: Field is not matched to existing column
## Impact
During schema evolution, the unmatched field is treated as a new field,
causing a duplicate column to be added to the table (e.g., both `"II"` and
`"ii"` columns exist). This results in:
- Unintended schema changes with redundant columns
- Data written to wrong columns
- Table schema corruption over time
## Suggested Fix
Modify `lookupStructField` and `createStructNameMap` to apply
case-insensitive logic when name mapping is present.
### Willingness to contribute
- [x] I can contribute a fix for this bug independently
- [ ] I would be willing to contribute a fix for this bug with guidance from
the Iceberg community
- [ ] I cannot contribute a fix for this bug at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
