seunggabi opened a new pull request, #19042:
URL: https://github.com/apache/iceberg/pull/19042
## Summary
`iceberg.tables.schema-case-insensitive` already makes `RecordConverter`
look up table columns by case-insensitive name, but
`RecordUtils.createTableWriter` resolved the configured `id-columns` with the
case-sensitive `Schema.findField`. With the option enabled, a record field `ID`
is written into column `id`, yet `id-columns=ID` fails with `ID column not
found: ID`.
This PR makes the id-columns lookup follow the same option and makes the
failure easier to act on.
## Changes
- When `iceberg.tables.schema-case-insensitive=true`, resolve `id-columns`
with `Schema.caseInsensitiveFindField`. The default (case-sensitive) behavior
is unchanged.
- The error now names the table and lists its top-level columns:
`ID column not found: ID (table: default.events, columns: id, data, id2)`.
The message still starts with the old text, so existing log searches keep
matching.
- Docs: the `schema-case-insensitive` row now says it also applies to ID
columns.
## Background
I hit this with CDC tables whose source column names were not normalized.
Before the code moved here, I had sent a similar fix to
`databricks/iceberg-kafka-connect` (databricks/iceberg-kafka-connect#349),
which I closed because that repository is no longer maintained. The NPE part of
that report is already fixed in this repository; the case handling and the
error detail are not.
## Testing
- `TestRecordUtils.createTableWriterWithNonMatchingNamespace`: updated for
the new message.
- `TestRecordUtils.createTableWriterWithIdColumnCaseMismatch`:
case-sensitive lookup still fails, with the table and columns in the message.
- `TestRecordUtils.createTableWriterWithIdColumnCaseInsensitive`: with the
option enabled, `id-columns=ID` resolves to `id`.
- Ran `./gradlew :iceberg-kafka-connect:iceberg-kafka-connect:spotlessApply
:iceberg-kafka-connect:iceberg-kafka-connect:test --tests
org.apache.iceberg.connect.data.TestRecordUtils` on JDK 21.
## AI disclosure
I used Claude Code to write the code change, the tests, and this
description, and to run the tests above locally.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]