LuciferYang opened a new pull request, #9568:
URL: https://github.com/apache/paimon/pull/9568
### Purpose
close #9567
`buildFieldsList` descends through an array's wrapper groups by comparing
each group's name with the table field name:
```java
while (!Objects.equals(groupColumnIO.getName(), fieldName)) {
groupColumnIO = (GroupColumnIO) groupColumnIO.getChild(0);
}
```
That comparison is case sensitive, while the two other name lookups in the
same file, `lookupColumnByName` and `getTypeIgnoreCase`, ignore case. With
catalog `case-sensitive = false` the requested schema keeps the file's
spelling, so a column the file spells `Tags` read as `tags` never matched: the
loop walked `Tags` to `list` to `element`, and casting the element's
`PrimitiveColumnIO` to `GroupColumnIO` threw `ClassCastException`. This
compares without case here too.
`lookupColumnByName` has ignored case since #5905, and `getTypeIgnoreCase`
was added by #8337, which is where case-insensitive reads came from. Its tests
cover scalar and nested-row columns but no array column, which is why this went
unnoticed. MAP and MULTISET are not affected, since `getMapKeyValueColumn`
descends structurally on `getChildrenCount() == 1` and never compares a name.
One thing I deliberately left alone. The loop's only exit is the name
matching, so if a column's name ever fails to match for some reason other than
case, it still descends until the cast fails. After this change the loop body
is unreachable for every caller in the repo today: the top-level and
declared-ROW-child paths look the column up by the same name they then compare,
the file-only extra ROW fields carry `parquetType.getName()` verbatim, and the
array and map element recursions pass an empty field name, which short-circuits
before the loop. That is how it happens to be today rather than something
enforced, so a guard may well be worth adding, but turning an unreachable state
into a `break` would swap a crash for silently reading the wrong element
column, and adding a thrown exception is a change of its own. Happy to follow
up separately.
### Tests
`ParquetCaseInsensitiveReadTest.testArrayCaseInsensitiveColumnMatching`
writes a three-level LIST spelled `Tags` and reads it as `tags` with
`caseSensitive=false`. The two rows hold two and three elements, so the
assertions separate a wrong offset from a wrong length. The column is genuinely
resolved rather than null-filled: a name miss would leave the value null and
`first.size()` would throw instead of comparing.
Against the unfixed reader it fails with `ClassCastException:
PrimitiveColumnIO cannot be cast to GroupColumnIO`.
`mvn -pl paimon-format test` on JDK 8: 597 tests, 0 failures.
`spotless:check` and `checkstyle:check` are clean.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
