JingsongLi commented on code in PR #9415:
URL: https://github.com/apache/paimon/pull/9415#discussion_r3879081319
##########
paimon-spark/paimon-spark-common/src/main/scala/org/apache/paimon/spark/catalyst/analysis/PaimonOutputResolver.scala:
##########
@@ -478,6 +501,36 @@ object PaimonOutputResolver extends SQLConfHelper {
attr.withDataType(CharVarcharUtils.getRawType(attr.metadata).getOrElse(attr.dataType))
}
+ private def renameFieldsInStruct(input: StructType, expected: StructType):
StructType = {
+ if (input.length == expected.length) {
Review Comment:
[P1] Preserve positional mapping when nested field counts differ
Returning `input` here falls back to the downstream by-name resolver
whenever merge-schema fills a missing nested field. For target `ARRAY<STRUCT<x:
INT, y: INT, z: INT>>`, a column-list write whose input struct fields are
`(y=20, x=10)` stores `[10, 20, null]` instead of the positional `[20, 10,
null]`. I reproduced this on Spark 3.5 with
`spark.paimon.write.merge-schema=true`. Please rename the common ordinal prefix
even when the lengths differ, preserve unmatched input fields, and let the
existing strict/merge-schema handling process missing or extra fields. A
regression test with unequal nested field counts would cover this path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
