LuciferYang opened a new issue, #9512: URL: https://github.com/apache/paimon/issues/9512
### Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon/issues) and found nothing similar. ### Paimon version master, `9c7deebbd` (2.1-SNAPSHOT) ### Compute Engine Java API (`paimon-common`). The class generates partition specs and partition directory names, so every engine reaches it. ### Minimal reproduce step `InternalRowPartitionComputer`'s constructor sizes `partitionFieldGetters` and `partitionCastExecutors` by the number of partition columns, then stores into them at the partition column's position in the full row schema: ```java this.partitionFieldGetters = new FieldGetter[partitionColumns.length]; this.partitionCastExecutors = new CastExecutor[partitionColumns.length]; for (String partitionColumn : partitionColumns) { int i = columnList.indexOf(partitionColumn); DataType type = rowType.getTypeAt(i); partitionFieldGetters[i] = createNullCheckingFieldGetter(type, i); partitionCastExecutors[i] = CastExecutors.resolve(type, VarCharType.STRING_TYPE); } ``` `generatePartValues` then reads `partitionFieldGetters[i]`, `partitionCastExecutors[i]` and `partitionColumns[i]` with a single loop variable running over the partition columns, so the array bound and the index expression have to agree. They only agree when the partition columns are a prefix of the row type, in row order. ```java RowType rowType = RowType.of( new DataType[] {DataTypes.INT(), DataTypes.STRING(), DataTypes.STRING()}, new String[] {"id", "dt", "region"}); // ArrayIndexOutOfBoundsException: 2 new InternalRowPartitionComputer( "__DEFAULT_PARTITION__", rowType, new String[] {"region", "dt"}, false); // In range, so no exception: every name gets the other column's getter. InternalRowPartitionComputer computer = new InternalRowPartitionComputer( "__DEFAULT_PARTITION__", rowType, new String[] {"dt", "id"}, false); // row (id=1, dt="20240731", region="hangzhou") yields {dt=1, id=20240731} ``` ### What doesn't meet your expectations? The constructor takes a `RowType` and a set of partition column names, and nothing in its signature requires the names to be a prefix of the row type in row order. It should either work for any row type that contains the partition columns, or reject the rest explicitly. Instead the array bound and the index expression disagree, so it is correct only for the case where the two happen to coincide. Every current caller passes a row type already projected to the partition columns in `partitionKeys` order (`TableSchema.logicalPartitionType()`, `store().partitionType()`, `rowType().project(partitionKeys())`, `TypeUtils.project(rowType, partitionKeys)`), so `indexOf` always returns the loop position and no deployment hits this today. It fires as soon as a caller hands over the full row type, which the signature invites. ### Anything else? The second form above is the one to worry about. It does not throw: the partition spec comes back with each key carrying another column's value, so partition directory names and partition values are both wrong and nothing reports an error. ### Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
