zhangyue19921010 opened a new issue, #11186:
URL: https://github.com/apache/paimon/issues/11186
### Search before asking
- [x] I searched in the issues and found nothing similar.
### Paimon version
2.2-SNAPSHOT, commit `f1ffd1369ae6db2438a4c9a98fe4216566e88ccc`.
### Compute Engine
Java API, JDK 11.0.18.
### Minimal reproduce step
Create a Data Evolution table with a partition column after a nullable value
column, then write only `id` and `pt`:
```java
catalog.createTable(identifier,
Schema.newBuilder()
.column("id", DataTypes.INT())
.column("value", DataTypes.INT())
.column("pt", DataTypes.STRING())
.partitionKeys("pt")
.option("row-tracking.enabled", "true")
.option("data-evolution.enabled", "true")
.build(),
false);
FileStoreTable table = (FileStoreTable) catalog.getTable(identifier);
BatchWriteBuilder builder = table.newBatchWriteBuilder();
try (BatchTableWrite write = builder.newWrite()
.withWriteType(table.rowType().project("id", "pt"));
BatchTableCommit commit = builder.newCommit()) {
write.write(GenericRow.of(1, BinaryString.fromString("p1")));
commit.commit(write.prepareCommit());
}
```
`write.write` throws `ArrayIndexOutOfBoundsException: Index 2 out of bounds
for length 2` from the partition projection. Reordering all three fields to
`pt,id,value` instead throws `ClassCastException` because it reads the integer
value as the partition string. Writing the original full field order succeeds.
### What doesn't meet your expectations?
The partial write should append the row to partition `p1`, retaining NULL
for the omitted nullable value. `withWriteType` updates the file writer's input
schema, but the routing extractor still uses the original table field positions.
### Anything else?
A reusable projected view can adapt the routing input to the table schema
while leaving the actual row sent to the file writer in its supplied write
type. Generated-column inputs that already reconstruct the complete physical
row must retain their existing routing.
Prepared with TRAE CLI. The reproduction uses the public Java API; Spark SQL
was not exercised.
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]