taoran92 opened a new pull request, #10481:
URL: https://github.com/apache/paimon/pull/10481
### Purpose
Fixes #10478.
Spark queries selecting `__paimon_bucket` or `__paimon_partition` fail with
a `ClassCastException` when a primary-key read merges records across files.
Both values come from `DataSplit`, but the reader currently casts every
metadata iterator to `FileRecordIterator`.
Require that cast only when reading `__paimon_row_index`. Bucket and
partition queries can then use the merge iterator, while the existing
restrictions on file-path and row-index metadata remain in place.
### Tests
Added a regression test with two separate writes to the same primary key,
compaction disabled, and deletion vectors disabled. It checks the single-file
result, the updated record, bucket-only and partition-only projections, the
combined projection, and an empty result. The existing negative test now covers
both file-path and row-index metadata.
`PaimonQueryTest`: 23 tests passed. Build and formatting checks passed
without the `fast-build` profile:
```shell
mvn -pl paimon-spark/paimon-spark-3.5,paimon-spark/paimon-spark-ut -am
-Pspark3 \
-DwildcardSuites=org.apache.paimon.spark.sql.PaimonQueryTest \
-Dtest=none -DfailIfNoTests=false -Dmaven.jar.forceCreation=true package
```
Also verified the three failing projections with local Spark SQL 3.5.9: each
fails on the previous JAR and returns the expected data with this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
