JingsongLi commented on PR #10077: URL: https://github.com/apache/paimon/pull/10077#issuecomment-5953678654
Reviewed current head `e3ae10af8792d200b5e63865e83de71e96f3dce9`. The key-type guard addresses a real correctness issue and belongs before schema persistence. **[P2] Update the remaining Spark schema-evolution fixtures to the new key contract.** `DataFrameWriteTestBase` still widens primary key `a` from INT to BIGINT and expects successful writes, both in `Schema evolution: write data into Paimon` (Case 2) and its `allowExplicitCast = true` variant. I ran these two existing cases against this exact head: both fail with `Cannot update primary key type from INT NOT NULL to BIGINT NOT NULL: [a]`. This matches the current Spark CI failures; they are directly related to this change. Please keep the key type unchanged in the positive evolution cases and add explicit rejection/data-preservation assertions for key widening. The shared UT `PaimonSinkTest` update now passes locally, but the independent copy in `paimon-spark-3.4/src/test/scala/org/apache/paimon/spark/PaimonSinkTest.scala` still uses `MemoryStream[(Long, Date, Int)]` against an INT primary key. Please update that copy too; it still fails in the Spark3 CI matrix. Validation: 77 Core tests passed on JDK 8 with normal Maven checks. 55 focused tests passed on Spark 3.5.8, including the corrected streaming sink and V1/V2 merge suites. An independent actual path-write probe confirmed that rejected primary-key and partition widening leaves the schema ID, snapshot ID, and committed rows unchanged, while non-key widening subsequently commits and reads correctly. The same 55 tests passed on Spark 4.1.2 with normal Maven checks. I found no additional production-logic regression in the guard after checking case-insensitive matching, retained nullability/names, and both path and catalog schema-persistence routes. Keep this PR open and finish the integration-test updates before merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
