LuciferYang opened a new pull request, #10944: URL: https://github.com/apache/paimon/pull/10944
### Purpose close #10942 For a partitioned format table read through the engine implementation, only SQL `INSERT INTO` and `INSERT OVERWRITE` are rewritten to Spark's V1 file write. DataFrame writes and RTAS go through Spark's V2 `FileWrite`, which writes no partition directories, so the rows landed in the table root and were never read back. - `PartitionedFormatTable` now rejects V2 writes to a partitioned table with an `UnsupportedOperationException` that points to `format-table.implementation = 'paimon'`. Non-partitioned engine tables are unchanged. - RTAS into a partitioned engine table is rejected while planning, before the old table is replaced, with the check CTAS already uses. Spark 3.2 and 3.3 plan RTAS with Spark's own strategy, so their `PaimonReplaceTableAsSelectStrategy` now runs the check and still returns `Nil`. ### Tests `FormatTableTestBase` "DataFrame writes to a partitioned engine table are rejected": `writeTo(...).append()`, `writeTo(...).overwrite(...)`, `insertInto` and `insertInto` in overwrite mode are rejected and leave the table unchanged; SQL `INSERT` still works; `CREATE OR REPLACE TABLE ... AS SELECT` and `saveAsTable` in overwrite mode are rejected and keep the old rows. On Spark 3.4+ it also checks that V2 writes and RTAS on a non-partitioned engine table still work. On master the first write succeeds and its row is lost. ### API and Format no ### Documentation no -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
