plusplusjiajia opened a new pull request, #10948: URL: https://github.com/apache/paimon/pull/10948
### Purpose A Format Table has no partition mark done today: `PartitionMarkDoneAction.createActions` and the `mark_partition_done` procedures only accept a `FileStoreTable`. Users moving Hive-style tables to Format Tables often have downstream jobs that wait for a `_SUCCESS` file in the partition directory. This PR supports the `success-file` action for Format Tables: - `FormatTableCommit#committedPartitions()` returns the partitions the last successful commit wrote, including the full static partition of an `INSERT OVERWRITE` that wrote no rows. - `FormatTableSuccessFileMarkDoneAction` writes `_SUCCESS` in the partition directory, following `format-table.partition-path-only-value`. With catalog-managed partitions, a partition registered at a custom location, or not registered, is skipped with a warning, so the file is only written inside the table directory. - `PartitionMarkDoneAction#createActions` gets a `FormatTable` overload. Only `success-file` is supported; other actions fail with a clear message. - Spark: with `partition.end-input-to-done = true`, a Format Table write marks the partitions it committed. Once the commit succeeds, a later abort (for example after marking fails) no longer deletes the committed data. - Spark and Flink `mark_partition_done` (procedure and Flink action) support Format Tables. Flink writes to a Format Table do not mark partitions done in this PR: each sink subtask commits on its own in `close()`, so there is no point where every subtask has finished. That needs a global committer and is left for a follow-up. ### Tests - `FormatTableSuccessFileMarkDoneActionTest`: key-value and value-only directories, escaped values, re-marking keeps the creation time, catalog-managed partitions at the default location / a custom location / not registered, unsupported actions, unpartitioned tables. - `FormatTableCommitCommittedPartitionsTest`: append, empty static overwrite, prefix overwrite, failed commit, unpartitioned table. - `FormatTablePartitionMarkDoneTest` (Spark): writes with and without catalog-managed partitions, empty static overwrite, value-only layout, disabled `end-input-to-done`, the procedure, and an unsupported action. - `FormatTableITCase#testMarkPartitionDoneProcedure` (Flink). - Existing `FormatTableCommit*Test`, `FormatTableScanTest`, `MarkPartitionDoneProcedureTest`, `PaimonFormatTableTest`, `MarkPartitionDoneActionITCase` pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
