Can you take a look to https://paimon.apache.org/docs/master/maintenance/dedicated-compaction/#dedicated-compaction-job ? As exception message.
You should enable write-only. Best, Jingsong On Fri, Oct 10, 2025 at 11:52 AM yuzhang qin <[email protected]> wrote: > > Hi,I have some questions about compaction and function of partial update. > In my business scenarios,I need to use 8 tasks write to the same table, and I > also need only a few columns keep the first non-null value, other be covered. > Now,I set table as: > WITH ( > 'bucket' = '40', > 'path' = > 's3://yj-datalake-test/paimon/qinyuzhang.db/Mt5HistoryTradeRecordModel', > 'write-mode' = 'WRITE_ONLY', > 'merge-engine' = 'partial-update', > 'bucket-key' = 'b_id,s_id' > ) > and run 8 tasks to write,but it prints logs as: > 2025-10-09 05:33:13,865 WARN org.apache.flink.runtime.taskmanager.Task [] - > Writer : Mt5HistoryTradeRecordModel -> Global Committer : > Mt5HistoryTradeRecordModel -> end: Writer (1/1)#677 > (70647f047568adcfd35f0ed8f64fd97d_f6dc7f4d2283f4605b127b9364e21148_0_677) > switched from RUNNING to FAILED with failure cause: > java.lang.RuntimeException: File deletion conflicts detected! Give up > committing. Don't panic! Conflicts during commits are normal and this failure > is intended to resolve the conflicts. Conflicts are mainly caused by the > following scenarios: 1. Multiple jobs are writing into the same partition at > the same time, or you use STATEMENT SET to execute multiple INSERT statements > into the same Paimon table. You'll probably see different base commit user > and current commit user below. You can use > https://paimon.apache.org/docs/master/maintenance/dedicated-compaction#dedicated-compaction-job > to support multiple writing. 2. You're recovering from an old savepoint, or > you're creating multiple jobs from a savepoint. The job will fail > continuously in this scenario to protect metadata from corruption. You can > either recover from the latest savepoint, or you can revert the table to the > snapshot corresponding to the old savepoint. Base commit user is: > d0d36272-9541-497b-8710-0ad75295d42b; Current commit user is: > d0d36272-9541-497b-8710-0ad75295d42b > So table does not save all data. > How should I set up the table to eliminate the problem and retain all the > data? > Help me thanks a lot. > yuzhang
