AlexNilone opened a new issue, #1871: URL: https://github.com/apache/incubator-paimon/issues/1871
### Search before asking - [X] I searched in the [issues](https://github.com/apache/incubator-paimon/issues) and found nothing similar. ### Paimon version paimon-flink-1.17-0.5-20230808.002215-115.jar ### Compute Engine Flink1.17.1 Spark3.3 ### Minimal reproduce step imulate an abnormal interruption of the Flink streaming job, where the process exists but is unable to write to Paimon. ### What doesn't meet your expectations? I've been using the tagging feature, but it's not performing very reliably. Whenever my Flink data lake job is abnormally interrupted (due to issues like insufficient memory or network problems), and I restart the job, I find that the tags are no longer being generated. This issue has occurred twice. The first time, there were some exceptions in Flink, while the second time there were no exceptions at all. SELECT * FROM paimon.abc.`xxxxxxxxxxxx$tags` ; +----------------+--------------+------------+--------------------------+---------------+ | tag_name | snapshot_id | schema_id | commit_time | record_count | +----------------+--------------+------------+--------------------------+---------------+ | 2023-08-15 21 | 12896 | 0 | 2023-08-15 22:05:17.967 | 36636402015 | | 2023-08-15 22 | 13016 | 0 | 2023-08-15 23:05:18.0 | 37539513865 | | 2023-08-15 23 | 13136 | 0 | 2023-08-16 00:05:17.944 | 38553088846 | | 2023-08-16 00 | 13256 | 0 | 2023-08-16 01:05:17.918 | 39481076700 | | 2023-08-16 01 | 13376 | 0 | 2023-08-16 02:05:17.942 | 40624835369 | | 2023-08-16 02 | 13496 | 0 | 2023-08-16 03:05:17.927 | 42175561025 | | 2023-08-16 03 | 13616 | 0 | 2023-08-16 04:05:17.928 | 42820694449 | | 2023-08-16 04 | 13736 | 0 | 2023-08-16 05:05:17.966 | 43368545506 | | 2023-08-16 05 | 13856 | 0 | 2023-08-16 06:05:17.969 | 44062803728 | | 2023-08-16 06 | 13976 | 0 | 2023-08-16 07:05:17.955 | 45004113384 | | 2023-08-16 07 | 14096 | 0 | 2023-08-16 08:05:18.196 | 46222116260 | | 2023-08-16 08 | 14216 | 0 | 2023-08-16 09:05:17.923 | 47385053423 | | 2023-08-16 09 | 14336 | 0 | 2023-08-16 10:05:17.999 | 47997117332 | +----------------+--------------+------------+--------------------------+---------------+ 13 rows selected (0.284 seconds) the first time exception: 2023-08-11 09:01:06 java.lang.RuntimeException: File deletion conflicts detected! Give up committing. Don't panic! Conflicts during commits are normal and this failure is intended to resolve the conflicts. Conflicts are mainly caused by the following scenarios: 1. Your job is suffering from back-pressuring. There are too many snapshots waiting to be committed and an exception occurred during the commit procedure (most probably due to checkpoint timeout). See https://paimon.apache.org/docs/master/maintenance/write-performance/ for how to improve writing performance. 2. Multiple jobs are writing into the same partition at the same time (you'll probably see different base commit user and current commit user below). You can use https://paimon.apache.org/docs/master/maintenance/multiple-writers/#dedicated-compaction-job to support multiple writing. 3. You're recovering from an old savepoint, or you're creating multiple jobs from a savepoint. The job will fail continuously in this scenario to protect metadata from corruption. You can either recover from the latest savepoint, or you can revert the table to the snapshot corresponding to the old savepoint. Base commit user is: 9b39c993-9cd4-412f-bf1f-38e8699d82d2; Current commit user is: a176e41e-4a55-4921-b0d0-19ef769e5b15 Base entries are: {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-5582a43d-3870-4f30-93c6-310446de8481-514.orc, 180051589, 8849408, org.apache.paimon.data.BinaryRow@d2c4b35, org.apache.paimon.data.BinaryRow@63c6233f, org.apache.paimon.stats.BinaryTableStats@88450b14, org.apache.paimon.stats.BinaryTableStats@1a49bb58, 1, 13697219, 0, 5, [], 2023-08-10T10:19:23.366}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-5582a43d-3870-4f30-93c6-310446de8481-515.orc, 80450397, 3949508, org.apache.paimon.data.BinaryRow@e24fa137, org.apache.paimon.data.BinaryRow@5011d334, org.apache.paimon.stats.BinaryTableStats@3f153741, org.apache.paimon.stats.BinaryTableStats@f97a9d21, 0, 13697220, 0, 5, [], 2023-08-10T10:19:30.175}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 1, 5, {data-c987f986-91f8-42a6-ad88-7b7b694dfce2-518.orc, 175454714, 8650752, org.apache.paimon.data.BinaryRow@86fe5854, org.apache.paimon.data.BinaryRow@e843106, org.apache.paimon.stats.BinaryTableStats@82cbf78c, org.apache.paimon.stats.BinaryTableStats@253e663d, 0, 14040581, 0, 5, [], 2023-08-10T10:51:24.413}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 1, 5, {data-c987f986-91f8-42a6-ad88-7b7b694dfce2-519.orc, 90217054, 4449499, org.apache.paimon.data.BinaryRow@80b5fd26, org.apache.paimon.data.BinaryRow@d981dce0, org.apache.paimon.stats.BinaryTableStats@81e57f44, org.apache.paimon.stats.BinaryTableStats@5733e6b1, 1, 14040579, 0, 5, [], 2023-08-10T10:51:31.551}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-c02862d6-29d6-4dfc-95a0-c1175c590b3b-123.orc, 174346111, 8807424, org.apache.paimon.data.BinaryRow@b39f8d55, org.apache.paimon.data.BinaryRow@d9419d91, org.apache.paimon.stats.BinaryTableStats@98e41522, org.apache.paimon.stats.BinaryTableStats@2daedf07, 0, 17019070, 0, 5, [], 2023-08-10T20:51:24.483}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-c02862d6-29d6-4dfc-95a0-c1175c590b3b-124.orc, 135467398, 6841807, org.apache.paimon.data.BinaryRow@70d0aa93, org.apache.paimon.data.BinaryRow@4138488b, org.apache.paimon.stats.BinaryTableStats@65218f26, org.apache.paimon.stats.BinaryTableStats@f4389763, 1, 17019069, 0, 5, [], 2023-08-10T20:51:38.485}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-cb6c2e2a-0091-44a7-a23d-51e96b1eee62-180.orc, 172326946, 8747008, org.apache.paimon.data.BinaryRow@199929e6, org.apache.paimon.data.BinaryRow@f5cd6583, org.apache.paimon.stats.BinaryTableStats@dbd607c1, org.apache.paimon.stats.BinaryTableStats@1c3882f7, 2, 18457671, 0, 5, [], 2023-08-10T23:07:24.700}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-cb6c2e2a-0091-44a7-a23d-51e96b1eee62-181.orc, 159692618, 8108901, org.apache.paimon.data.BinaryRow@e6da0036, org.apache.paimon.data.BinaryRow@c67bd9f9, org.apache.paimon.stats.BinaryTableStats@a15aa45b, org.apache.paimon.stats.BinaryTableStats@fde7ca53, 0, 18457672, 0, 5, [], 2023-08-10T23:07:38.272}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-5a764264-f095-43d9-8485-809dfb72cd76-205.orc, 171848395, 8738816, org.apache.paimon.data.BinaryRow@1e1b0b9b, org.apache.paimon.data.BinaryRow@1f3debce, org.apache.paimon.stats.BinaryTableStats@cc06934b, org.apache.paimon.stats.BinaryTableStats@cd7797a6, 1, 19151876, 0, 5, [], 2023-08-11T00:11:22.453}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-5a764264-f095-43d9-8485-809dfb72cd76-206.orc, 170942852, 8692606, org.apache.paimon.data.BinaryRow@cdac58e4, org.apache.paimon.data.BinaryRow@f522056d, org.apache.paimon.stats.BinaryTableStats@c318cc95, org.apache.paimon.stats.BinaryTableStats@ecb79baa, 0, 19151874, 0, 5, [], 2023-08-11T00:11:37.965}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-c02862d6-29d6-4dfc-95a0-c1175c590b3b-219.orc, 62288354, 2891042, org.apache.paimon.data.BinaryRow@5e67a9b0, org.apache.paimon.data.BinaryRow@5123171c, org.apache.paimon.stats.BinaryTableStats@36643e92, org.apache.paimon.stats.BinaryTableStats@1c9cd677, 17019071, 19952743, 0, 4, [], 2023-08-11T01:27:11.454}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-5a764264-f095-43d9-8485-809dfb72cd76-322.orc, 72600636, 3368832, org.apache.paimon.data.BinaryRow@f2161a1c, org.apache.paimon.data.BinaryRow@14a52808, org.apache.paimon.stats.BinaryTableStats@44f85092, org.apache.paimon.stats.BinaryTableStats@9f2c0e51, 19151877, 22578745, 0, 4, [], 2023-08-11T05:33:17.697}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-c02862d6-29d6-4dfc-95a0-c1175c590b3b-324.orc, 57465732, 2666599, org.apache.paimon.data.BinaryRow@c05a5f43, org.apache.paimon.data.BinaryRow@83794990, org.apache.paimon.stats.BinaryTableStats@feeff771, org.apache.paimon.stats.BinaryTableStats@11bbd5b, 19952744, 22655752, 0, 3, [], 2023-08-11T05:41:11.998}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-cb6c2e2a-0091-44a7-a23d-51e96b1eee62-343.orc, 93614604, 4344620, org.apache.paimon.data.BinaryRow@b0605997, org.apache.paimon.data.BinaryRow@178dfcd0, org.apache.paimon.stats.BinaryTableStats@f3e26185, org.apache.paimon.stats.BinaryTableStats@63c0f08d, 18457673, 22899494, 0, 4, [], 2023-08-11T06:05:16.414}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 1, 5, {data-076dd7b3-ae69-4d99-beeb-65a47c59f986-346.orc, 174368315, 8167424, org.apache.paimon.data.BinaryRow@3932de84, org.apache.paimon.data.BinaryRow@8b6100c7, org.apache.paimon.stats.BinaryTableStats@98b4da1b, org.apache.paimon.stats.BinaryTableStats@89cb8e85, 14040582, 23081415, 0, 4, [], 2023-08-11T06:21:20.511}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 1, 5, {data-076dd7b3-ae69-4d99-beeb-65a47c59f986-347.orc, 10176563, 476535, org.apache.paimon.data.BinaryRow@a2ae5bad, org.apache.paimon.data.BinaryRow@c7a702a7, org.apache.paimon.stats.BinaryTableStats@ddbfeb24, org.apache.paimon.stats.BinaryTableStats@2d3d4456, 14040585, 23081410, 0, 4, [], 2023-08-11T06:21:21.439}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-5a764264-f095-43d9-8485-809dfb72cd76-360.orc, 23641164, 1097223, org.apache.paimon.data.BinaryRow@a7b9ada3, org.apache.paimon.data.BinaryRow@73e096f4, org.apache.paimon.stats.BinaryTableStats@a8378ced, org.apache.paimon.stats.BinaryTableStats@bcc39e21, 22578746, 23681882, 0, 3, [], 2023-08-11T07:17:10.749}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-cb6c2e2a-0091-44a7-a23d-51e96b1eee62-381.orc, 23244694, 1078705, org.apache.paimon.data.BinaryRow@1094b21b, org.apache.paimon.data.BinaryRow@ddcfd72c, org.apache.paimon.stats.BinaryTableStats@1a5c1a2d, org.apache.paimon.stats.BinaryTableStats@d848db1f, 22899495, 23984022, 0, 3, [], 2023-08-11T07:47:11.655}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-c02862d6-29d6-4dfc-95a0-c1175c590b3b-382.orc, 29067300, 1348770, org.apache.paimon.data.BinaryRow@f159a293, org.apache.paimon.data.BinaryRow@30463f3d, org.apache.paimon.stats.BinaryTableStats@d9fd66b4, org.apache.paimon.stats.BinaryTableStats@eb70664e, 22655753, 24013786, 0, 2, [], 2023-08-11T07:49:10.431}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 1, 5, {data-076dd7b3-ae69-4d99-beeb-65a47c59f986-393.orc, 28183902, 1308174, org.apache.paimon.data.BinaryRow@cd0ca49a, org.apache.paimon.data.BinaryRow@a6bb7064, org.apache.paimon.stats.BinaryTableStats@ed28f6b4, org.apache.paimon.stats.BinaryTableStats@3fcf5963, 23081416, 24398362, 0, 3, [], 2023-08-11T08:25:10.683}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-9c9f35e1-85c6-4b57-a16e-c2b9980167b7-419.orc, 178977369, 8482816, org.apache.paimon.data.BinaryRow@8a60c0a5, org.apache.paimon.data.BinaryRow@635270bb, org.apache.paimon.stats.BinaryTableStats@7c2f4688, org.apache.paimon.stats.BinaryTableStats@ea8383c3, 13697221, 24599968, 0, 4, [], 2023-08-11T08:45:21.171}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-9c9f35e1-85c6-4b57-a16e-c2b9980167b7-420.orc, 38963984, 1847112, org.apache.paimon.data.BinaryRow@bf45907f, org.apache.paimon.data.BinaryRow@3f07f1bd, org.apache.paimon.stats.BinaryTableStats@a61a2720, org.apache.paimon.stats.BinaryTableStats@11959bf8, 13697252, 24599960, 0, 4, [], 2023-08-11T08:45:23.995}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-cb6c2e2a-0091-44a7-a23d-51e96b1eee62-408.orc, 14576280, 676332, org.apache.paimon.data.BinaryRow@92d8656, org.apache.paimon.data.BinaryRow@7651ccf6, org.apache.paimon.stats.BinaryTableStats@c7cd8f1e, org.apache.paimon.stats.BinaryTableStats@db6e980c, 23984023, 24662692, 0, 2, [], 2023-08-11T08:51:09.483}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-5a764264-f095-43d9-8485-809dfb72cd76-402.orc, 21860610, 1014167, org.apache.paimon.data.BinaryRow@27c00962, org.apache.paimon.data.BinaryRow@ac7f6e2c, org.apache.paimon.stats.BinaryTableStats@1b5204b4, org.apache.paimon.stats.BinaryTableStats@3b117eec, 23681883, 24701213, 0, 2, [], 2023-08-11T08:53:08.950}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 1, 5, {data-5d54d730-79b9-4efb-8093-e3fdea465ba3-489.orc, 492362, 20879, org.apache.paimon.data.BinaryRow@7f0f2764, org.apache.paimon.data.BinaryRow@d0131a8e, org.apache.paimon.stats.BinaryTableStats@2a338a14, org.apache.paimon.stats.BinaryTableStats@95223b7d, 24738139, 24759019, 0, 0, [], 2023-08-11T08:59:05.346}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-55f6ceba-ed82-4891-95b6-3a13023da2b8-489.orc, 499414, 21114, org.apache.paimon.data.BinaryRow@1d5216e5, org.apache.paimon.data.BinaryRow@7a72fe16, org.apache.paimon.stats.BinaryTableStats@e5152acd, org.apache.paimon.stats.BinaryTableStats@dfd9eb9e, 24736086, 24757201, 0, 0, [], 2023-08-11T08:59:05.509}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-93f4f03f-3fc4-41bb-8372-43ba6501b1e0-489.orc, 504137, 21400, org.apache.paimon.data.BinaryRow@703fd4a6, org.apache.paimon.data.BinaryRow@7d21019, org.apache.paimon.stats.BinaryTableStats@520daaab, org.apache.paimon.stats.BinaryTableStats@8a2957b7, 24743734, 24765134, 0, 0, [], 2023-08-11T08:59:05.637}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-e9caee58-39aa-4839-9120-75c65b124320-489.orc, 498711, 21100, org.apache.paimon.data.BinaryRow@aa783978, org.apache.paimon.data.BinaryRow@a9a19832, org.apache.paimon.stats.BinaryTableStats@77e19e04, org.apache.paimon.stats.BinaryTableStats@8b532fe1, 24726098, 24747197, 0, 0, [], 2023-08-11T08:59:05.893}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 1, 5, {data-076dd7b3-ae69-4d99-beeb-65a47c59f986-405.orc, 7303938, 339181, org.apache.paimon.data.BinaryRow@ccaf8706, org.apache.paimon.data.BinaryRow@b4428970, org.apache.paimon.stats.BinaryTableStats@32f8f694, org.apache.paimon.stats.BinaryTableStats@ca8ec371, 24398363, 24738138, 0, 2, [], 2023-08-11T08:57:06.113}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-c02862d6-29d6-4dfc-95a0-c1175c590b3b-415.orc, 15517915, 719797, org.apache.paimon.data.BinaryRow@eb572f8c, org.apache.paimon.data.BinaryRow@80802704, org.apache.paimon.stats.BinaryTableStats@1ceaa86, org.apache.paimon.stats.BinaryTableStats@9c839572, 24013787, 24736085, 0, 1, [], 2023-08-11T08:57:07.427}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-5a764264-f095-43d9-8485-809dfb72cd76-403.orc, 988705, 42506, org.apache.paimon.data.BinaryRow@c0127f74, org.apache.paimon.data.BinaryRow@c8eb4e78, org.apache.paimon.stats.BinaryTableStats@59ef677a, org.apache.paimon.stats.BinaryTableStats@83624f2c, 24701214, 24743733, 0, 1, [], 2023-08-11T08:57:07.656}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-cb6c2e2a-0091-44a7-a23d-51e96b1eee62-410.orc, 1383313, 63384, org.apache.paimon.data.BinaryRow@c9342386, org.apache.paimon.data.BinaryRow@3a79f52d, org.apache.paimon.stats.BinaryTableStats@617e7ef7, org.apache.paimon.stats.BinaryTableStats@e9fd9ce, 24662693, 24726097, 0, 1, [], 2023-08-11T08:57:07.927}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-bb73fcd6-0d4d-4ea5-b618-95dadc05c972-0.orc, 3195014, 147950, org.apache.paimon.data.BinaryRow@8a60c0a5, org.apache.paimon.data.BinaryRow@e37e895b, org.apache.paimon.stats.BinaryTableStats@18641e8, org.apache.paimon.stats.BinaryTableStats@264203bc, 24599969, 24748028, 0, 3, [], 2023-08-11T09:00:14.188}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-eaa72aea-418b-4d76-b8cd-44bccc78b8d5-490.orc, 503817, 21367, org.apache.paimon.data.BinaryRow@566026b4, org.apache.paimon.data.BinaryRow@e80a9a3c, org.apache.paimon.stats.BinaryTableStats@583a6b76, org.apache.paimon.stats.BinaryTableStats@68c5329e, 24748028, 24769397, 0, 0, [], 2023-08-11T09:01:05.213}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 1, 5, {data-5d54d730-79b9-4efb-8093-e3fdea465ba3-490.orc, 513005, 21747, org.apache.paimon.data.BinaryRow@8ef8aa46, org.apache.paimon.data.BinaryRow@42974672, org.apache.paimon.stats.BinaryTableStats@c3c92d12, org.apache.paimon.stats.BinaryTableStats@70227ad7, 24759020, 24780771, 0, 0, [], 2023-08-11T09:01:05.348}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-55f6ceba-ed82-4891-95b6-3a13023da2b8-490.orc, 502380, 21313, org.apache.paimon.data.BinaryRow@1e796802, org.apache.paimon.data.BinaryRow@5a9e0d42, org.apache.paimon.stats.BinaryTableStats@5ee37efe, org.apache.paimon.stats.BinaryTableStats@d74c349a, 24757202, 24778517, 0, 0, [], 2023-08-11T09:01:05.488}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-93f4f03f-3fc4-41bb-8372-43ba6501b1e0-490.orc, 508241, 21508, org.apache.paimon.data.BinaryRow@6a253a66, org.apache.paimon.data.BinaryRow@23d9e453, org.apache.paimon.stats.BinaryTableStats@cd235371, org.apache.paimon.stats.BinaryTableStats@dde07857, 24765135, 24786647, 0, 0, [], 2023-08-11T09:01:05.638}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-e9caee58-39aa-4839-9120-75c65b124320-490.orc, 508282, 21516, org.apache.paimon.data.BinaryRow@1fc52ccf, org.apache.paimon.data.BinaryRow@c4962275, org.apache.paimon.stats.BinaryTableStats@115bd5b8, org.apache.paimon.stats.BinaryTableStats@a1846cfa, 24747198, 24768716, 0, 0, [], 2023-08-11T09:01:05.770}} Changes are: {DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-9c9f35e1-85c6-4b57-a16e-c2b9980167b7-421.orc, 1397321, 63959, org.apache.paimon.data.BinaryRow@afcf919a, org.apache.paimon.data.BinaryRow@f819d73f, org.apache.paimon.stats.BinaryTableStats@5531839, org.apache.paimon.stats.BinaryTableStats@6fab9030, 24599969, 24663951, 0, 3, [], 2023-08-11T08:51:05.452}} {DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-9c9f35e1-85c6-4b57-a16e-c2b9980167b7-422.orc, 971727, 41848, org.apache.paimon.data.BinaryRow@7c2d676c, org.apache.paimon.data.BinaryRow@27fc23fe, org.apache.paimon.stats.BinaryTableStats@fdfa0cac, org.apache.paimon.stats.BinaryTableStats@f86f450e, 24663952, 24705809, 0, 2, [], 2023-08-11T08:55:05.418}} {DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-eaa72aea-418b-4d76-b8cd-44bccc78b8d5-488.orc, 498797, 21186, org.apache.paimon.data.BinaryRow@4aff31d4, org.apache.paimon.data.BinaryRow@4c4c70fb, org.apache.paimon.stats.BinaryTableStats@c53c2fb7, org.apache.paimon.stats.BinaryTableStats@f5d33dc2, 24705810, 24726995, 0, 0, [], 2023-08-11T08:57:05.212}} {DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-eaa72aea-418b-4d76-b8cd-44bccc78b8d5-489.orc, 497459, 21030, org.apache.paimon.data.BinaryRow@43275b62, org.apache.paimon.data.BinaryRow@e37e895b, org.apache.paimon.stats.BinaryTableStats@a3112165, org.apache.paimon.stats.BinaryTableStats@8d65844, 24726996, 24748027, 0, 0, [], 2023-08-11T08:59:05.210}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 0, 5, {data-9c9f35e1-85c6-4b57-a16e-c2b9980167b7-423.orc, 3194974, 147949, org.apache.paimon.data.BinaryRow@afcf919a, org.apache.paimon.data.BinaryRow@e37e895b, org.apache.paimon.stats.BinaryTableStats@8684a99d, org.apache.paimon.stats.BinaryTableStats@1242ff5d, 24599969, 24748027, 0, 3, [], 2023-08-11T08:59:05.921}} {DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-55f6ceba-ed82-4891-95b6-3a13023da2b8-489.orc, 499414, 21114, org.apache.paimon.data.BinaryRow@1d5216e5, org.apache.paimon.data.BinaryRow@7a72fe16, org.apache.paimon.stats.BinaryTableStats@e5152acd, org.apache.paimon.stats.BinaryTableStats@dfd9eb9e, 24736086, 24757201, 0, 0, [], 2023-08-11T08:59:05.509}} {DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-c02862d6-29d6-4dfc-95a0-c1175c590b3b-415.orc, 15517915, 719797, org.apache.paimon.data.BinaryRow@eb572f8c, org.apache.paimon.data.BinaryRow@80802704, org.apache.paimon.stats.BinaryTableStats@1ceaa86, org.apache.paimon.stats.BinaryTableStats@9c839572, 24013787, 24736085, 0, 1, [], 2023-08-11T08:57:07.427}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 2, 5, {data-c02862d6-29d6-4dfc-95a0-c1175c590b3b-416.orc, 15971002, 740757, org.apache.paimon.data.BinaryRow@eb572f8c, org.apache.paimon.data.BinaryRow@80802704, org.apache.paimon.stats.BinaryTableStats@1ceaa86, org.apache.paimon.stats.BinaryTableStats@9c839572, 24013787, 24757201, 0, 1, [], 2023-08-11T08:59:07.324}} {DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-93f4f03f-3fc4-41bb-8372-43ba6501b1e0-489.orc, 504137, 21400, org.apache.paimon.data.BinaryRow@703fd4a6, org.apache.paimon.data.BinaryRow@7d21019, org.apache.paimon.stats.BinaryTableStats@520daaab, org.apache.paimon.stats.BinaryTableStats@8a2957b7, 24743734, 24765134, 0, 0, [], 2023-08-11T08:59:05.637}} {DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-5a764264-f095-43d9-8485-809dfb72cd76-403.orc, 988705, 42506, org.apache.paimon.data.BinaryRow@c0127f74, org.apache.paimon.data.BinaryRow@c8eb4e78, org.apache.paimon.stats.BinaryTableStats@59ef677a, org.apache.paimon.stats.BinaryTableStats@83624f2c, 24701214, 24743733, 0, 1, [], 2023-08-11T08:57:07.656}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 3, 5, {data-5a764264-f095-43d9-8485-809dfb72cd76-404.orc, 1396832, 63894, org.apache.paimon.data.BinaryRow@c0127f74, org.apache.paimon.data.BinaryRow@c8eb4e78, org.apache.paimon.stats.BinaryTableStats@59ef677a, org.apache.paimon.stats.BinaryTableStats@77d32333, 24701214, 24765134, 0, 1, [], 2023-08-11T08:59:07.643}} {DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-e9caee58-39aa-4839-9120-75c65b124320-489.orc, 498711, 21100, org.apache.paimon.data.BinaryRow@aa783978, org.apache.paimon.data.BinaryRow@a9a19832, org.apache.paimon.stats.BinaryTableStats@77e19e04, org.apache.paimon.stats.BinaryTableStats@8b532fe1, 24726098, 24747197, 0, 0, [], 2023-08-11T08:59:05.893}} {DELETE, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-cb6c2e2a-0091-44a7-a23d-51e96b1eee62-410.orc, 1383313, 63384, org.apache.paimon.data.BinaryRow@c9342386, org.apache.paimon.data.BinaryRow@3a79f52d, org.apache.paimon.stats.BinaryTableStats@617e7ef7, org.apache.paimon.stats.BinaryTableStats@e9fd9ce, 24662693, 24726097, 0, 1, [], 2023-08-11T08:57:07.927}} {ADD, org.apache.paimon.data.BinaryRow@9c67b85d, 4, 5, {data-cb6c2e2a-0091-44a7-a23d-51e96b1eee62-411.orc, 1825478, 84474, org.apache.paimon.data.BinaryRow@aa783978, org.apache.paimon.data.BinaryRow@3a79f52d, org.apache.paimon.stats.BinaryTableStats@214e069, org.apache.paimon.stats.BinaryTableStats@b303e251, 24662693, 24747197, 0, 1, [], 2023-08-11T08:59:07.990}} at org.apache.paimon.operation.FileStoreCommitImpl.createConflictException(FileStoreCommitImpl.java:901) at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:813) at org.apache.paimon.operation.FileStoreCommitImpl.commit(FileStoreCommitImpl.java:253) at org.apache.paimon.table.sink.TableCommitImpl.commitMultiple(TableCommitImpl.java:145) at org.apache.paimon.flink.sink.StoreCommitter.commit(StoreCommitter.java:65) at org.apache.paimon.flink.sink.CommitterOperator.commitUpToCheckpoint(CommitterOperator.java:157) at org.apache.paimon.flink.sink.CommitterOperator.notifyCheckpointComplete(CommitterOperator.java:151) at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.notifyCheckpointComplete(StreamOperatorWrapper.java:104) at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.notifyCheckpointComplete(RegularOperatorChain.java:145) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.notifyCheckpoint(SubtaskCheckpointCoordinatorImpl.java:467) at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.notifyCheckpointComplete(SubtaskCheckpointCoordinatorImpl.java:400) at org.apache.flink.streaming.runtime.tasks.StreamTask.notifyCheckpointComplete(StreamTask.java:1430) at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$notifyCheckpointCompleteAsync$16(StreamTask.java:1371) at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$notifyCheckpointOperation$19(StreamTask.java:1410) at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50) at org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMail(MailboxProcessor.java:398) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMailsWhenDefaultActionUnavailable(MailboxProcessor.java:367) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:352) at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:229) at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:839) at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:788) at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:952) at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:931) at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:745) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.IllegalStateException: Trying to delete file data-9c9f35e1-85c6-4b57-a16e-c2b9980167b7-421.orc which is not previously added. Manifest might be corrupted. at org.apache.paimon.utils.Preconditions.checkState(Preconditions.java:204) at org.apache.paimon.manifest.ManifestEntry.assertNoDelete(ManifestEntry.java:154) at org.apache.paimon.operation.FileStoreCommitImpl.noConflictsOrFail(FileStoreCommitImpl.java:810) ... 25 more ### Anything else? _No response_ ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
