goutamadwant opened a new issue, #12847: URL: https://github.com/apache/seatunnel/issues/12847
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/seatunnel/issues?q=is%3Aissue+label%3A%22bug%22) and found no similar issues. ### What happened With the Databend sink in CDC mode, rows can be written to the raw table but never merged into the target table. `DatabendCDCSinkIT#testDatabendSinkCDC` fails this way on Flink 1.18 in CI (`Expected 3 records in sink_table ==> expected: <3> but was: <0>`), and Flink 1.15 was disabled for the same symptom in #11624. Two things combine: 1. On Flink 1.15–1.18 the aggregated committer runs in a separate global committer task, which can finish before the last commit arrives. If no checkpoint completes after the rows are written, `DatabendSinkAggregatedCommitter` never runs its `MERGE`. 2. `DatabendSinkWriter.close()` is meant to run a final `MERGE`, but `performMerge()` returns early when `batchCount <= 0`, and `executeBatch()` has already reset `batchCount` to 0. So the writer-side merge never runs on any engine. Examples: [PR #12364 run](https://github.com/goutamadwant/seatunnel/actions/runs/38504052967/job/115658456159), [PR #12776 run](https://github.com/goutamadwant/seatunnel/actions/runs/38665967085/job/116198804582). ### SeaTunnel Version dev ### SeaTunnel Config ```conf env { execution.parallelism = 1 job.mode = "BATCH" checkpoint.interval = 30000 } # FakeSource -> Databend sink with conflict_key set (CDC mode), # as in connector-databend-e2e/src/test/resources/databend/fake_to_databend_cdc.conf ``` ### Running Command ```shell # DatabendCDCSinkIT on the Flink 1.18 container, with checkpoint.interval raised so no checkpoint completes before the batch ends ./mvnw verify -DskipUT=true -DskipIT=false -Pci -pl :connector-databend-e2e -am -Dit.test=DatabendCDCSinkIT ``` ### Error Exception ```log org.awaitility.core.ConditionTimeoutException: ... Expected 3 records in sink_table ==> expected: <3> but was: <0> within 3 minutes at org.apache.seatunnel.e2e.connector.databend.DatabendCDCSinkIT.testDatabendSinkCDC(DatabendCDCSinkIT.java:88) ``` ### Zeta or Flink or Spark Version Flink 1.15–1.18 ### Java or Scala Version _No response_ ### Screenshots _No response_ ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
