ayers-ltd opened a new pull request, #4617: URL: https://github.com/apache/flink-cdc/pull/4617
## What is the purpose of this pull request? The Doris pipeline connector still bundles Doris Flink Connector 25.1.0. Upgrade it to the released 26.3.0 version to pick up important schema evolution and sink reliability fixes: - **Column type changes with empty string defaults fail.** The old SQL builder drops `DEFAULT ''` (and whitespace-only defaults) from `ALTER TABLE ... MODIFY COLUMN`, which can make Doris reject the change with `Can not change default value`. Fixed by [apache/doris-flink-connector#600](https://github.com/apache/doris-flink-connector/pull/600). - **Batch writes can freeze indefinitely with compression or partial columns.** Asymmetric buffered-byte accounting can leave the writer waiting for cache space even when its buffers are empty. Fixed by [apache/doris-flink-connector#653](https://github.com/apache/doris-flink-connector/pull/653). - **A writer can abort a transaction already handed to the committer.** In two-phase-commit multi-table loading, transaction cleanup now restricts itself to tables still being loaded. Fixed by [apache/doris-flink-connector#683](https://github.com/apache/doris-flink-connector/pull/683). These are fixes provided by the upgraded dependency, not new implementations of those fixes in Flink CDC. See the [official release notes](https://doris.apache.org/releases/ecosystem/doris-flink-connector/). ## Brief change log - Upgrade `flink-doris-connector-1.20` from 25.1.0 to 26.3.0, retaining the existing Flink 1.20 artifact and CDC's Flink 2.x compatibility shim. - Update both `DorisBatchSink#createWriter` overloads to use the upstream `DorisBatchWriterAdapter` introduced by the newer connector. - Add a regression test covering empty, whitespace-only, ordinary and absent column defaults. - Document the dependency's new default gzip compression and `sink.properties.compress_type: ""` opt-out in English and Chinese. ## Verifying this change - Flink 1.20.3 / Java 11: the Doris module and its reactor dependencies build successfully; all 17 Doris unit test cases pass. Spotless apply/check and Checkstyle pass. - The new four-case regression test was also run with the original 25.1.0 dependency: empty and whitespace-only defaults fail, while absent and ordinary defaults pass. All four pass with 26.3.0 on Java 11 and Java 17. - Flink 2.2.0 / Java 17: clean reactor package succeeds, with all 17 Doris unit test cases passing; Spotless and Checkstyle checks pass. - Docker integration and end-to-end tests have not been run. Batch deadlock and transaction fixes are attributed to the upstream changes, rather than claimed as locally reproduced. ## Documentation - Does this pull request introduce a new feature? No new Flink CDC feature; the dependency changes the default Stream Load compression to gzip. - How is the behavior documented? Updated the English and Chinese Doris connector option references with the compression default and opt-out. ##### Was generative AI tooling used to co-author this PR? - [X] Yes: OpenAI Codex Generated-by: OpenAI Codex -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
