Thank you both! I have checked and couldn't find a matching issue, so I created this one: https://github.com/apache/beam/issues/34747
On Sat, Apr 26, 2025 at 12:32 AM XQ Hu via dev <dev@beam.apache.org> wrote: > Can we create the GitHub issue to track this work if we haven't? > > On Fri, Apr 25, 2025, 4:39 PM Yi Hu via dev <dev@beam.apache.org> wrote: > >> Hi Tobi, >> >> Thanks for your interest and willing to take on the task. First of all, >> could you please create a GitHub Issue to track the effort? >> >> I would like to share some comments if find useful >> >> - We already have infrastructure for the goal of "PR 1". We use >> BeamPluginModulePlugin [1] to initialize java modules and there is a >> setJavaVerOptions method. Here is a module could be an example how >> compiling with different Java version works in Beam: [2]. >> >> - We'll need to handle Debezium with newer Java version as well (That's >> the reason test currently failing on #34733). We'll need to modify test >> workflow file [3]. However due to GitHub limitation changes to >> .github/workflows won't be effective on unsubmitted PRs. You'd need to test >> your change locally. >> >> - Given that current Debezium (1.x) is already old I'm +1 with the change. >> >> [1] >> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy >> >> [2] >> https://github.com/apache/beam/blob/master/sdks/java/container/agent/build.gradle >> >> [3] >> https://github.com/apache/beam/blob/master/.github/workflows/beam_PreCommit_Java_Debezium_IO_Direct.yml >> >> >> Thanks. >> >> Yi >> >> On Fri, Apr 25, 2025 at 3:06 PM Tobi Kaymak via dev <dev@beam.apache.org> >> wrote: >> >>> Hi Beam Devs, >>> >>> I'm interested in helping to upgrade the Debezium IO connector ( >>> sdks:java:io:debezium) to Debezium version 3.1.1.Final. This version of >>> Debezium requires its dependencies (like debezium-core) to be compiled >>> against Java 17 >>> <https://debezium.io/releases/3.0/release-notes#breaking_changes_14> >>> bytecode. >>> >>> I took the work from Danny McCormick who started doing something similar >>> for Debezium 2.7.4 and created a WIP branch >>> <https://github.com/apache/beam/pull/34733> to get my feet wet with >>> Java and Gradle's build system again. >>> >>> To manage this upgrade gracefully and make the review process smoother, >>> I was thinking to split the work into a two-step PR: >>> >>> 1. >>> >>> *PR 1 (Build Infrastructure):* A smaller, focused PR to enable Java >>> 17 compilation specifically for the sdks:java:io:debezium module. >>> This would likely involve: >>> - Modifying the sdks:java:io:debezium/build.gradle to allow forking >>> its compileJava (and compileTestJava) tasks using a JDK 17 (e.g., >>> via a java17Home Gradle property derived from a CI environment >>> variable like JAVA_HOME_17_X64). >>> - Updating the relevant PreCommit workflow ( >>> beam_PreCommit_Java_Debezium_IO_Direct.yml) >>> >>> <https://github.com/apache/beam/blob/master/.github/workflows/beam_PreCommit_Java_Debezium_IO_Direct.yml#L87> >>> to ensure a JDK 17 is available and its path is passed to Gradle for >>> this >>> module. >>> - The goal of this first PR would be to get the module to compile >>> successfully with Java 17 against the existing (older) Debezium >>> version. >>> 2. >>> >>> *PR 2 (Debezium 3.1.1 Upgrade & API Adaptation):* A subsequent PR >>> that would: >>> - Actually update the Debezium dependencies to 3.1.1.Final. >>> - Incorporate the necessary code changes within >>> sdks:java:io:debezium to adapt to any API changes in Debezium >>> 3.1.1. >>> - Address any new functionality or considerations with Debezium >>> 3.1.1 (which I understand uses now libprotoc-dev 1.5, >>> >>> <https://github.com/debezium/container-images/pull/427/files#diff-bf3b0b8b023cf3cd20ca9d862bb8301523cdb9c934dec1ce96cd9640a613ae7e> >>> - >>> according to its release notes >>> <https://debezium.io/releases/3.1/release-notes#other_changes_5>. >>> >>> My reasoning for this split is to isolate the build system changes from >>> the actual library upgrade and code adaptation, allowing for smaller code >>> reviews. >>> >>> I'd appreciate any feedback or suggestions on the best way to upgrade >>> Debezium IO. >>> >>> Best >>> Tobi >>> >>
