I did create the first PR, after building the IO on my local machine with gradle and Java 17. This should not have an effect other than changing the build system. Feedback is very welcome: https://github.com/apache/beam/pull/34751
On Sat, Apr 26, 2025 at 12:49 AM Tobi Kaymak <kay...@google.com> wrote: > Thank you both! > I have checked and couldn't find a matching issue, so I created this one: > https://github.com/apache/beam/issues/34747 > > On Sat, Apr 26, 2025 at 12:32 AM XQ Hu via dev <dev@beam.apache.org> > wrote: > >> Can we create the GitHub issue to track this work if we haven't? >> >> On Fri, Apr 25, 2025, 4:39 PM Yi Hu via dev <dev@beam.apache.org> wrote: >> >>> Hi Tobi, >>> >>> Thanks for your interest and willing to take on the task. First of all, >>> could you please create a GitHub Issue to track the effort? >>> >>> I would like to share some comments if find useful >>> >>> - We already have infrastructure for the goal of "PR 1". We use >>> BeamPluginModulePlugin [1] to initialize java modules and there is a >>> setJavaVerOptions method. Here is a module could be an example how >>> compiling with different Java version works in Beam: [2]. >>> >>> - We'll need to handle Debezium with newer Java version as well (That's >>> the reason test currently failing on #34733). We'll need to modify test >>> workflow file [3]. However due to GitHub limitation changes to >>> .github/workflows won't be effective on unsubmitted PRs. You'd need to test >>> your change locally. >>> >>> - Given that current Debezium (1.x) is already old I'm +1 with the >>> change. >>> >>> [1] >>> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy >>> >>> [2] >>> https://github.com/apache/beam/blob/master/sdks/java/container/agent/build.gradle >>> >>> [3] >>> https://github.com/apache/beam/blob/master/.github/workflows/beam_PreCommit_Java_Debezium_IO_Direct.yml >>> >>> >>> Thanks. >>> >>> Yi >>> >>> On Fri, Apr 25, 2025 at 3:06 PM Tobi Kaymak via dev <dev@beam.apache.org> >>> wrote: >>> >>>> Hi Beam Devs, >>>> >>>> I'm interested in helping to upgrade the Debezium IO connector ( >>>> sdks:java:io:debezium) to Debezium version 3.1.1.Final. This version >>>> of Debezium requires its dependencies (like debezium-core) to be >>>> compiled against Java 17 >>>> <https://debezium.io/releases/3.0/release-notes#breaking_changes_14> >>>> bytecode. >>>> >>>> I took the work from Danny McCormick who started doing something >>>> similar for Debezium 2.7.4 and created a WIP branch >>>> <https://github.com/apache/beam/pull/34733> to get my feet wet with >>>> Java and Gradle's build system again. >>>> >>>> To manage this upgrade gracefully and make the review process smoother, >>>> I was thinking to split the work into a two-step PR: >>>> >>>> 1. >>>> >>>> *PR 1 (Build Infrastructure):* A smaller, focused PR to enable Java >>>> 17 compilation specifically for the sdks:java:io:debezium module. >>>> This would likely involve: >>>> - Modifying the sdks:java:io:debezium/build.gradle to allow forking >>>> its compileJava (and compileTestJava) tasks using a JDK 17 >>>> (e.g., via a java17Home Gradle property derived from a CI >>>> environment variable like JAVA_HOME_17_X64). >>>> - Updating the relevant PreCommit workflow ( >>>> beam_PreCommit_Java_Debezium_IO_Direct.yml) >>>> >>>> <https://github.com/apache/beam/blob/master/.github/workflows/beam_PreCommit_Java_Debezium_IO_Direct.yml#L87> >>>> to ensure a JDK 17 is available and its path is passed to Gradle for >>>> this >>>> module. >>>> - The goal of this first PR would be to get the module to >>>> compile successfully with Java 17 against the existing (older) >>>> Debezium >>>> version. >>>> 2. >>>> >>>> *PR 2 (Debezium 3.1.1 Upgrade & API Adaptation):* A subsequent PR >>>> that would: >>>> - Actually update the Debezium dependencies to 3.1.1.Final. >>>> - Incorporate the necessary code changes within >>>> sdks:java:io:debezium to adapt to any API changes in Debezium >>>> 3.1.1. >>>> - Address any new functionality or considerations with Debezium >>>> 3.1.1 (which I understand uses now libprotoc-dev 1.5, >>>> >>>> <https://github.com/debezium/container-images/pull/427/files#diff-bf3b0b8b023cf3cd20ca9d862bb8301523cdb9c934dec1ce96cd9640a613ae7e> >>>> - >>>> according to its release notes >>>> <https://debezium.io/releases/3.1/release-notes#other_changes_5>. >>>> >>>> My reasoning for this split is to isolate the build system changes from >>>> the actual library upgrade and code adaptation, allowing for smaller code >>>> reviews. >>>> >>>> I'd appreciate any feedback or suggestions on the best way to upgrade >>>> Debezium IO. >>>> >>>> Best >>>> Tobi >>>> >>>
