I did the second PR and removed the enforcement of proto3 in the build.gradle ( https://github.com/apache/beam/pull/34763/files#diff-1572bcc322ede7d3a757381b84791a29b691df4d8947f6ad39ce4597653787edL95) - All tests are green on my local machine, but they will of course fail on the CI since the first PR has not been merged yet.
I would be very happy about a review by Danny McCormick, who did work on this a few months ago in his PR and added the enforcement ( https://github.com/apache/beam/pull/33526). On Sun, Apr 27, 2025 at 5:00 PM Tobi Kaymak <kay...@google.com> wrote: > I did create the first PR, after building the IO on my local machine with > gradle and Java 17. This should not have an effect other than changing the > build system. Feedback is very welcome: > https://github.com/apache/beam/pull/34751 > > On Sat, Apr 26, 2025 at 12:49 AM Tobi Kaymak <kay...@google.com> wrote: > >> Thank you both! >> I have checked and couldn't find a matching issue, so I created this one: >> https://github.com/apache/beam/issues/34747 >> >> On Sat, Apr 26, 2025 at 12:32 AM XQ Hu via dev <dev@beam.apache.org> >> wrote: >> >>> Can we create the GitHub issue to track this work if we haven't? >>> >>> On Fri, Apr 25, 2025, 4:39 PM Yi Hu via dev <dev@beam.apache.org> wrote: >>> >>>> Hi Tobi, >>>> >>>> Thanks for your interest and willing to take on the task. First of all, >>>> could you please create a GitHub Issue to track the effort? >>>> >>>> I would like to share some comments if find useful >>>> >>>> - We already have infrastructure for the goal of "PR 1". We use >>>> BeamPluginModulePlugin [1] to initialize java modules and there is a >>>> setJavaVerOptions method. Here is a module could be an example how >>>> compiling with different Java version works in Beam: [2]. >>>> >>>> - We'll need to handle Debezium with newer Java version as well (That's >>>> the reason test currently failing on #34733). We'll need to modify test >>>> workflow file [3]. However due to GitHub limitation changes to >>>> .github/workflows won't be effective on unsubmitted PRs. You'd need to test >>>> your change locally. >>>> >>>> - Given that current Debezium (1.x) is already old I'm +1 with the >>>> change. >>>> >>>> [1] >>>> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy >>>> >>>> [2] >>>> https://github.com/apache/beam/blob/master/sdks/java/container/agent/build.gradle >>>> >>>> [3] >>>> https://github.com/apache/beam/blob/master/.github/workflows/beam_PreCommit_Java_Debezium_IO_Direct.yml >>>> >>>> >>>> Thanks. >>>> >>>> Yi >>>> >>>> On Fri, Apr 25, 2025 at 3:06 PM Tobi Kaymak via dev < >>>> dev@beam.apache.org> wrote: >>>> >>>>> Hi Beam Devs, >>>>> >>>>> I'm interested in helping to upgrade the Debezium IO connector ( >>>>> sdks:java:io:debezium) to Debezium version 3.1.1.Final. This version >>>>> of Debezium requires its dependencies (like debezium-core) to be >>>>> compiled against Java 17 >>>>> <https://debezium.io/releases/3.0/release-notes#breaking_changes_14> >>>>> bytecode. >>>>> >>>>> I took the work from Danny McCormick who started doing something >>>>> similar for Debezium 2.7.4 and created a WIP branch >>>>> <https://github.com/apache/beam/pull/34733> to get my feet wet with >>>>> Java and Gradle's build system again. >>>>> >>>>> To manage this upgrade gracefully and make the review process >>>>> smoother, I was thinking to split the work into a two-step PR: >>>>> >>>>> 1. >>>>> >>>>> *PR 1 (Build Infrastructure):* A smaller, focused PR to enable >>>>> Java 17 compilation specifically for the sdks:java:io:debezium >>>>> module. This would likely involve: >>>>> - Modifying the sdks:java:io:debezium/build.gradle to allow >>>>> forking its compileJava (and compileTestJava) tasks using a JDK >>>>> 17 (e.g., via a java17Home Gradle property derived from a CI >>>>> environment variable like JAVA_HOME_17_X64). >>>>> - Updating the relevant PreCommit workflow ( >>>>> beam_PreCommit_Java_Debezium_IO_Direct.yml) >>>>> >>>>> <https://github.com/apache/beam/blob/master/.github/workflows/beam_PreCommit_Java_Debezium_IO_Direct.yml#L87> >>>>> to ensure a JDK 17 is available and its path is passed to Gradle >>>>> for this >>>>> module. >>>>> - The goal of this first PR would be to get the module to >>>>> compile successfully with Java 17 against the existing (older) >>>>> Debezium >>>>> version. >>>>> 2. >>>>> >>>>> *PR 2 (Debezium 3.1.1 Upgrade & API Adaptation):* A subsequent PR >>>>> that would: >>>>> - Actually update the Debezium dependencies to 3.1.1.Final. >>>>> - Incorporate the necessary code changes within >>>>> sdks:java:io:debezium to adapt to any API changes in Debezium >>>>> 3.1.1. >>>>> - Address any new functionality or considerations with Debezium >>>>> 3.1.1 (which I understand uses now libprotoc-dev 1.5, >>>>> >>>>> <https://github.com/debezium/container-images/pull/427/files#diff-bf3b0b8b023cf3cd20ca9d862bb8301523cdb9c934dec1ce96cd9640a613ae7e> >>>>> - >>>>> according to its release notes >>>>> <https://debezium.io/releases/3.1/release-notes#other_changes_5> >>>>> . >>>>> >>>>> My reasoning for this split is to isolate the build system changes >>>>> from the actual library upgrade and code adaptation, allowing for smaller >>>>> code reviews. >>>>> >>>>> I'd appreciate any feedback or suggestions on the best way to upgrade >>>>> Debezium IO. >>>>> >>>>> Best >>>>> Tobi >>>>> >>>>
