I do have a silly question, I am struggling to find the reason for the fallback need of Java 11 in the beam_PreCommit_Java_Debezium_IO_Direct.yml [0] I was trying to not interfere with the build system outside of the module by doing some tricks within the build.gradle of Debzium IO in the second PR. [1] However, it seems that the CI always picks up Java 11 and therefore fails the build. It builds completely fine with Java 17 on my machine though. Is there a way to force Java 17 for the module by, for example, removing the 11 fallback from the PreCommit Workflow or does this have known side-effects?
Thank you! [0] https://github.com/apache/beam/pull/34751/files#diff-de752bcab62398ee28adb75a8fdb1933d8d855f2ae46417ec1d47180174a1800R91 [1] https://github.com/apache/beam/pull/34763 On Mon, Apr 28, 2025 at 5:04 PM Tobi Kaymak <kay...@google.com> wrote: > I did the second PR and removed the enforcement of proto3 in the > build.gradle ( > https://github.com/apache/beam/pull/34763/files#diff-1572bcc322ede7d3a757381b84791a29b691df4d8947f6ad39ce4597653787edL95) > - > All tests are green on my local machine, but they will of course fail on > the CI since the first PR has not been merged yet. > > I would be very happy about a review by Danny McCormick, who did work on > this a few months ago in his PR and added the enforcement ( > https://github.com/apache/beam/pull/33526). > > > On Sun, Apr 27, 2025 at 5:00 PM Tobi Kaymak <kay...@google.com> wrote: > >> I did create the first PR, after building the IO on my local machine with >> gradle and Java 17. This should not have an effect other than changing the >> build system. Feedback is very welcome: >> https://github.com/apache/beam/pull/34751 >> >> On Sat, Apr 26, 2025 at 12:49 AM Tobi Kaymak <kay...@google.com> wrote: >> >>> Thank you both! >>> I have checked and couldn't find a matching issue, so I created this >>> one: https://github.com/apache/beam/issues/34747 >>> >>> On Sat, Apr 26, 2025 at 12:32 AM XQ Hu via dev <dev@beam.apache.org> >>> wrote: >>> >>>> Can we create the GitHub issue to track this work if we haven't? >>>> >>>> On Fri, Apr 25, 2025, 4:39 PM Yi Hu via dev <dev@beam.apache.org> >>>> wrote: >>>> >>>>> Hi Tobi, >>>>> >>>>> Thanks for your interest and willing to take on the task. First of >>>>> all, could you please create a GitHub Issue to track the effort? >>>>> >>>>> I would like to share some comments if find useful >>>>> >>>>> - We already have infrastructure for the goal of "PR 1". We use >>>>> BeamPluginModulePlugin [1] to initialize java modules and there is a >>>>> setJavaVerOptions method. Here is a module could be an example how >>>>> compiling with different Java version works in Beam: [2]. >>>>> >>>>> - We'll need to handle Debezium with newer Java version as well >>>>> (That's the reason test currently failing on #34733). We'll need to modify >>>>> test workflow file [3]. However due to GitHub limitation changes to >>>>> .github/workflows won't be effective on unsubmitted PRs. You'd need to >>>>> test >>>>> your change locally. >>>>> >>>>> - Given that current Debezium (1.x) is already old I'm +1 with the >>>>> change. >>>>> >>>>> [1] >>>>> https://github.com/apache/beam/blob/master/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy >>>>> >>>>> [2] >>>>> https://github.com/apache/beam/blob/master/sdks/java/container/agent/build.gradle >>>>> >>>>> [3] >>>>> https://github.com/apache/beam/blob/master/.github/workflows/beam_PreCommit_Java_Debezium_IO_Direct.yml >>>>> >>>>> >>>>> Thanks. >>>>> >>>>> Yi >>>>> >>>>> On Fri, Apr 25, 2025 at 3:06 PM Tobi Kaymak via dev < >>>>> dev@beam.apache.org> wrote: >>>>> >>>>>> Hi Beam Devs, >>>>>> >>>>>> I'm interested in helping to upgrade the Debezium IO connector ( >>>>>> sdks:java:io:debezium) to Debezium version 3.1.1.Final. This version >>>>>> of Debezium requires its dependencies (like debezium-core) to be >>>>>> compiled against Java 17 >>>>>> <https://debezium.io/releases/3.0/release-notes#breaking_changes_14> >>>>>> bytecode. >>>>>> >>>>>> I took the work from Danny McCormick who started doing something >>>>>> similar for Debezium 2.7.4 and created a WIP branch >>>>>> <https://github.com/apache/beam/pull/34733> to get my feet wet with >>>>>> Java and Gradle's build system again. >>>>>> >>>>>> To manage this upgrade gracefully and make the review process >>>>>> smoother, I was thinking to split the work into a two-step PR: >>>>>> >>>>>> 1. >>>>>> >>>>>> *PR 1 (Build Infrastructure):* A smaller, focused PR to enable >>>>>> Java 17 compilation specifically for the sdks:java:io:debezium >>>>>> module. This would likely involve: >>>>>> - Modifying the sdks:java:io:debezium/build.gradle to allow >>>>>> forking its compileJava (and compileTestJava) tasks using a >>>>>> JDK 17 (e.g., via a java17Home Gradle property derived from a >>>>>> CI environment variable like JAVA_HOME_17_X64). >>>>>> - Updating the relevant PreCommit workflow ( >>>>>> beam_PreCommit_Java_Debezium_IO_Direct.yml) >>>>>> >>>>>> <https://github.com/apache/beam/blob/master/.github/workflows/beam_PreCommit_Java_Debezium_IO_Direct.yml#L87> >>>>>> to ensure a JDK 17 is available and its path is passed to Gradle >>>>>> for this >>>>>> module. >>>>>> - The goal of this first PR would be to get the module to >>>>>> compile successfully with Java 17 against the existing (older) >>>>>> Debezium >>>>>> version. >>>>>> 2. >>>>>> >>>>>> *PR 2 (Debezium 3.1.1 Upgrade & API Adaptation):* A subsequent PR >>>>>> that would: >>>>>> - Actually update the Debezium dependencies to 3.1.1.Final. >>>>>> - Incorporate the necessary code changes within >>>>>> sdks:java:io:debezium to adapt to any API changes in Debezium >>>>>> 3.1.1. >>>>>> - Address any new functionality or considerations with >>>>>> Debezium 3.1.1 (which I understand uses now libprotoc-dev 1.5, >>>>>> >>>>>> <https://github.com/debezium/container-images/pull/427/files#diff-bf3b0b8b023cf3cd20ca9d862bb8301523cdb9c934dec1ce96cd9640a613ae7e> >>>>>> - >>>>>> according to its release notes >>>>>> <https://debezium.io/releases/3.1/release-notes#other_changes_5> >>>>>> . >>>>>> >>>>>> My reasoning for this split is to isolate the build system changes >>>>>> from the actual library upgrade and code adaptation, allowing for smaller >>>>>> code reviews. >>>>>> >>>>>> I'd appreciate any feedback or suggestions on the best way to upgrade >>>>>> Debezium IO. >>>>>> >>>>>> Best >>>>>> Tobi >>>>>> >>>>>
