Good day, everyone, For clarity, I organize the following into situation, background, assessment, and proposal.
Best, Damon ----------------------------------------- Situation Issue #26981 reports an IllegalArgumentException associated with the ByteBuddy dependency throwing the message "<some class> must be defined in the same package as <some other class>"[1]. I personally discovered this error blocking my own Schema-related tests. Background *1. PR #25578 introduced the error* As Issue #26981 reports[1], the error seems to be introduced with 2.48. Comparing v2.47.0 and v2.48.0[2] reveals that PR #25578 may have introduced this breaking change[3]. Said PR replaced ByteBuddy's ClassLoadingStrategy.Default.INJECTION[4] with getClassLoadingStrategy[5]. *2. Reverting PR #25578 resolves the error* To test this hypothesis, I cloned 41e6628 and ran: ./gradlew :sdks:java:core:check revealing several failing tests (see *Failing :sdks:java:core:check at 41e6628* below), some of which contained the familiar IllegalArgumentException "<some class> must be defined in the same package as <some other class>" message. After reverting changes found in #25578, the failing tests and the IllegalArgumentException were resolved. *3. Code related to PR #25578 has a back and forth history* There seems to be a back and forth removal and replacement history[6] between ByteBuddy's ClassLoadingStrategy.Default.INJECTION and getClassLoadingStrategy most recently PR #25578. Said PR's motivation is to prepare Beam for Java 17 compatibility, which explains the re-introduction of the breaking changes. *4. PR #25578 GitHub Actions checks pass* Examining the GitHub actions run reveals that PR #25578 checks passed[7]. However, examining the setup[8] more closely reveals that Java tests are executed using Java Version 8. The same is true in the latest 41e6628 commit[9]. To test whether the version of Java drives Issue #26981's error, I submitted a draft PR[10] with the version of Java set to 11 and found that the same errors resulted[11] as I found on my machine using the same Java version. Assessment My main impression is that: 1. checks did not reveal PR #25578's breaking changes[7] because the environment[8] used Java 8 instead of 11 2. the back and forth removal and addition of PR #25578's changes does not solve current and future Java version compatibilities Proposal May we consider: 1. If not already planned, set .github/actions/setup-self-hosted-action/action.yml's Java version[12] to 11. 2. arriving at a consensus regarding PR #25578's breaking changes and what we need to do today and in the future; I don't have anything practical to propose or recommend References 1. https://github.com/apache/beam/issues/26981 2. https://github.com/apache/beam/compare/v2.47.0...v2.48.0 3. https://github.com/apache/beam/pull/25578 4. https://javadoc.io/static/net.bytebuddy/byte-buddy/1.12.23/net/bytebuddy/dynamic/loading/ClassLoadingStrategy.Default.html#INJECTION 5. https://github.com/apache/beam/blob/68e19a596a5d0136ba4592be01888f487463c2f3/sdks/java/core/src/main/java/org/apache/beam/sdk/util/ByteBuddyUtils.java#L32 6. https://github.com/apache/beam/commits/68e19a596a5d0136ba4592be01888f487463c2f3/sdks/java/core/src/main/java/org/apache/beam/sdk/util/ByteBuddyUtils.java 7. https://github.com/apache/beam/actions/runs/4759753495/jobs/8459342173 8. https://github.com/apache/beam/actions/runs/4759753495/jobs/8459342173#step:3:28 9. https://github.com/apache/beam/actions/runs/5558349809/jobs/10153336615#step:3:16 10. https://github.com/apache/beam/actions/runs/5571881403/jobs/10177348958?pr=27515#step:3:16 11. https://github.com/apache/beam/actions/runs/5571881403/jobs/10177348958?pr=27515#step:5:123 12. https://github.com/apache/beam/blob/561870be80cc8696a926d7322b0f47ddeddcc222/.github/actions/setup-self-hosted-action/action.yml#L56 Failing :sdks:java:core:check at 41e6628 *Ran locally on*: openjdk 11.0.14.1 2022-02-08 OpenJDK Runtime Environment JBR-11.0.14.1.1-2043.11-jcef (build 11.0.14.1+1-b2043.11) OpenJDK 64-Bit Server VM JBR-11.0.14.1.1-2043.11-jcef (build 11.0.14.1+1-b2043.11, mixed mode) org.apache.beam.sdk.schemas.AvroSchemaTest > testPojoRecordToRow FAILED java.lang.IllegalArgumentException at AvroSchemaTest.java:451 org.apache.beam.sdk.schemas.AvroSchemaTest > testSpecificRecordToRow FAILED java.lang.IllegalArgumentException at AvroSchemaTest.java:386 org.apache.beam.sdk.schemas.JavaBeanSchemaTest > testMapFieldSetters FAILED java.lang.IllegalArgumentException at JavaBeanSchemaTest.java:460 org.apache.beam.sdk.schemas.JavaBeanSchemaTest > testRecursiveArraySetters FAILED java.lang.IllegalArgumentException at JavaBeanSchemaTest.java:385 org.apache.beam.sdk.schemas.JavaBeanSchemaTest > testMapFieldGetters FAILED java.lang.IllegalArgumentException at JavaBeanSchemaTest.java:437 org.apache.beam.sdk.schemas.JavaBeanSchemaTest > testRecursiveArrayGetters FAILED java.lang.IllegalArgumentException at JavaBeanSchemaTest.java:369 org.apache.beam.sdk.schemas.JavaFieldSchemaTest > testNestedArraysToRow FAILED java.lang.IllegalArgumentException at JavaFieldSchemaTest.java:602 org.apache.beam.sdk.schemas.JavaFieldSchemaTest > testEnumFieldToRow FAILED java.lang.IllegalArgumentException at JavaFieldSchemaTest.java:658 org.apache.beam.sdk.schemas.JavaFieldSchemaTest > testRecursiveArraySetters FAILED java.lang.IllegalArgumentException at JavaFieldSchemaTest.java:411 org.apache.beam.sdk.schemas.JavaFieldSchemaTest > testMapFieldGetters FAILED java.lang.IllegalArgumentException at JavaFieldSchemaTest.java:463 org.apache.beam.sdk.schemas.JavaFieldSchemaTest > testRecursiveArrayGetters FAILED java.lang.IllegalArgumentException at JavaFieldSchemaTest.java:394 org.apache.beam.sdk.schemas.JavaFieldSchemaTest > testNestedArraysFromRow FAILED java.lang.IllegalArgumentException at JavaFieldSchemaTest.java:580 org.apache.beam.sdk.schemas.utils.JsonUtilsTest > testGetRowToJsonBytesFunction FAILED java.lang.IllegalArgumentException at JsonUtilsTest.java:92 org.apache.beam.sdk.schemas.utils.JsonUtilsTest > testGetJsonStringToRowFunction FAILED java.lang.IllegalArgumentException at JsonUtilsTest.java:92 org.apache.beam.sdk.schemas.utils.JsonUtilsTest > testGetRowToJsonStringsFunction FAILED java.lang.IllegalArgumentException at JsonUtilsTest.java:92 org.apache.beam.sdk.schemas.utils.JsonUtilsTest > testGetJsonBytesToRowFunction FAILED java.lang.IllegalArgumentException at JsonUtilsTest.java:92
