-1 We should rebuild the artifacts with a Java 8 compiler as Robert suggested.
We have a set of 'forward compatibility' tests for new releases of Beam at Talend. When i updated the tests to use the 2.24.0 RC1 artifacts I got this exception: 2020-09-01T08:19:46.7681646Z [0m [91mSep 01, 2020 8:19:46 AM org.apache.beam.runners.direct.DirectTransformExecutor run 2020-09-01T08:19:46.7682307Z SEVERE: Error occurred within org.apache.beam.runners.direct.DirectTransformExecutor@2d667b5e 2020-09-01T08:19:46.7682612Z java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer; 2020-09-01T08:19:46.7682915Z at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.outputBuffer(BufferedElementCountingOutputStream.java:197) 2020-09-01T08:19:46.7683223Z at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.flush(BufferedElementCountingOutputStream.java:180) 2020-09-01T08:19:46.7683520Z at org.talend.sdk.component.runtime.beam.coder.registry.SchemaRegistryCoder.encode(SchemaRegistryCoder.java:62) 2020-09-01T08:19:46.7683801Z at org.talend.sdk.component.runtime.beam.coder.registry.SchemaRegistryCoder.encode(SchemaRegistryCoder.java:41) 2020-09-01T08:19:46.7684099Z at org.apache.beam.sdk.coders.Coder.encode(Coder.java:136) 2020-09-01T08:19:46.7684695Z at org.apache.beam.sdk.transforms.join.UnionCoder.encode(UnionCoder.java:71) 2020-09-01T08:19:46.7700729Z at org.apache.beam.sdk.transforms.joi [0m [91mn.UnionCoder.encode(UnionCoder.java:58) 2020-09-01T08:19:46.7701365Z at org.apache.beam.sdk.transforms.join.UnionCoder.encode(UnionCoder.jav [0m [91ma:32) 2020-09-01T08:19:46.7701874Z at org.apache.beam.sdk.coders.IterableLikeCoder.en [0m [91mcode(IterableLikeCoder.java:125) 2020-09-01T08:19:46.7702401Z at org.apache.beam.sdk.coders.Iterabl [0m [91me [0m [91mLikeCoder.encode(IterableLikeCoder.java:60) 2020-09-01T08:19:46.7702867Z at [0m [91m org.apache.beam.sdk.coders.Coder.encode(Coder.java:136) 2020-09-01T08:19:46.7703362Z at org.apache.bea [0m [91mm. [0m [91msdk.coders.KvCoder.encode(KvCoder.java:73) [0m [91m 2020-09-01T08:19:46.7703927Z at org.apache [0m [91m.beam.sdk.coders.KvCoder.enc [0m [91mode(KvCoder.java:37) 2020-09-01T08:19:46.7704447Z at org.apache [0m [91m.beam.sdk.util.CoderUtils.encodeToSafeStream(CoderUtils.java:82) [0m [91m 2020-09-01T08:19:46.7705126Z at org.apache.beam.sdk.util.CoderUt [0m [91mils.encodeToBy [0m [91mteArray(CoderUtils.java:66) 2020-09-01T08:19:46.7705616Z at org.apache.beam.sdk.util.CoderUtils.encodeToB [0m [91my [0m [91mteArray(CoderUtils.java:51) 2020-09-01T08:19:46.7706133Z at org.apache.b [0m [91meam.sdk.util.C [0m [91moderUtils.clone(CoderUtils.java:141) 2020-09-01T08:19:46.7725057Z at org.apache.beam.ru [0m [91mnn [0m [91mers.direct.CloningBun [0m [91mdleFactory$CloningBundle.add(CloningBundleFactory.java:84) 2020-09-01T08:19:46.7725890Z at org.apache.beam.runners.direct.GroupAlsoBy [0m [91mWindowEvaluatorFactory$OutputWindowedVal [0m [91mueToBundle.output [0m [91mWindowedValue(GroupAlsoByWindowEvaluatorFactory.java:252) 2020-09-01T08:19:46.7726579Z at org.apache.bea [0m [91mm. [0m [91mrunners.direct.GroupAlsoByWindowEvaluatorFac [0m [91mtory$OutputWind [0m [91mowedValueToBundle.outputWindowedValue(GroupAlsoByWind [0m [91mowEvaluatorFactory.java:238) 2020-09-01T08:19:46.7727231Z [0m [91m at org.apache. [0m [91mbeam.repackaged.direct_java.runners.core.ReduceFnRunner.lambda$onTrigger$1(Re [0m [91mdu [0m [91mceFnRunner.java:1056) 2020-09-01T08:19:46.7727989Z at org.apache.beam.re [0m [91mpackaged.direct [0m [91m_java.runners.core.ReduceFnCont [0m [91mextFactory$OnTriggerCont [0m [91mextImpl.output(ReduceFnContextF [0m [91mactory.java:442) 2020-09-01T08:19:46.7728626Z at org.apac [0m [91mhe.beam.repackaged.direct_java.runners.core.SystemReduceFn.onTrigge [0m [91mr(SystemReduceFn.java:127) 2020-09-01T08:19:46.7729253Z at org. [0m [91mapache.beam.repa [0m [91mckaged.direct_java.runners.core.ReduceFnRunner.onTrigger(ReduceFnRunner.java:1059 [0m [91m) 2020-09-01T08:19:46.7729907Z at org.apache.beam.repackag [0m [91med.direct_java. [0m [91mrunners.core.R [0m [91meduceFnRunner.onTimers(ReduceFnR [0m [91munner.java:768) 2020-09-01T08:19:46.7758126Z at or [0m [91mg. [0m [91mapache.beam.runner [0m [91ms.direct.GroupAlsoByWindowEvaluatorFactory$GroupAlsoByWindowEvaluator.processElement(GroupAlsoByWin [0m [91mdo [0m [91mwEvaluatorFactory.java:185) 2020-09-01T08:19:46.7759005Z at org.apache.beam.runners.direct.Direct [0m [91mTransformExecutor.processElements(DirectTransformExecutor.ja [0m [91mv [0m [91ma:160) 2020-09-01T08:19:46.7759577Z at org.apache.beam.runners.direct.DirectT [0m [91mransformExecutor.ru [0m [91mn(DirectTransformExecutor.java:124) 2020-09-01T08:19:46.7760252Z at java.util.concurrent.Executors$Ru [0m [91mnnableAdapter.call(Executors.java:511 [0m [91m) 2020-09-01T08:19:46.7760843Z at java.util.concurrent.Futu [0m [91mreTask.run(FutureTask.java:266) 2020-09-01T08:19:46.7761345Z at java.util.concurrent.Threa [0m [91mdP [0m [91moolExecutor.runWorker(ThreadPoolExecutor.java:1149) 2020-09-01T08:19:46.7761910Z at java.util [0m [91m.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java [0m [91m:624) 2020-09-01T08:19:46.7762418Z at java.lang.Thread.run(Threa [0m [91md.java:748) I checked the generated classes and they are v52 (Java 8 compatible) but something seems to be wrong. Both the latest 2.24.0-SNAPSHOT version and the current 2.25.0-SNAPSHOT version pass the tests without issues. The issue seems to be only with the RC1 artifacts. On Tue, Sep 1, 2020 at 8:17 PM Robert Bradshaw <[email protected]> wrote: > > On Tue, Sep 1, 2020 at 10:41 AM Daniel Oliveira <[email protected]> > wrote: > > > > I should probably call out that Dataflow containers aren't built yet (I > > will be building them today), so testing of Dataflow should probably wait > > until tomorrow. > > > > > If Java 11 was used to build the release artifacts, does this create any > > > backwards-compatibility challenges for Java 8 users? > > > > It's definitely possible, but I only realized this at the end of the > > process, so I elected to just finish the release candidate rather than > > restarting the whole process, so we can find out in testing. > > I am concerned testing alone may not cover the issues in corner cases > that this may cause (and users would hit). I'd prefer we just re-build > it with Java 8 to be safe. (That doesn't preclude validating the > artifacts in other ways in the meantime.) > > > On Tue, Sep 1, 2020 at 9:14 AM Valentyn Tymofieiev <[email protected]> > > wrote: > >> > >> > * Java artifacts were built with Maven 3.6.3 and OpenJDK 11.0.7. > >> > >> If Java 11 was used to build the release artifacts, does this create any > >> backwards-compatibility challenges for Java 8 users? > >> > >> On Mon, Aug 31, 2020 at 8:59 PM Daniel Oliveira <[email protected]> > >> wrote: > >>> > >>> Hi everyone, > >>> Please review and vote on the release candidate #1 for the version > >>> 2.24.0, as follows: > >>> [ ] +1, Approve the release > >>> [ ] -1, Do not approve the release (please provide specific comments) > >>> > >>> > >>> The complete staging area is available for your review, which includes: > >>> * JIRA release notes [1], > >>> * the official Apache source release to be deployed to dist.apache.org > >>> [2], which is signed with the key with fingerprint > >>> D0E7B69D911ADA3C0482BAA1C4E6B2F8C71D742F [3], > >>> * all artifacts to be deployed to the Maven Central Repository [4], > >>> * source code tag "v2.24.0-RC1" [5], > >>> * website pull request listing the release [6], publishing the API > >>> reference manual [7], and the blog post [8]. > >>> * Java artifacts were built with Maven 3.6.3 and OpenJDK 11.0.7. > >>> * Python artifacts are deployed along with the source release to the > >>> dist.apache.org [2]. > >>> * Validation sheet with a tab for 2.24.0 release to help with validation > >>> [9]. > >>> * Docker images published to Docker Hub [10]. > >>> > >>> The vote will be open for at least 72 hours. It is adopted by majority > >>> approval, with at least 3 PMC affirmative votes. > >>> > >>> Thanks, > >>> Release Manager > >>> > >>> [1] > >>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12347146 > >>> [2] https://dist.apache.org/repos/dist/dev/beam/2.24.0/ > >>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS > >>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1108/ > >>> [5] https://github.com/apache/beam/tree/v2.24.0-RC1 > >>> [6] https://github.com/apache/beam/pull/12743 > >>> [7] https://github.com/apache/beam-site/pull/607 > >>> [8] https://github.com/apache/beam/pull/12745 > >>> [9] > >>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1432428331 > >>> [10] https://hub.docker.com/search?q=apache%2Fbeam&type=image
