I think it is more complicated. I validated the Java SDK with Flink Runner and it passed through definitely more complex tests than a simple Pipeline with DoFn [1].

 Jan

[1] https://github.com/datadrivencz/proxima-platform/pull/624

On 8/17/22 15:19, Alexey Romanenko wrote:


On 16 Aug 2022, at 23:23, Kenneth Knowles <k...@apache.org> wrote:

And as a follow up we should make sure there is some test that would exercise this, since that PR was green and was a while ago too and our postcommits did not catch it either.

I’m concerned that it was not caught with Beam tests - it has to fail with literally every Java Beam pipeline that contains, at least, new DoFn instance creation. Though, it fails only when we compile and run a pipeline outside of Beam codebase.

Does it mean that Beam tests for some reasons still use an old vendored byte-buddy version?

—
Alexey




Kenn

On Tue, Aug 16, 2022 at 12:50 PM Kiley Sok via dev <dev@beam.apache.org> wrote:

    cc: @Liam Miller-Cushon <mailto:cus...@google.com>, who worked on
    the bytebuddy update.

    Liam, do you have any context on this error?

    On Tue, Aug 16, 2022 at 10:11 AM Alexey Romanenko
    <aromanenko....@gmail.com> wrote:

        I tested with "beam-samples" [1] and found that a rather
        simple test pipeline fails [2] with this runtime error:

        Error:
         Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time
        elapsed: 2.776 s <<< FAILURE! - in SerializationTest

        5809
        Error:
         SerializationTest.nonSerilizableTest  Time elapsed: 2.708 s
         <<< ERROR!

        5810
        
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException:
        java.lang.IllegalStateException: Invisible parameter type of
        SerializationTest$1 arg0 for public
        SerializationTest$1$DoFnInvoker(SerializationTest$1)

        5811
        Caused by: java.lang.IllegalStateException: Invisible
        parameter type of SerializationTest$1 arg0 for public
        SerializationTest$1$DoFnInvoker(SerializationTest$1)


        Seems like that it’s caused by “bytebuddy” dependency update
        [3] from version 1.11.0 to 1.12.9 and it was vendored before
        (not sure if it’s related).

        Downgrading the “bytebuddy” version to 1.11.0 fixes an error.

        I’ve not yet gone deep into a cause of this problem but maybe
        someone knows some details?

        [1] https://github.com/Talend/beam-samples/
        [2]
        
https://github.com/Talend/beam-samples/runs/7856722514?check_suite_focus=true

        [3] https://github.com/apache/beam/pull/17317

        —
        Alexey

        On 16 Aug 2022, at 17:54, Ritesh Ghorse via dev
        <dev@beam.apache.org> wrote:

        +1 (non-binding), Validated Go SDK Quickstart on Direct and
        Dataflow runner


        On Tue, Aug 16, 2022 at 4:26 AM Jan Lukavský
        <je...@seznam.cz> wrote:

            +1 (non-binding)

            Validated Java SDK with classical Flink Runner.

            On 8/15/22 23:06, Chamikara Jayalath via dev wrote:
            +1 as well
            (I believe Kiley is addressing the container tags issue)

            Thanks,
            Cham

            On Mon, Aug 15, 2022 at 1:00 PM Robert Bradshaw
            <rober...@google.com> wrote:

                +1 (binding).

                I verified the release artifacts and signatures,
                and ran a couple of
                simple Python pipelines.

                On Mon, Aug 15, 2022 at 12:40 PM Chamikara Jayalath
                via dev
                <dev@beam.apache.org> wrote:
                >
                >
                >
                > On Mon, Aug 15, 2022 at 11:37 AM Kiley Sok
                <kiley...@google.com> wrote:
                >>
                >> Thanks everyone!
                >>
                >> @Chamikara Jayalath The Spark issue is running
                successfully for me, could you try it again? I'll
                look into the container tags.
                >
                >
                > Thanks. Regarding the Spark issue, it could just
                be my setup then. Feel free to close the Github issue.
                >
                > - Cham
                >
                >>
                >>
                >> On Mon, Aug 15, 2022 at 11:04 AM Pablo Estrada
                <pabl...@google.com> wrote:
                >>>
                >>> +1 - I validated tests/build with existing
                Dataflow Templates
                >>> Best
                >>> -P.
                >>>
                >>> On Fri, Aug 12, 2022 at 9:20 PM Ahmet Altay via
                dev <dev@beam.apache.org> wrote:
                >>>>
                >>>> +1 - I validated python quickstarts on direct
                runner.
                >>>>
                >>>> Thank you Kiley!
                >>>>
                >>>>
                >>>>
                >>>> On Thu, Aug 11, 2022 at 9:56 PM Kiley Sok via
                dev <dev@beam.apache.org> wrote:
                >>>>>
                >>>>> Hi everyone,
                >>>>> Please review and vote on the release
                candidate #1 for the version 2.41.0, as follows:
                >>>>> [ ] +1, Approve the release
                >>>>> [ ] -1, Do not approve the release (please
                provide specific comments)
                >>>>>
                >>>>>
                >>>>> Reviewers are encouraged to test their own
                use cases with the release candidate, and vote +1
                if no issues are found.
                >>>>>
                >>>>> The complete staging area is available for
                your review, which includes:
                >>>>> * GitHub Release notes [1],
                >>>>> * the official Apache source release to be
                deployed to dist.apache.org
                <http://dist.apache.org/> [2], which is signed with
                the key with fingerprint
                4D5731CC0AA38097D091EB091E7B28884452AE5D [3],
                >>>>> * all artifacts to be deployed to the Maven
                Central Repository [4],
                >>>>> * source code tag "v2.41.0-RC1" [5],
                >>>>> * website pull request listing the release
                [6], the blog post [6], and publishing the API
                reference manual [7].
                >>>>> * Java artifacts were built with Gradle 7.4
                and OpenJDK/Oracle JDK 1.8.0_232.
                >>>>> * Python artifacts are deployed along with
                the source release to the dist.apache.org
                <http://dist.apache.org/> [2] and PyPI[8].
                >>>>> * Validation sheet with a tab for 2.41.0
                release to help with validation [9].
                >>>>> * Docker images published to Docker Hub [10].
                >>>>>
                >>>>> The vote will be open for at least 72 hours.
                It is adopted by majority approval, with at least 3
                PMC affirmative votes.
                >>>>>
                >>>>> For guidelines on how to try the release in
                your projects, check out our blog post at
                https://beam.apache.org/blog/validate-beam-release/.
                >>>>>
                >>>>> Thanks,
                >>>>> Release Manager
                >>>>>
                >>>>> [1] https://github.com/apache/beam/milestone/3
                >>>>> [2]
                https://dist.apache.org/repos/dist/dev/beam/2.41.0/
                >>>>> [3]
                https://dist.apache.org/repos/dist/release/beam/KEYS
                >>>>> [4]
                
https://repository.apache.org/content/repositories/orgapachebeam-1282/
                >>>>> [5]
                https://github.com/apache/beam/tree/v2.41.0-RC1
                >>>>> [6] https://github.com/apache/beam/pull/22706
                >>>>> [7] https://github.com/apache/beam-site/pull/633
                >>>>> [8]
                https://pypi.org/project/apache-beam/2.41.0rc1/
                >>>>> [9]
                
https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=331459080
                >>>>> [10]
                https://hub.docker.com/search?q=apache%2Fbeam&type=image
                <https://hub.docker.com/search?q=apache%2Fbeam&type=image>


Reply via email to