I'm seeing a potential regression when running Python x-lang Kafka jobs on Datafllow.
https://pantheon.corp.google.com/dataflow/jobs/us-central1/2023-05-30_16_31_32-1219154560944228293;step=;mainTab=JOB_GRAPH;bottomTab=JOB_LOGS;logsSeverity=INFO;graphView=0?project=google.com:clouddfe&pageState=(%22dfTime%22:(%22l%22:%22dfJobMaxTime%22)) "Topic kafka_taxirides_realtime not present in metadata after 60000 ms" Currently not sure if this is due to my Kafka cluster setup or not. Thanks, Cham On Tue, May 30, 2023 at 5:52 PM Robert Bradshaw via dev <dev@beam.apache.org> wrote: > +1 (binding) > > On Tue, May 30, 2023 at 5:42 PM Robert Bradshaw <rober...@google.com> > wrote: > >> On Tue, May 30, 2023 at 2:01 PM Ritesh Ghorse via dev < >> dev@beam.apache.org> wrote: >> >>> Thanks Danny and Jack! Dataflow containers are up! >>> >>> Only PMC votes count but feel free to test your use cases and vote on >>> this thread! >>> >> >> While we need at least 3 affirmative PMC votes to formally do a release, >> it is definitely the case that all votes are valuable input and are taken >> into consideration when deciding to do so. >> >> >>> On Tue, May 30, 2023 at 11:26 AM Alexey Romanenko < >>> aromanenko....@gmail.com> wrote: >>> >>>> +1 (binding) >>>> >>>> Tested with https://github.com/Talend/beam-samples/ >>>> (Java SDK v8/v11/v17, Spark 3.x runner). >>>> >>>> On 27 May 2023, at 19:38, Bruno Volpato via dev <dev@beam.apache.org> >>>> wrote: >>>> >>>> I was able to check that containers are all there and complete >>>> my validation. >>>> >>>> +1 (non-binding). >>>> >>>> Tested with https://github.com/GoogleCloudPlatform/DataflowTemplates (Java >>>> SDK 11, Dataflow runner). >>>> >>>> >>>> Thanks Ritesh and Danny! >>>> >>>> On Fri, May 26, 2023 at 10:09 AM Danny McCormick via dev < >>>> dev@beam.apache.org> wrote: >>>> >>>>> It looks like some Dataflow containers didn't get published, so some >>>>> jobs using the legacy runner (runner v2 disabled) will fail. I kicked off >>>>> the container release, so that should hopefully be available later today. >>>>> >>>>> Thanks, >>>>> Danny >>>>> >>>>> On Thu, May 25, 2023 at 11:19 PM Ritesh Ghorse via dev < >>>>> dev@beam.apache.org> wrote: >>>>> >>>>>> Hi everyone, >>>>>> Please review and vote on the release candidate #2 for the version >>>>>> 2.48.0, as follows: >>>>>> [ ] +1, Approve the release >>>>>> [ ] -1, Do not approve the release (please provide specific comments) >>>>>> >>>>>> >>>>>> Reviewers are encouraged to test their own use cases with the release >>>>>> candidate, and vote +1 if no issues are found. Only PMC member votes will >>>>>> count towards the final vote, but votes from all community members are >>>>>> encouraged and helpful for finding regressions; you can either test your >>>>>> own use cases or use cases from the validation sheet [10]. >>>>>> >>>>>> The complete staging area is available for your review, which >>>>>> includes: >>>>>> * GitHub Release notes [1], >>>>>> * the official Apache source release to be deployed to >>>>>> dist.apache.org [2], which is signed with the key with fingerprint >>>>>> E4C74BEC861570F5A3E44E46280A0AC32DBAE62B [3], >>>>>> * all artifacts to be deployed to the Maven Central Repository [4], >>>>>> * source code tag "v2.48.0-RC2" [5], >>>>>> * website pull request listing the release [6], the blog post [6], >>>>>> and publishing the API reference manual [7] (to be generated). >>>>>> * Java artifacts were built with Gradle 7.5.1 and OpenJDK/Oracle JDK >>>>>> 8.0.322. >>>>>> * Python artifacts are deployed along with the source release to the >>>>>> dist.apache.org [2] and PyPI[8]. >>>>>> * Go artifacts and documentation are available at pkg.go.dev [9] >>>>>> * Validation sheet with a tab for 2.48.0 release to help with >>>>>> validation [10]. >>>>>> * Docker images published to Docker Hub [11]. >>>>>> * PR to run tests against release branch [12]. >>>>>> >>>>>> The vote will be open for at least 72 hours. It is adopted by >>>>>> majority approval, with at least 3 PMC affirmative votes. >>>>>> >>>>>> For guidelines on how to try the release in your projects, check out >>>>>> our blog post at /blog/validate-beam-release/. >>>>>> >>>>>> *NOTE: Dataflow containers for Python are not finalized yet (likely >>>>>> to happen on tuesday). I will follow up on this thread once that is done. >>>>>> Feel free to test it on other runners until then. * >>>>>> >>>>>> Thanks, >>>>>> Ritesh Ghorse >>>>>> >>>>>> [1] https://github.com/apache/beam/milestone/12 >>>>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.48.0/ >>>>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS >>>>>> [4] >>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1346/ >>>>>> [5] https://github.com/apache/beam/tree/v2.48.0-RC2 >>>>>> [6] https://github.com/apache/beam/pull/26903 >>>>>> [7] https://github.com/apache/beam-site/pull/645 >>>>>> [8] https://pypi.org/project/apache-beam/2.48.0rc2/ >>>>>> [9] >>>>>> https://pkg.go.dev/github.com/apache/beam/sdks/v2@v2.48.0-RC2/go/pkg/beam >>>>>> <https://pkg.go.dev/github.com/apache/beam/sdks/v2@v2.48.0-RC1/go/pkg/beam> >>>>>> [10] >>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=458120434 >>>>>> [11] https://hub.docker.com/search?q=apache%2Fbeam&type=image >>>>>> [12] https://github.com/apache/beam/pull/26811 >>>>>> >>>>>> >>>>
