-1
Seems like Dataflow x-lang jobs might be broken for Java 11 users due to
Java SDK defaulting to Java 11 containers [1] which are not released with
Beam 2.26.0 [2]. +Emily Ye <[email protected]> is looking into this.
Thanks,
Cham
[1]
Error: Error syncing pod 0a07bd35d4a1e02006563901e421445f
("dataflow-kafka-taxi-20201207-10105-12071011-oij4-harness-6ss0_default(0a07bd35d4a1e02006563901e421445f)"),
skipping: failed to "StartContainer" for "sdk-1-0" with ImagePullBackOff:
"Back-off pulling image \"apache/beam_java11_sdk:2.26.0\""
[2] https://hub.docker.com/r/apache/beam_java11_sdk
On Mon, Dec 7, 2020 at 9:41 AM Robert Burke <[email protected]> wrote:
> Last Friday I re-built and re-pushed the containers that Ahmet found. I'm
> going to redo that validation and try and get through additional validation
> today.
>
> As it stands I'm back to a +0 due to Ahmets position on the python 3.7
> tests. (I'm not familiar with the python suites, so I defer to an expert)
>
> I'll send another update at the end of my day
>
> On Fri, Dec 4, 2020, 2:13 PM Ahmet Altay <[email protected]> wrote:
>
>>
>>
>> On Fri, Dec 4, 2020 at 2:03 PM Robert Burke <[email protected]> wrote:
>>
>>> Is this an issue on the dataflow side or with the quick start
>>> instructions?
>>>
>>
>> It is an issue with how the dataflow container is created. Quick start
>> instructions are fine.
>>
>>
>>>
>>> On Fri, Dec 4, 2020, 1:43 PM Ahmet Altay <[email protected]> wrote:
>>>
>>>> -1 - I validated the python quickstarts with python 3.8. All worked
>>>> except for the Dataflow batch pipeline. I will update my vote once this is
>>>> resolved.
>>>>
>>>> Dataflow batch pipeline failed with the "RuntimeError: Beam SDK base
>>>> version 2.26.0 does not match Dataflow Python worker version 2.25.0.dev.
>>>> Please check Dataflow worker startup logs and make sure that correct
>>>> version of Beam SDK is installed." error. This could be fixed by updating
>>>> the relevant dataflow container and keeping this RC as is otherwise.
>>>>
>>>> This was an issue in the previous 2 releases as well. Does anyone have
>>>> a suggestion on how we could catch this before an RC email.
>>>>
>>>> On Fri, Dec 4, 2020 at 10:38 AM Ahmet Altay <[email protected]> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Thu, Dec 3, 2020 at 8:41 PM Robert Burke <[email protected]>
>>>>> wrote:
>>>>>
>>>>>> Whoops, apologies for the title typo. Several years before we hit
>>>>>> 2.62.0 !
>>>>>>
>>>>>> Regarding this RC:
>>>>>>
>>>>>> -1 There's an extant issue with Python 3.7
>>>>>> :sdks:python:test-suites:direct:py37:directRunnerIT . discuss in
>>>>>> https://issues.apache.org/jira/browse/BEAM-11396
>>>>>>
>>>>>
>>>>> I do not think this is a blocker.
>>>>>
>>>>> I looked at the issue. It looks like it is either a network or an auth
>>>>> issue contained in the pubsub dependency used to set up the test
>>>>> environment. The test fails to create a pubsub subscription (and this is
>>>>> unrelated to pubsub io). Given that the same IT test runs and works on
>>>>> other runners, we can consider this an issue with the test setup itself
>>>>> for
>>>>> the direct runner.
>>>>>
>>>>>
>>>>>> I felt the need to finish RC tasks at least once end to end. This has
>>>>>> been a challenging month to get anything done. On top of that among the
>>>>>> multitude of Beam contributors, I'm one of the few who do not interact
>>>>>> with
>>>>>> either Python or Java regularly. On top of that, I do most of my work
>>>>>> from
>>>>>> a Chromebook using Crostini, which is essentially a lightweight Debian
>>>>>> container. It doesn't have much pre-installed.
>>>>>> Conveniently this means I get to find every single configuration
>>>>>> assumption that's been made as part of the release process, which I've
>>>>>> taken notes on. I'll be contributing these learnings back to the release
>>>>>> guide in a PR in the next few weeks. Overall it feels like an
>>>>>> unobstructed
>>>>>> RC should only take about a day to sort out, if some parts of the process
>>>>>> are clarified via expectation setting and other guide updates.
>>>>>> It's in that spirit that I sent out this RC00, so validation on
>>>>>> 2.26.0 can start on the rest of the components in the meantime. Now that
>>>>>> I've gone through the process once, remaining RCs will be prompt.
>>>>>>
>>>>>> With that in mind, if there are any additional regressions from
>>>>>> 2.25.0 nows the time to file a JIRA for triage in order to have fixes
>>>>>> cherry picked for them.
>>>>>>
>>>>>> Cheers,
>>>>>> Robert Burke
>>>>>> 2.26.0 Release manager.
>>>>>>
>>>>>>
>>>>>> On Thu, 3 Dec 2020 at 20:01, Robert Burke <[email protected]> wrote:
>>>>>>
>>>>>>> Hi everyone,
>>>>>>> Please review and vote on the release candidate #00 for the version
>>>>>>> 2.26.0, as follows:
>>>>>>> [ ] +1, Approve the release
>>>>>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>>>>>
>>>>>>>
>>>>>>> Reviewers are encouraged to test their own use cases with the
>>>>>>> release candidate, and vote +1
>>>>>>> if no issues are found.
>>>>>>>
>>>>>>> The complete staging area is available for your review, which
>>>>>>> includes:
>>>>>>> * JIRA release notes [1],
>>>>>>> * the official Apache source release to be deployed to
>>>>>>> dist.apache.org [2], which is signed with the key with fingerprint
>>>>>>> A52F5C83BAE26160120EC25F3D56ACFBFB2975E1 [3],
>>>>>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>>>>>> * source code tag "v2.26.0-RC00" [5],
>>>>>>> * website pull request listing the release [6], publishing the API
>>>>>>> reference manual [7], and the blog post [8].
>>>>>>> * Java artifacts were built with Maven 3.6.0 and OpenJDK 1.8.0_275.
>>>>>>> * Python artifacts are deployed along with the source release to the
>>>>>>> dist.apache.org [2].
>>>>>>> * Validation sheet with a tab for 2.26.0 release to help with
>>>>>>> validation [9].
>>>>>>> * Docker images published to Docker Hub [10].
>>>>>>>
>>>>>>> The vote will be open for at least 72 hours. It is adopted by
>>>>>>> majority approval, with at least 3 PMC affirmative votes.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Robert Burke
>>>>>>> 2.26.0 Release Manager
>>>>>>>
>>>>>>> [1]
>>>>>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12348833
>>>>>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.26.0/
>>>>>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>>>>>> [4]
>>>>>>> https://repository.apache.org/content/repositories/orgapachebeam-1143/
>>>>>>> [5] https://github.com/apache/beam/tree/v2.26.0-RC00
>>>>>>> [6] https://github.com/apache/beam/pull/13481
>>>>>>> [7] https://github.com/apache/beam-site/pull/609
>>>>>>> [8] https://github.com/apache/beam/pull/13482
>>>>>>> [9]
>>>>>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=.
>>>>>>> ..
>>>>>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&type=image
>>>>>>>
>>>>>>
