Re: 2.41.0 Release PMC Finalization

[Kiley Sok via dev]

Thanks Cham!

On Tue, Aug 23, 2022 at 5:33 PM Chamikara Jayalath 
wrote:

> Should be done.
>
> Thanks,
> Cham
>
> On Tue, Aug 23, 2022 at 4:53 PM Chamikara Jayalath 
> wrote:
>
>> I can look into it.
>>
>> On Tue, Aug 23, 2022 at 2:02 PM Kiley Sok via dev 
>> wrote:
>>
>>> Could a PMC member help with the PMC-only finalization steps for 2.41.0
>>> [1]? Specifically:
>>>
>>> - Deploy source release to dist.apache.org
>>> - Recordkeeping with ASF
>>>
>>> Once those steps are done all that's left is to promote the release [2].
>>>
>>> Thank you!
>>> Kiley
>>>
>>> [1] https://beam.apache.org/contribute/release-guide/#pmc
>>> -only-finalization
>>> [2]
>>> https://beam.apache.org/contribute/release-guide/#12-promote-the-release
>>>
>>

2.41.0 Release PMC Finalization

[Kiley Sok via dev]

Could a PMC member help with the PMC-only finalization steps for 2.41.0
[1]? Specifically:

- Deploy source release to dist.apache.org
- Recordkeeping with ASF

Once those steps are done all that's left is to promote the release [2].

Thank you!
Kiley

[1] https://beam.apache.org/contribute/release-guide/#pmc-only-finalization
[2] https://beam.apache.org/contribute/release-guide/#12-promote-the-release

[RESULT] [VOTE] Release 2.41.0, release candidate #2

[Kiley Sok via dev]

I'm happy to announce that we have unanimously approved this release.

There are 5 approving votes, 3 of which are binding:
* Ahmet Altay
* Alexey Romanenko
* Chamikara Jayalath

There are no disapproving votes.

Thanks everyone!

[VOTE] Release 2.41.0, release candidate #2

[Kiley Sok via dev]

Hi everyone,
Please review and vote on the release candidate #1 for the version 2.41.0,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)


Reviewers are encouraged to test their own use cases with the release
candidate, and vote +1 if no issues are found.

The complete staging area is available for your review, which includes:
* GitHub Release notes [1],
* the official Apache source release to be deployed to dist.apache.org [2],
which is signed with the key with fingerprint
4D5731CC0AA38097D091EB091E7B28884452AE5D [3],
* all artifacts to be deployed to the Maven Central Repository [4],
* source code tag "v2.41.0-RC2" [5],
* website pull request listing the release [6], the blog post [6], and
publishing the API reference manual [7].
* Java artifacts were built with Gradle 7.4 and OpenJDK/Oracle JDK
1.8.0_312.
* Python artifacts are deployed along with the source release to the
dist.apache.org [2] and PyPI[8].
* Validation sheet with a tab for 2.41.0 release to help with validation
[9].
* Docker images published to Docker Hub [10].

The vote will be open for at least 72 hours. It is adopted by majority
approval, with at least 3 PMC affirmative votes.

For guidelines on how to try the release in your projects, check out our
blog post at https://beam.apache.org/blog/validate-beam-release/.

Thanks,
Release Manager

[1] https://github.com/apache/beam/milestone/3
[2] https://dist.apache.org/repos/dist/dev/beam/2.41.0/
[3] https://dist.apache.org/repos/dist/release/beam/KEYS
[4] https://repository.apache.org/content/repositories/orgapachebeam-1283/
[5] https://github.com/apache/beam/tree/v2.41.0-RC2
[6] https://github.com/apache/beam/pull/22706
[7] https://github.com/apache/beam-site/pull/633
[8] https://pypi.org/project/apache-beam/2.41.0rc2/
[9]
https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=331459080
[10] https://hub.docker.com/search?q=apache%2Fbeam=image

Re: [VOTE] Release 2.41.0, release candidate #1

[Kiley Sok via dev]

PR to revert the change for the release
https://github.com/apache/beam/pull/22759

I'll rebuild a new RC once the tests pass and the PR is merged.

On Wed, Aug 17, 2022 at 8:16 AM Liam Miller-Cushon 
wrote:

> On Tue, Aug 16, 2022 at 2:42 PM Kiley Sok  wrote:
>
>> Liam, are we okay to roll back this change for this release?
>>
>
> No concerns from me with rolling back to unblock the release.
>
> It looks like this is a change between bytebuddy 1.12.3 and 1.12.4. I
> filed https://github.com/raphw/byte-buddy/issues/1301 to get help
> understanding what changed, it sounds like the change might be WAI but
> there's a suggested fix. I will prepare a PR for that as a follow-up.
>

Re: [VOTE] Release 2.41.0, release candidate #1

[Kiley Sok via dev]

If we roll back, do we need to roll back +Lukasz Cwik
's change
[1] on master as well?

Liam, are we okay to roll back this change for this release?

[1] https://github.com/apache/beam/pull/22594

On Tue, Aug 16, 2022 at 2:25 PM Kenneth Knowles  wrote:

> Two options to unblock the release are:
>
> 1. Roll back https://github.com/apache/beam/pull/17317/files
> 2. Downgrade unvendored bytebuddy to 1.11.0 (if the above doesn't work or
> is too high risk)
>
> And as a follow up we should make sure there is some test that would
> exercise this, since that PR was green and was a while ago too and our
> postcommits did not catch it either.
>
> Kenn
>
> On Tue, Aug 16, 2022 at 12:50 PM Kiley Sok via dev 
> wrote:
>
>> cc: @Liam Miller-Cushon , who worked on the bytebuddy
>> update.
>>
>> Liam, do you have any context on this error?
>>
>> On Tue, Aug 16, 2022 at 10:11 AM Alexey Romanenko <
>> aromanenko@gmail.com> wrote:
>>
>>> I tested with "beam-samples" [1] and found that a rather simple test
>>> pipeline fails [2] with this runtime error:
>>>
>>> Error:
>>>  Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.776 s
>>> <<< FAILURE! - in SerializationTest
>>>
>>> 5809
>>> Error:
>>>  SerializationTest.nonSerilizableTest  Time elapsed: 2.708 s  <<< ERROR!
>>>
>>> 5810
>>> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException:
>>> java.lang.IllegalStateException: Invisible parameter type of
>>> SerializationTest$1 arg0 for public
>>> SerializationTest$1$DoFnInvoker(SerializationTest$1)
>>>
>>> 5811
>>> Caused by: java.lang.IllegalStateException: Invisible parameter type of
>>> SerializationTest$1 arg0 for public
>>> SerializationTest$1$DoFnInvoker(SerializationTest$1)
>>>
>>>
>>> Seems like that it’s caused by “bytebuddy” dependency update [3] from
>>> version 1.11.0 to 1.12.9 and it was vendored before (not sure if it’s
>>> related).
>>>
>>> Downgrading the “bytebuddy” version to 1.11.0 fixes an error.
>>>
>>> I’ve not yet gone deep into a cause of this problem but maybe someone
>>> knows some details?
>>>
>>> [1] https://github.com/Talend/beam-samples/
>>> [2]
>>> https://github.com/Talend/beam-samples/runs/7856722514?check_suite_focus=true
>>>
>>> [3] https://github.com/apache/beam/pull/17317
>>>
>>> —
>>> Alexey
>>>
>>> On 16 Aug 2022, at 17:54, Ritesh Ghorse via dev 
>>> wrote:
>>>
>>> +1 (non-binding), Validated Go SDK Quickstart on Direct and Dataflow
>>> runner
>>>
>>>
>>> On Tue, Aug 16, 2022 at 4:26 AM Jan Lukavský  wrote:
>>>
>>>> +1 (non-binding)
>>>>
>>>> Validated Java SDK with classical Flink Runner.
>>>> On 8/15/22 23:06, Chamikara Jayalath via dev wrote:
>>>>
>>>> +1 as well
>>>> (I believe Kiley is addressing the container tags issue)
>>>>
>>>> Thanks,
>>>> Cham
>>>>
>>>> On Mon, Aug 15, 2022 at 1:00 PM Robert Bradshaw 
>>>> wrote:
>>>>
>>>>> +1 (binding).
>>>>>
>>>>> I verified the release artifacts and signatures, and ran a couple of
>>>>> simple Python pipelines.
>>>>>
>>>>> On Mon, Aug 15, 2022 at 12:40 PM Chamikara Jayalath via dev
>>>>>  wrote:
>>>>> >
>>>>> >
>>>>> >
>>>>> > On Mon, Aug 15, 2022 at 11:37 AM Kiley Sok 
>>>>> wrote:
>>>>> >>
>>>>> >> Thanks everyone!
>>>>> >>
>>>>> >> @Chamikara Jayalath The Spark issue is running successfully for me,
>>>>> could you try it again? I'll look into the container tags.
>>>>> >
>>>>> >
>>>>> > Thanks. Regarding the Spark issue, it could just be my setup then.
>>>>> Feel free to close the Github issue.
>>>>> >
>>>>> > - Cham
>>>>> >
>>>>> >>
>>>>> >>
>>>>> >> On Mon, Aug 15, 2022 at 11:04 AM Pablo Estrada 
>>>>> wrote:
>>>>> >>>
>>>>> >>> +1 - I validated tests/build with existing Dataflow Templates
>>>>> 

Re: [VOTE] Release 2.41.0, release candidate #1

[Kiley Sok via dev]

cc: @Liam Miller-Cushon , who worked on the bytebuddy
update.

Liam, do you have any context on this error?

On Tue, Aug 16, 2022 at 10:11 AM Alexey Romanenko 
wrote:

> I tested with "beam-samples" [1] and found that a rather simple test
> pipeline fails [2] with this runtime error:
>
> Error:
>  Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 2.776 s
> <<< FAILURE! - in SerializationTest
>
> 5809
> Error:
>  SerializationTest.nonSerilizableTest  Time elapsed: 2.708 s  <<< ERROR!
>
> 5810
> org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.UncheckedExecutionException:
> java.lang.IllegalStateException: Invisible parameter type of
> SerializationTest$1 arg0 for public
> SerializationTest$1$DoFnInvoker(SerializationTest$1)
>
> 5811
> Caused by: java.lang.IllegalStateException: Invisible parameter type of
> SerializationTest$1 arg0 for public
> SerializationTest$1$DoFnInvoker(SerializationTest$1)
>
>
> Seems like that it’s caused by “bytebuddy” dependency update [3] from
> version 1.11.0 to 1.12.9 and it was vendored before (not sure if it’s
> related).
>
> Downgrading the “bytebuddy” version to 1.11.0 fixes an error.
>
> I’ve not yet gone deep into a cause of this problem but maybe someone
> knows some details?
>
> [1] https://github.com/Talend/beam-samples/
> [2]
> https://github.com/Talend/beam-samples/runs/7856722514?check_suite_focus=true
>
> [3] https://github.com/apache/beam/pull/17317
>
> —
> Alexey
>
> On 16 Aug 2022, at 17:54, Ritesh Ghorse via dev 
> wrote:
>
> +1 (non-binding), Validated Go SDK Quickstart on Direct and Dataflow runner
>
>
> On Tue, Aug 16, 2022 at 4:26 AM Jan Lukavský  wrote:
>
>> +1 (non-binding)
>>
>> Validated Java SDK with classical Flink Runner.
>> On 8/15/22 23:06, Chamikara Jayalath via dev wrote:
>>
>> +1 as well
>> (I believe Kiley is addressing the container tags issue)
>>
>> Thanks,
>> Cham
>>
>> On Mon, Aug 15, 2022 at 1:00 PM Robert Bradshaw 
>> wrote:
>>
>>> +1 (binding).
>>>
>>> I verified the release artifacts and signatures, and ran a couple of
>>> simple Python pipelines.
>>>
>>> On Mon, Aug 15, 2022 at 12:40 PM Chamikara Jayalath via dev
>>>  wrote:
>>> >
>>> >
>>> >
>>> > On Mon, Aug 15, 2022 at 11:37 AM Kiley Sok 
>>> wrote:
>>> >>
>>> >> Thanks everyone!
>>> >>
>>> >> @Chamikara Jayalath The Spark issue is running successfully for me,
>>> could you try it again? I'll look into the container tags.
>>> >
>>> >
>>> > Thanks. Regarding the Spark issue, it could just be my setup then.
>>> Feel free to close the Github issue.
>>> >
>>> > - Cham
>>> >
>>> >>
>>> >>
>>> >> On Mon, Aug 15, 2022 at 11:04 AM Pablo Estrada 
>>> wrote:
>>> >>>
>>> >>> +1 - I validated tests/build with existing Dataflow Templates
>>> >>> Best
>>> >>> -P.
>>> >>>
>>> >>> On Fri, Aug 12, 2022 at 9:20 PM Ahmet Altay via dev <
>>> dev@beam.apache.org> wrote:
>>> >>>>
>>> >>>> +1 - I validated python quickstarts on direct runner.
>>> >>>>
>>> >>>> Thank you Kiley!
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> On Thu, Aug 11, 2022 at 9:56 PM Kiley Sok via dev <
>>> dev@beam.apache.org> wrote:
>>> >>>>>
>>> >>>>> Hi everyone,
>>> >>>>> Please review and vote on the release candidate #1 for the version
>>> 2.41.0, as follows:
>>> >>>>> [ ] +1, Approve the release
>>> >>>>> [ ] -1, Do not approve the release (please provide specific
>>> comments)
>>> >>>>>
>>> >>>>>
>>> >>>>> Reviewers are encouraged to test their own use cases with the
>>> release candidate, and vote +1 if no issues are found.
>>> >>>>>
>>> >>>>> The complete staging area is available for your review, which
>>> includes:
>>> >>>>> * GitHub Release notes [1],
>>> >>>>> * the official Apache source release to be deployed to
>>> dist.apache.org [2], which is signed with the key with fingerprint

Re: [VOTE] Release 2.41.0, release candidate #1

[Kiley Sok via dev]

Thanks everyone!

@Chamikara Jayalath  The Spark issue is running
successfully for me, could you try it again? I'll look into the container
tags.

On Mon, Aug 15, 2022 at 11:04 AM Pablo Estrada  wrote:

> +1 - I validated tests/build with existing Dataflow Templates
> Best
> -P.
>
> On Fri, Aug 12, 2022 at 9:20 PM Ahmet Altay via dev 
> wrote:
>
>> +1 - I validated python quickstarts on direct runner.
>>
>> Thank you Kiley!
>>
>>
>>
>> On Thu, Aug 11, 2022 at 9:56 PM Kiley Sok via dev 
>> wrote:
>>
>>> Hi everyone,
>>> Please review and vote on the release candidate #1 for the version
>>> 2.41.0, as follows:
>>> [ ] +1, Approve the release
>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>
>>>
>>> Reviewers are encouraged to test their own use cases with the release
>>> candidate, and vote +1 if no issues are found.
>>>
>>> The complete staging area is available for your review, which includes:
>>> * GitHub Release notes [1],
>>> * the official Apache source release to be deployed to dist.apache.org
>>> [2], which is signed with the key with fingerprint
>>> 4D5731CC0AA38097D091EB091E7B28884452AE5D [3],
>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>> * source code tag "v2.41.0-RC1" [5],
>>> * website pull request listing the release [6], the blog post [6], and
>>> publishing the API reference manual [7].
>>> * Java artifacts were built with Gradle 7.4 and OpenJDK/Oracle JDK
>>> 1.8.0_232.
>>> * Python artifacts are deployed along with the source release to the
>>> dist.apache.org [2] and PyPI[8].
>>> * Validation sheet with a tab for 2.41.0 release to help with validation
>>> [9].
>>> * Docker images published to Docker Hub [10].
>>>
>>> The vote will be open for at least 72 hours. It is adopted by majority
>>> approval, with at least 3 PMC affirmative votes.
>>>
>>> For guidelines on how to try the release in your projects, check out our
>>> blog post at https://beam.apache.org/blog/validate-beam-release/.
>>>
>>> Thanks,
>>> Release Manager
>>>
>>> [1] https://github.com/apache/beam/milestone/3
>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.41.0/
>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>> [4]
>>> https://repository.apache.org/content/repositories/orgapachebeam-1282/
>>> [5] https://github.com/apache/beam/tree/v2.41.0-RC1
>>> [6] https://github.com/apache/beam/pull/22706
>>> [7] https://github.com/apache/beam-site/pull/633
>>> [8] https://pypi.org/project/apache-beam/2.41.0rc1/
>>> [9]
>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=331459080
>>> [10] https://hub.docker.com/search?q=apache%2Fbeam=image
>>>
>>

[VOTE] Release 2.41.0, release candidate #1

[Kiley Sok via dev]

Hi everyone,
Please review and vote on the release candidate #1 for the version 2.41.0,
as follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)


Reviewers are encouraged to test their own use cases with the release
candidate, and vote +1 if no issues are found.

The complete staging area is available for your review, which includes:
* GitHub Release notes [1],
* the official Apache source release to be deployed to dist.apache.org [2],
which is signed with the key with fingerprint
4D5731CC0AA38097D091EB091E7B28884452AE5D [3],
* all artifacts to be deployed to the Maven Central Repository [4],
* source code tag "v2.41.0-RC1" [5],
* website pull request listing the release [6], the blog post [6], and
publishing the API reference manual [7].
* Java artifacts were built with Gradle 7.4 and OpenJDK/Oracle JDK
1.8.0_232.
* Python artifacts are deployed along with the source release to the
dist.apache.org [2] and PyPI[8].
* Validation sheet with a tab for 2.41.0 release to help with validation
[9].
* Docker images published to Docker Hub [10].

The vote will be open for at least 72 hours. It is adopted by majority
approval, with at least 3 PMC affirmative votes.

For guidelines on how to try the release in your projects, check out our
blog post at https://beam.apache.org/blog/validate-beam-release/.

Thanks,
Release Manager

[1] https://github.com/apache/beam/milestone/3
[2] https://dist.apache.org/repos/dist/dev/beam/2.41.0/
[3] https://dist.apache.org/repos/dist/release/beam/KEYS
[4] https://repository.apache.org/content/repositories/orgapachebeam-1282/
[5] https://github.com/apache/beam/tree/v2.41.0-RC1
[6] https://github.com/apache/beam/pull/22706
[7] https://github.com/apache/beam-site/pull/633
[8] https://pypi.org/project/apache-beam/2.41.0rc1/
[9]
https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=331459080
[10] https://hub.docker.com/search?q=apache%2Fbeam=image

Re: [Release] 2.41.0 release update

[Kiley Sok via dev]

There's one new issue found while building the RC. Looking into it now
https://github.com/apache/beam/issues/22621

On Fri, Aug 5, 2022 at 10:21 AM Ahmet Altay  wrote:

> Kiley, do we still have the same blockers? Do you need any help?
>
> On Thu, Aug 4, 2022 at 12:18 PM Kiley Sok via dev 
> wrote:
>
>> Last remaining issue was cherry-picked. There may be one last issue with
>> gRPC that's being investigated.
>>
>> https://github.com/apache/beam/issues/22283
>>
>> On Thu, Jul 28, 2022 at 2:20 PM Kiley Sok  wrote:
>>
>>> Quick update for today:
>>>
>>> I'm still working through the validation tests, but we currently have 2
>>> open issues:
>>> https://github.com/apache/beam/issues/22454
>>> https://github.com/apache/beam/issues/22188
>>>
>>>
>>>
>>> On Wed, Jul 27, 2022 at 5:03 PM Kiley Sok  wrote:
>>>
>>>> Hi all,
>>>>
>>>> I've cut the release branch:
>>>> https://github.com/apache/beam/tree/release-2.41.0
>>>>
>>>> There's one known issue
>>>> <https://github.com/apache/beam/issues?q=is%3Aopen+is%3Aissue+milestone%3A%222.41.0+Release%22>
>>>>  that
>>>> needs to be cherry picked. Please let me know if you have a change that
>>>> needs to go in.
>>>>
>>>> Thanks,
>>>> Kiley
>>>>
>>>>

Re: [Release] 2.41.0 release update

[Kiley Sok via dev]

Last remaining issue was cherry-picked. There may be one last issue with
gRPC that's being investigated.

https://github.com/apache/beam/issues/22283

On Thu, Jul 28, 2022 at 2:20 PM Kiley Sok  wrote:

> Quick update for today:
>
> I'm still working through the validation tests, but we currently have 2
> open issues:
> https://github.com/apache/beam/issues/22454
> https://github.com/apache/beam/issues/22188
>
>
>
> On Wed, Jul 27, 2022 at 5:03 PM Kiley Sok  wrote:
>
>> Hi all,
>>
>> I've cut the release branch:
>> https://github.com/apache/beam/tree/release-2.41.0
>>
>> There's one known issue
>> 
>>  that
>> needs to be cherry picked. Please let me know if you have a change that
>> needs to go in.
>>
>> Thanks,
>> Kiley
>>
>>

Re: [Release] 2.41.0 release update

[Kiley Sok via dev]

Quick update for today:

I'm still working through the validation tests, but we currently have 2
open issues:
https://github.com/apache/beam/issues/22454
https://github.com/apache/beam/issues/22188



On Wed, Jul 27, 2022 at 5:03 PM Kiley Sok  wrote:

> Hi all,
>
> I've cut the release branch:
> https://github.com/apache/beam/tree/release-2.41.0
>
> There's one known issue
> 
>  that
> needs to be cherry picked. Please let me know if you have a change that
> needs to go in.
>
> Thanks,
> Kiley
>
>

[Release] 2.41.0 release update

[Kiley Sok via dev]

Hi all,

I've cut the release branch:
https://github.com/apache/beam/tree/release-2.41.0

There's one known issue

that
needs to be cherry picked. Please let me know if you have a change that
needs to go in.

Thanks,
Kiley

[PROPOSAL] Preparing for 2.41.0 Release

[Kiley Sok via dev]

Hi all,

The next (2.41.0) release branch cut is scheduled for July 27th, according
to
the release calendar [1].

I would like to volunteer myself to do this release. My plan is to cut the
branch on that date, and cherrypick release-blocking fixes afterwards, if
any.

Please help me make sure the release goes smoothly by:
- Making sure that any unresolved release blocking issues for 2.41.0 should
have their "Milestone" marked as "2.41.0" as soon as possible.
- Reviewing the current release blockers [2] and remove the Milestone if
they don't meet the criteria at [3].

Let me know if you have any comments/objections/questions.

Thanks,
Kiley

[1]
https://calendar.google.com/calendar/embed?src=0p73sl034k80oob7seouanigd0%40group.calendar.google.com
[2] https://github.com/apache/beam/milestone/3
[3] https://beam.apache.org/contribute/release-blocking/