Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-02-02 Thread Kenneth Knowles
Update: some blockers were discovered and we have fixes, but I need help getting fixes reviewed and cherrypicked into the release branch. I'd appreciate any attention on the following: Fix certain dependency errors and add analysis: https://github.com/apache/beam/pull/7700 Cherrypick the above to

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-25 Thread Kenneth Knowles
Just a quick update - I'm working through the RC generating process still. I've run into issues in the scripts / gradle release plugin and some flakes but proceeding manually seems to be going OK. Kenn On Thu, Jan 24, 2019 at 12:21 PM Kenneth Knowles wrote: > My mistake - the problem was that

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-24 Thread Kenneth Knowles
My mistake - the problem was that the Dataflow container image was not yet built, but I did not dig deep enough into StackDriver to find that message. All issues are resolved and all PRs against the release branch are now merged. I am now in process of building those containers and RC1. Kenn On

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-23 Thread Ahmet Altay
Python precommit cron job ( https://builds.apache.org/view/A-D/view/Beam/view/PostCommit/job/beam_PreCommit_Python_Cron/) seems to be healthy. Could you share one of the PRs that you noticed the flakiness, I would be interested in debugging. On Wed, Jan 23, 2019 at 12:25 PM Kenneth Knowles

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-23 Thread Kenneth Knowles
I've seen many PRs that I am reviewing with flakiness problems in the python precommit IT. Anyone have any insights? Kenn On Wed, Jan 23, 2019 at 12:23 PM Kenneth Knowles wrote: > Thanks for the reminder. I was reading from > https://issues.apache.org/jira/projects/BEAM/versions/12344540. RC1

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-23 Thread Kenneth Knowles
Thanks for the reminder. I was reading from https://issues.apache.org/jira/projects/BEAM/versions/12344540. RC1 also gated on clearing https://github.com/apache/beam/pulls?utf8=%E2%9C%93=is%3Apr+is%3Aopen+base%3Arelease-2.10.0 . Kenn On Wed, Jan 23, 2019 at 9:44 AM Maximilian Michels wrote: >

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-23 Thread Maximilian Michels
What about the revert of "Parse SDK-unknown pipeline options"? https://github.com/apache/beam/pull/7564 Should we merge this for the release? On 23.01.19 11:56, Scott Wegner wrote: Cherry-pick PR for the last-remaining issue: https://github.com/apache/beam/pull/7603 On Wed, Jan 23, 2019 at

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-23 Thread Scott Wegner
Cherry-pick PR for the last-remaining issue: https://github.com/apache/beam/pull/7603 On Wed, Jan 23, 2019 at 7:15 AM Kenneth Knowles wrote: > The last remaining issue is split into a non-blocker and a revert that is > confirmed to fix the issue. > > Once

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-23 Thread Kenneth Knowles
The last remaining issue is split into a non-blocker and a revert that is confirmed to fix the issue. Once https://github.com/apache/beam/pull/7600 is merged and cherry-picked, I will cut RC1 today. Kenn On Tue, Jan 22, 2019 at 6:03 PM Kenneth Knowles wrote: > OK. There is just one release

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-22 Thread Kenneth Knowles
OK. There is just one release blocker remaining; https://issues.apache.org/jira/browse/BEAM-6354 I have no insights yet, but I am bisecting. It was healthy in 2.9.0. Kenn On Tue, Jan 22, 2019 at 9:38 AM Scott Wegner wrote: > The rollback for BEAM-6352 is now in and cherry-picked into the

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-22 Thread Scott Wegner
The rollback for BEAM-6352 is now in and cherry-picked into the release branch. On Fri, Jan 18, 2019 at 9:04 AM Scott Wegner wrote: > For BEAM-6352, I have a rollback ready for review: > https://github.com/apache/beam/pull/7540 > Conversation about the decision to rollback vs. roll-forward for

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-18 Thread Scott Wegner
For BEAM-6352, I have a rollback ready for review: https://github.com/apache/beam/pull/7540 Conversation about the decision to rollback vs. roll-forward for this change is on the JIRA issue. On Fri, Jan 18, 2019 at 8:22 AM Maximilian Michels wrote: > I've created the revert for the pipeline

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-18 Thread Maximilian Michels
I've created the revert for the pipeline options parsing which we agreed on: https://github.com/apache/beam/pull/7564 On 17.01.19 15:16, Maximilian Michels wrote: An issue with the Flink Runner when restarting streaming pipelines: https://jira.apache.org/jira/browse/BEAM-6460 Looks like it

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-17 Thread Maximilian Michels
An issue with the Flink Runner when restarting streaming pipelines: https://jira.apache.org/jira/browse/BEAM-6460 Looks like it will be easy to fix by invalidating the Jackson cache. -Max On 16.01.19 23:00, Kenneth Knowles wrote: Quick update on this. There are three remaining issues:  -

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-16 Thread Kenneth Knowles
Quick update on this. There are three remaining issues: - https://issues.apache.org/jira/browse/BEAM-6407: A DirectRunner self-check was broken from 2.8.0 to 2.9.0 - PR looks good modulo our infra flakes - https://issues.apache.org/jira/browse/BEAM-6354: PAssert + DirectRunner + Unbounded data

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-16 Thread Kenneth Knowles
Thanks, Ismaël! On Wed, Jan 16, 2019 at 2:13 AM Ismaël Mejía wrote: > Ok since there were not many issues I did the 'update' for the > misplaced issues to version 2.10. We are good to go. New resolved > issues in master musg go now into 2.11.0 > > On Wed, Jan 16, 2019 at 10:38 AM Ismaël Mejía

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-16 Thread Ismaël Mejía
Ok since there were not many issues I did the 'update' for the misplaced issues to version 2.10. We are good to go. New resolved issues in master musg go now into 2.11.0 On Wed, Jan 16, 2019 at 10:38 AM Ismaël Mejía wrote: > > This means that the tickets resolved and marked for 2.11 since

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-16 Thread Ismaël Mejía
This means that the tickets resolved and marked for 2.11 since January 2 should be reviewed and retargetted to version 2.10. So this is a call for action for committers who have merged fixes after the cut to update the tickets if required. Ismaël On Tue, Jan 15, 2019 at 9:22 PM Kenneth Knowles

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-15 Thread Kenneth Knowles
As a heads up, I did not realize that the release guide specified a custom process for starting a release branch. It makes sense; cut_release_branch.sh consolidates knowledge about all the places the version is hardcoded in the codebase. To keep the history simple, I will re-cut the release branch

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-15 Thread Kenneth Knowles
I'm on it. On Tue, Jan 15, 2019 at 8:10 AM Ismaël Mejía wrote: > There is also another issue, after the 2.10.0 branch cut some > identifier in the build was not changed and the Apache Beam Snapshots > keep generating SNAPSHOTS for 2.10.0 instead of the now current > 2.11.0-SNAPSHOT. Can

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-15 Thread Ismaël Mejía
There is also another issue, after the 2.10.0 branch cut some identifier in the build was not changed and the Apache Beam Snapshots keep generating SNAPSHOTS for 2.10.0 instead of the now current 2.11.0-SNAPSHOT. Can somebody PTAL? On Thu, Jan 3, 2019 at 6:17 PM Maximilian Michels wrote: > >

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-03 Thread Maximilian Michels
Thanks for driving this Kenn! I'm in favor of a strict cut off, but I'd like to propose a week for cherry-picking relevant changes to the release branch. It looks like many people are returning from holidays or are still off. Cheers, Max On 02.01.19 17:20, Kenneth Knowles wrote: Done. I've

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-02 Thread Ismaël Mejía
Can you please create 2.11 tag in JIRA so we can move the JIRAs that are not blocking. I have quite a bunch of pending code reviews that hoped to get into this one but well now probably they shall wait. (My excuses for the people who may be impacted, I had not checked that the date was in the

Re: [PROPOSAL] Prepare Beam 2.10.0 release

2019-01-02 Thread Jean-Baptiste Onofré
It sounds good to me. Regards JB On 02/01/2019 16:16, Kenneth Knowles wrote: > Hi All, > > According to the release calendar [1] branch cut date for > Beam 2.10.0 release is today, 2019 January 2. I'd like to volunteer to > manage this release. Does anyone have any reason we should not release

[PROPOSAL] Prepare Beam 2.10.0 release

2019-01-02 Thread Kenneth Knowles
Hi All, According to the release calendar [1] branch cut date for Beam 2.10.0 release is today, 2019 January 2. I'd like to volunteer to manage this release. Does anyone have any reason we should not release on schedule? Otherwise, if you know of release-blocking bugs, please mark their "Fix