Thank you Kyle and Ismael! In that case, I'll move forward to build RC2. Best -P.
On Mon, Dec 28, 2020 at 10:53 AM Kyle Weaver <[email protected]> wrote: > > I am not sure what's the status of this work - should this block the > release until we can get those containers out? > > We can release Flink 1.11 and Flink 1.12 containers and I'm *pretty sure* > they would work correctly, but to be totally sure we'd have to upgrade our > portable Flink tests to cover the new containers. Rather than rush new > portable Flink versions into the release, I think it may be better to > document that they aren't officially supported for now (users can always > build the job server containers manually if they need). > > The larger problem here is that we need to improve the process of adding > new Flink versions. Specifically, we need a single source (or at least > fewer sources) of truth for supported Flink versions so that we don't have > things getting out of sync again. > https://issues.apache.org/jira/browse/BEAM-9541 > > On Mon, Dec 28, 2020 at 10:31 AM Pablo Estrada <[email protected]> wrote: > >> As an update on the release, I've been made aware of, and merged all >> cherry picks. I was in the process of building rc2, but I'm pausing to >> resolve the issue with these containers. >> Best >> -P. >> >> On Mon, Dec 28, 2020, 1:11 PM Pablo Estrada <[email protected]> wrote: >> >>> Good catch Ismael. Thanks! >>> >>> I've created https://issues.apache.org/jira/browse/INFRA-21238 to >>> request the repositories to be created. >>> >>> I am not sure what's the status of this work - should this block the >>> release until we can get those containers out? >>> cc: @Kyle Weaver <[email protected]> >>> Best >>> -P. >>> >>> On Mon, Dec 28, 2020 at 9:48 AM Ismaël Mejía <[email protected]> wrote: >>> >>>> It seems we are not publishing the latest versions of the Flink Job >>>> Server (Flink 1.11 and 1.12) docker images >>>> >>>> These do not exist: >>>> >>>> https://hub.docker.com/r/apache/beam_flink1.11_job_server >>>> https://hub.docker.com/r/apache/beam_flink1.12_job_server >>>> >>>> but this does and has the good 2.27.0_rc1 tag: >>>> https://hub.docker.com/r/apache/beam_flink1.10_job_server >>>> >>>> I wonder if the issue might be related to the fact that we need to >>>> request the repo to be created or if it is something different. >>>> >>>> On Thu, Dec 24, 2020 at 5:33 PM Brian Hulette <[email protected]> >>>> wrote: >>>> > >>>> > +Boyuan Zhang helped me get to the bottom of the sql_taxi issue. The >>>> problem is with the WriteStringsToPubSub API, which is deprecated since >>>> 2.7.0, but used in the example. Boyuan has [1] out to fix >>>> WriteStringsToPubSub and I just sent [2] to replace WriteStringsToPubSub >>>> with WriteToPubSub in example code. Issue is tracked in [3]. >>>> > >>>> > [1] https://github.com/apache/beam/pull/13614 >>>> > [2] https://github.com/apache/beam/pull/13615 >>>> > [3] https://issues.apache.org/jira/browse/BEAM-11524 >>>> > >>>> > On Thu, Dec 24, 2020 at 8:26 AM Pablo Estrada <[email protected]> >>>> wrote: >>>> >> >>>> >> Alright! Thanks everyone for your validations. I'm cancelling this >>>> RC, and I'll perform cherry picks to prepare the next one. >>>> >> >>>> >> Please update this thread with any other cherry pick requests! >>>> >> -P. >>>> >> >>>> >> On Thu, Dec 24, 2020, 3:17 AM Ismaël Mejía <[email protected]> >>>> wrote: >>>> >>> >>>> >>> It might be a good idea to include also: >>>> >>> >>>> >>> [BEAM-11403] Cache UnboundedReader per UnboundedSourceRestriction in >>>> >>> SDF Wrapper DoFn >>>> >>> https://github.com/apache/beam/pull/13592 >>>> >>> >>>> >>> So Java development experience is less affected (as with 2.26.0) >>>> (There >>>> >>> is a flag to exclude but defaults matter). >>>> >>> >>>> >>> On Thu, Dec 24, 2020 at 2:56 AM Valentyn Tymofieiev < >>>> [email protected]> wrote: >>>> >>> > >>>> >>> > We discovered a regression on CombineFn.from_callable() started >>>> in 2.26.0. Even though it's not a regression in 2.27.0, I strongly prefer >>>> we fix it in 2.27.0 as it leads to buggy behavior, so I vote -1. >>>> >>> > >>>> >>> > The fix to release branch is in flight: >>>> https://github.com/apache/beam/pull/13613. >>>> >>> > >>>> >>> > >>>> >>> > >>>> >>> > On Wed, Dec 23, 2020 at 3:38 PM Brian Hulette < >>>> [email protected]> wrote: >>>> >>> >> >>>> >>> >> -1 (non-binding) >>>> >>> >> Good news: I validated a dataframe pipeline on Dataflow which >>>> looked good (with expected performance improvements!) >>>> >>> >> Bad news: I also tried to run the sql_taxi example pipeline >>>> (streaming SQL in python) on Dataflow and ran into PubSub IO related >>>> issues. The example fails in the same way with 2.26.0, but it works in >>>> 2.25.0. It's possible this is a Dataflow bug and not a Beam one, but I'd >>>> like to investigate further to make sure. >>>> >>> >> >>>> >>> >> On Wed, Dec 23, 2020 at 12:25 PM Kyle Weaver < >>>> [email protected]> wrote: >>>> >>> >>> >>>> >>> >>> +1 (non-binding) Validated wordcount with Python source + Flink >>>> and Spark job server jars. Also checked that the ...:sql:udf jar was added >>>> and includes our cherry-picks. Thanks Pablo :) >>>> >>> >>> >>>> >>> >>> On Wed, Dec 23, 2020 at 12:02 PM Ahmet Altay <[email protected]> >>>> wrote: >>>> >>> >>>> >>>> >>> >>>> +1 (binding). >>>> >>> >>>> >>>> >>> >>>> I validated python quickstarts. Thank you Pablo. >>>> >>> >>>> >>>> >>> >>>> On Tue, Dec 22, 2020 at 10:04 PM Jean-Baptiste Onofre < >>>> [email protected]> wrote: >>>> >>> >>>>> >>>> >>> >>>>> +1 (binding) >>>> >>> >>>>> >>>> >>> >>>>> Regards >>>> >>> >>>>> JB >>>> >>> >>>>> >>>> >>> >>>>> Le 23 déc. 2020 à 06:46, Pablo Estrada <[email protected]> >>>> a écrit : >>>> >>> >>>>> >>>> >>> >>>>> Hi everyone, >>>> >>> >>>>> Please review and vote on the release candidate #1 for the >>>> version 2.27.0, as follows: >>>> >>> >>>>> [ ] +1, Approve the release >>>> >>> >>>>> [ ] -1, Do not approve the release (please provide specific >>>> comments) >>>> >>> >>>>> >>>> >>> >>>>> >>>> >>> >>>>> Reviewers are encouraged to test their own use cases with the >>>> release candidate, and vote +1 >>>> >>> >>>>> if no issues are found. >>>> >>> >>>>> >>>> >>> >>>>> The complete staging area is available for your review, which >>>> includes: >>>> >>> >>>>> * JIRA release notes [1], >>>> >>> >>>>> * the official Apache source release to be deployed to >>>> dist.apache.org [2], which is signed with the key with fingerprint >>>> C79DDD47DAF3808F0B9DDFAC02B2D9F742008494 [3], >>>> >>> >>>>> * all artifacts to be deployed to the Maven Central >>>> Repository [4], >>>> >>> >>>>> * source code tag "v2.27.0-RC1" [5], >>>> >>> >>>>> * website pull request listing the release [6], publishing >>>> the API reference manual [7], and the blog post [8]. >>>> >>> >>>>> * Python artifacts are deployed along with the source release >>>> to the dist.apache.org [2]. >>>> >>> >>>>> * Validation sheet with a tab for 2.27.0 release to help with >>>> validation [9]. >>>> >>> >>>>> * Docker images published to Docker Hub [10]. >>>> >>> >>>>> >>>> >>> >>>>> The vote will be open for at least 72 hours, but given the >>>> holidays, we will likely extend for a few more days. The release will be >>>> adopted by majority approval, with at least 3 PMC affirmative votes. >>>> >>> >>>>> >>>> >>> >>>>> Thanks, >>>> >>> >>>>> -P. >>>> >>> >>>>> >>>> >>> >>>>> [1] >>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12349380 >>>> >>> >>>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.27.0/ >>>> >>> >>>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS >>>> >>> >>>>> [4] >>>> https://repository.apache.org/content/repositories/orgapachebeam-1145/ >>>> >>> >>>>> [5] https://github.com/apache/beam/tree/v2.27.0-RC1 >>>> >>> >>>>> [6] https://github.com/apache/beam/pull/13602 >>>> >>> >>>>> [7] https://github.com/apache/beam-site/pull/610 >>>> >>> >>>>> [8] https://github.com/apache/beam/pull/13603 >>>> >>> >>>>> [9] >>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=194829106 >>>> >>> >>>>> [10] https://hub.docker.com/search?q=apache%2Fbeam&type=image >>>> >>> >>>>> >>>> >>> >>>>> >>>> >>>
