See my comment below. > On 26 Aug 2021, at 03:44, Ahmet Altay <[email protected]> wrote: > > On Wed, Aug 25, 2021 at 1:50 PM Udi Meiri <[email protected] > <mailto:[email protected]>> wrote: > I've found 2 performance regressions in IO. Could someone triage these? > > > https://issues.apache.org/jira/browse/BEAM-12801 > <https://issues.apache.org/jira/browse/BEAM-12801> > > @Alexey Romanenko <mailto:[email protected]> @Tim Robertson > <mailto:[email protected]> - Could you comment on this? It is related > to HadoopFormatIO. >
I don’t see any code changes for “sdks/java/io/hadoop-format” package since June 2nd while the Write performance dropped at Aug 10th for the first time. I’ll try to reproduce it locally with a direct runner but seems it was caused by something else. Any other ideas about this? > > https://issues.apache.org/jira/browse/BEAM-12800 > <https://issues.apache.org/jira/browse/BEAM-12800> > > @Chamikara Jayalath <mailto:[email protected]> @Yifan Mai > <mailto:[email protected]> - Do you know who could comment on this? > > > > +14 other blockers currently > > I have not looked at the other blockers. It might help to check with the > assignees. They may not know that they are release blockers. > > > On Wed, Aug 25, 2021 at 12:22 PM Udi Meiri <[email protected] > <mailto:[email protected]>> wrote: > I'll try cutting the release today. From what I can tell [1], the only > perma-reds are python postcommits. > > I believe this is fixed now. (Thanks to @Kyle Weaver > <mailto:[email protected]>) > > > [1] > https://github.com/apache/beam/blob/master/.github/PULL_REQUEST_TEMPLATE.md > <https://github.com/apache/beam/blob/master/.github/PULL_REQUEST_TEMPLATE.md> > On Tue, Aug 17, 2021 at 10:12 AM Udi Meiri <[email protected] > <mailto:[email protected]>> wrote: > Please mark 2.33.0 release blockers by setting the "Fix version" field to > 2.33.0. Otherwise, it doesn't appear in the dashboard > <https://issues.apache.org/jira/projects/BEAM/versions/12350404>. > > On Tue, Aug 17, 2021 at 9:32 AM Luke Cwik <[email protected] > <mailto:[email protected]>> wrote: > +Rui Wang <mailto:[email protected]> as the current build monitor. > > At the time I did not mean to add any additional tests as it was a copy paste > error. > > I do have another fix for the failing Python PreCommit which I fixed since it > seemed like a trivial one: > https://issues.apache.org/jira/browse/BEAM-12768 > <https://issues.apache.org/jira/browse/BEAM-12768> - > test_df_agg_method_invalid_kwarg_raises too strict on error checking. fix in > https://github.com/apache/beam/pull/15341 > <https://github.com/apache/beam/pull/15341> > > https://issues.apache.org/jira/browse/BEAM-12733 > <https://issues.apache.org/jira/browse/BEAM-12733> (RecommendationAIIT Java > Version) > +Matthias Baetens <mailto:[email protected]> for the above issue. > > On Mon, Aug 16, 2021 at 6:54 PM Ahmet Altay <[email protected] > <mailto:[email protected]>> wrote: > Thank you Udi and +1 to the release cut on schedule. > > On Mon, Aug 16, 2021 at 3:53 PM Luke Cwik <[email protected] > <mailto:[email protected]>> wrote: > It seems as though several tests were broken due to commits during the issue > with some existing flakes. It looks like the Python PostCommit tests are > perma-red due to these failures. So far I'm tracking: > > Thank you very much Luke. Adding bug owners here to explicitly call out > release blocking issues. > > https://issues.apache.org/jira/browse/BEAM-12764 > <https://issues.apache.org/jira/browse/BEAM-12764> (new_block method missing > in pandas) > > +Sam Rohde <mailto:[email protected]> for the above issue. > > https://issues.apache.org/jira/browse/BEAM-12683 > <https://issues.apache.org/jira/browse/BEAM-12683> (RecommendationAIIT) > > +Matthias Baetens <mailto:[email protected]> for the above issue. > > https://issues.apache.org/jira/browse/BEAM-12765 > <https://issues.apache.org/jira/browse/BEAM-12765> (cannot access field fruit) > https://issues.apache.org/jira/browse/BEAM-12766 > <https://issues.apache.org/jira/browse/BEAM-12766> (Already Exists: Dataset > apache-beam-testing:python_bq_file_loads_) > > +Pablo Estrada <mailto:[email protected]> for the two issues above. > > https://issues.apache.org/jira/browse/BEAM-12694 > <https://issues.apache.org/jira/browse/BEAM-12694> (DICOMIoIntegrationTest) > > +Frank Yang <mailto:[email protected]> +Dave Anderson > <mailto:[email protected]> for the DICOMIO. Could one of you look at > this? Looks like it might be a simple fix. > > > https://issues.apache.org/jira/browse/BEAM-12766 > <https://issues.apache.org/jira/browse/BEAM-12766> (BQ Already exists Dataset) > > Luke, did you mean link to another issue? This one is a duplicate. > > > > > > On Mon, Aug 16, 2021 at 10:45 AM Luke Cwik <[email protected] > <mailto:[email protected]>> wrote: > There are the ones that I believe went perma-red due to the issue: > beam_PostCommit_Python36 > beam_PostCommit_Python37 > beam_PostCommit_Python38 > beam_PreCommit_Python_Cron > beam_PostCommit_Java > beam_PostCommit_Java_DataflowV2 > beam_PreCommit_Java_Examples_Dataflow_Cron > beam_PostRelease_NightlySnapshot > > > On Mon, Aug 16, 2021 at 10:42 AM Udi Meiri <[email protected] > <mailto:[email protected]>> wrote: > Sure, I'll verify the status of the tests in our PR template. Any others I > should be aware of? > > On Mon, Aug 16, 2021 at 10:15 AM Andrew Pilloud <[email protected] > <mailto:[email protected]>> wrote: > That sounds reasonable to me. A week should give everyone enough time to get > their changes in. > > On Mon, Aug 16, 2021 at 9:57 AM Luke Cwik <[email protected] > <mailto:[email protected]>> wrote: > The issue impacting Dataflow should be resolved. We can see if the precommits > stabilize by Wednesday. Udi before cutting the first release can you post > what is the status of the testing to the group and if it looks green then we > can go ahead (assuming no blockers)? > > On Fri, Aug 13, 2021 at 5:47 PM Andrew Pilloud <[email protected] > <mailto:[email protected]>> wrote: > Thanks for volunteering! > > I'm going to object to the 2.33.0 release being cut until we stabilize our > precommits. > > Both java and python precommits are reporting the last successful run being > in July. > https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/ > <https://ci-beam.apache.org/job/beam_PreCommit_Python_Cron/> > https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Cron/ > <https://ci-beam.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Cron/> > On Wed, Aug 11, 2021 at 10:36 AM Udi Meiri <[email protected] > <mailto:[email protected]>> wrote: > Hi everyone! > > Beam 2.33.0 release is scheduled to be cut on 2021-08-25 (Wednesday) and > released by 2021-09-29 according to the release calendar [1]. > > If there are no objections then I would like to nominate myself to do this > release. > Please review your bug list and mark the blockers as blocker for the release > and move non blockers to the next release by 2021-08-25. > > Please let me know if you have any questions. > > Thanks, > Udi > > [1] > https://calendar.google.com/calendar/u/0/[email protected]&ctz=America/Los_Angeles > > <https://calendar.google.com/calendar/u/0/[email protected]&ctz=America/Los_Angeles>
