[Python SDK] Use pre-released dependencies for Beam python unit testing

Hi all, For Apache Beam Python we are considering using pre-released dependencies for unit testing by using the --pre flag to install pre-released dependencies of packages. We believe that using pre-released dependencies may help us to identify and resolve bugs more quickly, and to take

Re: Jenkins Flakes

This seems to have fixed the problem, please let me know if you see any further issues. On Tue, Apr 11, 2023 at 3:51 PM Danny McCormick wrote: > I went ahead and made the limit 40 runs on the following jobs (PR > ): > > beam_PostCommit_Go_VR_Flink >

Re: Jenkins Flakes

I went ahead and made the limit 40 runs on the following jobs (PR ): beam_PostCommit_Go_VR_Flink beam_PostCommit_Java_Nexmark_Flink beam_PostCommit_Python_Examples_Flink beam_PreCommit_Java_* beam_PreCommit_Python_* beam_PreCommit_SQL_* It doesn't quite

Re: Jenkins Flakes

> Regarding the "(and not guaranteed to work)" part, is the resolution that the memory issues may still persist and we restore the normal retention limit (and we look for another fix), or that we never restore back to the normal retention limit? Mostly, I'm just not 100% certain that this is the

Re: Jenkins Flakes

Is there a technical reason we can't migrate Java code coverage over to the Codecov tool/Actions like we have with Go and Python? On Tue, Apr 11, 2023 at 11:25 AM Moritz Mack wrote: > Yes, sorry Robert for being so unspecific. With everywhere I meant Java > only, my bad! > > > > On 11.04.23,

Re: Jenkins Flakes

Yes, sorry Robert for being so unspecific. With everywhere I meant Java only, my bad! On 11.04.23, 17:17, "Robert Burke" wrote: The coverage issue is only with the Java builds in specific. Go abd Python have their coverage numbers codecov uploads done in GitHub Actions instead. On Tue, Apr

Re: Jenkins Flakes

The coverage issue is only with the Java builds in specific. Go abd Python have their coverage numbers codecov uploads done in GitHub Actions instead. On Tue, Apr 11, 2023, 8:14 AM Moritz Mack wrote: > Thanks so much for looking into this! > > I’m absolutely +1 for removing Jenkins related

Re: Jenkins Flakes

Thanks so much for looking into this! I’m absolutely +1 for removing Jenkins related friction and the proposed changes sound legitimate. Also, considering the number of flaky tests in general [1], code coverage might not be the pressing issue. Should it be disabled everywhere in favor of more

Re: Jenkins Flakes

+1 SGTM Remember, if an issue is being investigated, a committer can always mark a build to be retained longer in the Jenkins UI. Just be sure to clean it up once it's resolved though. (TBH there may also be some old retained builds like that, but I doubt there's a good way to see which are

Re: Jenkins Flakes

+1 Thanks Danny for figuring out a solution. Best, Yi On Tue, Apr 11, 2023 at 10:56 AM Svetak Sundhar via dev wrote: > +1 to the proposal. > > Regarding the "(and not guaranteed to work)" part, is the resolution that > the memory issues may still persist and we restore the normal retention >

Re: Jenkins Flakes

+1 to the proposal. Regarding the "(and not guaranteed to work)" part, is the resolution that the memory issues may still persist and we restore the normal retention limit (and we look for another fix), or that we never restore back to the normal retention limit? Svetak Sundhar Technical

Re: Jenkins Flakes

+1 for getting Jenkins back into a happier state, getting release blockers resolved ahead of building an RC has been severely hindered by Jenkins not picking up tests or running them properly. On Tue, Apr 11, 2023 at 10:24 AM Danny McCormick via dev < dev@beam.apache.org> wrote: > *;tldr - I

Jenkins Flakes

*;tldr - I want to temporarily reduce the number of builds that we retain to reduce pressure on Jenkins* Hey everyone, over the past few days our Jenkins runs have been particularly flaky across the board, with errors like the following showing up all over the place [1]:

Beam High Priority Issue Report (26)

This is your daily summary of Beam's current high priority issues that may need attention. See https://beam.apache.org/contribute/issue-priorities for the meaning and expectations around issue priorities. Unassigned P1 Issues: https://github.com/apache/beam/issues/26126 [Failing Test]: