bq. We start demoting tests to flaky list which never get fixed. I think this may happen. Jenkins jobs for 1.x builds are relatively stable - meaning, we would have either green Java 7 build or green Java 8 build (if not both). Broken test(s) would be quickly identified and fixed.
I don't think we need dual builds for 1.x branches. On Tue, Mar 29, 2016 at 4:25 PM, Apekshit Sharma <[email protected]> wrote: > Proposal: > Maintain a list of flaky tests which can be ignored in main builds (for > patches) and setup a new job which will periodically run those flaky tests. > > Benefits: > - Cleaner main builds. Less runs per patch, ideally no re-runs because of > timeouts and other flakiness. Less frustration. Increased developer > productivity. > - Logs from various runs available when anyone tries to fix these tests. > > Possible con: We start demoting tests to flaky list which never get fixed. > > How? > Internally, we have setup up two jenkins jobs, one for main build and one > for flaky. In execute shell step, they curl to get the list of flaky tests > and use surefire plugin flags to ignore/run particular tests. > I volunteer to set them up. However, not sure if this approach will work > since we have yetus upstream. > > - Appy >
