I think our nightly, flakey, and pre commit jobs should be transferred as a whole? They depend on each other.
I offer my help on the transition. And on github CI, does ASF have a special deal with github? If not, I do not think the default resource can fit our requirements... Sean Busbey <[email protected]> 于2020年7月21日周二 上午1:49写道： > Hi folks! > > Back in April there was a brief discussion[1] about ASF Infra's > notification that builds.a.o is going away and we are currently slated > to migrate to a set of CI servers for "Hadoop and related projects". > This is the ci farm that will contain the bulk of the H* worker nodes > that are donated by Yahoo!, which are the nodes we've been running on > for ages[2]. > > Migration discussion still happens on the [email protected] > list[3] and recently ASF Infra set a target date of August 15th for > turning off the existing builds.a.o server[4]. > > That gives us a little under 4 weeks to have things up and working on > the new ci-hadoop.a.o jenkins coordinator[5]. it’s not clear to me > that the level of effort we’ll need to spend is worth what we get out > of a continuation of the status quo on builds.a.o. I did a quick test > by updating the nightly job on ci-hadoop.a.o to run just branch-2, > since that has been stable on builds.a.o. It failed with a Jenkins > pipeline DSL syntax error[6] so I'm assuming migrating will be a slog. > > As far as I can see our options are: > > * Do nothing. Have no testing or automated website publication in mid > August. > * Transition website publication and nothing else (probably can be > done in a day) > * Transition just precommit testing for various repos (probably can be > done in a few days) > * Transition everything (no idea how long it takes due to nightly, > flaky stuff, etc) > > The alternatives if we do not transition any given job to ci-hadoop: > > * Try to move to GitHub Actions > * Try to move to Travis CI > * Try to move to Jenkins infra we maintain ourselves (presumably by > soliciting project specific donations for worker nodes on cloud > vendors) > > It's important to remember that as a project we have a heavy footprint > wherever our nightly tests run. For context, a given branch's nightly > can keep 3-4 executors busy for 6+ hours on the current builds.a.o > setup. There's been a bunch of great work lately on bringing down what > it takes to run the full test suite, but applying that work to nightly > is itself a significant undertaking. > > What are folks thinking? Most importantly who is ready to work towards > any given approach? > > [1] [DISCUSS] Migrating HBase to new CI Master > https://s.apache.org/fux1o > > [2] https://builds.apache.org/view/H-L/view/HBase/ > > [3] https://lists.apache.org/[email protected] > > [4] [IMPORTANT] - 2 more HADOOP nodes migrated over to ci-hadoop > https://s.apache.org/7e1nq > > [5] https://ci-hadoop.apache.org/job/HBase/ > > [6] > https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/2/console >
