We could check with ASF infra for the current state of things wrt GitHub actions. I believe there is a queue set up across ASF projects.
It has the same resource issue Travis had; things are fine until some critical mass of projects seeking better perf realize some new option is available and then quickly all available resources are consumed. AFAICT the only option that gets us the same or better as the H* nodes will be finding sponsors and running our own. On Mon, Jul 20, 2020, 21:55 张铎(Duo Zhang) <[email protected]> wrote: > I think our nightly, flakey, and pre commit jobs should be transferred as a > whole? They depend on each other. > > I offer my help on the transition. > > And on github CI, does ASF have a special deal with github? If not, I do > not think the default resource can fit our requirements... > > > > Sean Busbey <[email protected]> 于2020年7月21日周二 上午1:49写道： > > > Hi folks! > > > > Back in April there was a brief discussion[1] about ASF Infra's > > notification that builds.a.o is going away and we are currently slated > > to migrate to a set of CI servers for "Hadoop and related projects". > > This is the ci farm that will contain the bulk of the H* worker nodes > > that are donated by Yahoo!, which are the nodes we've been running on > > for ages[2]. > > > > Migration discussion still happens on the [email protected] > > list[3] and recently ASF Infra set a target date of August 15th for > > turning off the existing builds.a.o server[4]. > > > > That gives us a little under 4 weeks to have things up and working on > > the new ci-hadoop.a.o jenkins coordinator[5]. it’s not clear to me > > that the level of effort we’ll need to spend is worth what we get out > > of a continuation of the status quo on builds.a.o. I did a quick test > > by updating the nightly job on ci-hadoop.a.o to run just branch-2, > > since that has been stable on builds.a.o. It failed with a Jenkins > > pipeline DSL syntax error[6] so I'm assuming migrating will be a slog. > > > > As far as I can see our options are: > > > > * Do nothing. Have no testing or automated website publication in mid > > August. > > * Transition website publication and nothing else (probably can be > > done in a day) > > * Transition just precommit testing for various repos (probably can be > > done in a few days) > > * Transition everything (no idea how long it takes due to nightly, > > flaky stuff, etc) > > > > The alternatives if we do not transition any given job to ci-hadoop: > > > > * Try to move to GitHub Actions > > * Try to move to Travis CI > > * Try to move to Jenkins infra we maintain ourselves (presumably by > > soliciting project specific donations for worker nodes on cloud > > vendors) > > > > It's important to remember that as a project we have a heavy footprint > > wherever our nightly tests run. For context, a given branch's nightly > > can keep 3-4 executors busy for 6+ hours on the current builds.a.o > > setup. There's been a bunch of great work lately on bringing down what > > it takes to run the full test suite, but applying that work to nightly > > is itself a significant undertaking. > > > > What are folks thinking? Most importantly who is ready to work towards > > any given approach? > > > > [1] [DISCUSS] Migrating HBase to new CI Master > > https://s.apache.org/fux1o > > > > [2] https://builds.apache.org/view/H-L/view/HBase/ > > > > [3] > https://lists.apache.org/[email protected] > > > > [4] [IMPORTANT] - 2 more HADOOP nodes migrated over to ci-hadoop > > https://s.apache.org/7e1nq > > > > [5] https://ci-hadoop.apache.org/job/HBase/ > > > > [6] > > > https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/2/console > > >
