Wes McKinney <wesmck...@gmail.com> writes: > I think we should take a more serious look at Buildkite for some of our CI. > > * First of all, it's very easy to connect self-hosted workers and > supports ephemeral cloud workers in a way that would be difficult or > impossible with GHA. No need to have Infra fiddle with the admin > dashboard. So we could spin up extra workers during peak hours, or use > autoscaling to respond to demand. > > * We can set up more complex / dependent job pipelines rather than the > current GHA monolithic "long list of independent jobs" setup. For > example, we could have a fast gatekeeper job for C++ builds (which > lints and makes sure that everything compiles) that must pass before > more exhaustive longer-running jobs run.
I don't have experience with Buildkite, but note that gitlab-runner is also lightweight and well-featured as above. Here's an example with gatekeeping stages across about 60 environments (mostly on-prem at multiple sites), including explicit "pause-for-approval" to avoid unnecessary time-consuming jobs. https://gitlab.com/petsc/petsc/-/pipelines/286655535 We also use it for on-prem GPU-equipped CI with repositories hosted on GitHub, reporting status to PRs. The Kubernetes and docker-machine executors are intended for autoscaling. https://docs.gitlab.com/runner/executors/README.html
