GitHub user pjfanning edited a discussion: Replace sbt/setup-sbt in CI:
Coursier or our own install step?
Our CI installs sbt with `sbt/setup-sbt`, and its signature verification step
has now broken builds several times. I'd like to discuss replacing it, either
with `coursier/setup-action` or with a small install step of our own.
## What happened this time
On 2026-11-06, jobs in #3751 started failing in `Install sbt`, before sbt even
started:
```
Error: compiling policy: unable to parse a policy, group or policySet from data
```
We pin `sbt/setup-sbt` to a commit SHA, but v1.5.13 fetches its verification
policy from the default branch of `carabiner-dev/policies`. That policy added a
`meta.name` field which, according to the policy repo's commit messages, ampel
versions before v1.3.10 can't parse. setup-sbt uses ampel v1.3.6.
Three things made this worse:
- **Pinning didn't protect us.** The policy wasn't pinned, so a change in
another repo broke our CI with no change on our side.
- **It only shows up on a cache miss.** Verification is skipped when the sbt
download is cached, so builds kept passing for hours after the policy changed.
The `sbt-runner` cache entry later disappeared, probably through eviction, and
jobs started failing.
- **Re-running doesn't help.** The failing job doesn't save the cache, so
nothing restores it. The latest `Nightly Multi JVM Tests` run on `main` failed
with the same error. setup-sbt passes `fail: false` ("monitor only"), but that
doesn't stop this failure.
setup-sbt v1.5.14 already fixes this by pinning the policy, and it was released
before the breaking policy commit landed. We were still broken because
Dependabot only checks weekly. So we need that update regardless of the outcome
here. This discussion is about the longer term.
## Earlier problems
All of these involve the same verification step:
- sbt/setup-sbt#118 (July 2026): `ampel: command not found` on a cache miss,
affecting several releases. The cause was a bug in `carabiner-dev/actions`
(carabiner-dev/actions#97), and setup-sbt releases still pinned a version from
before the fix.
- sbt/setup-sbt#142 (September 2026): v1.5.10 pinned a `carabiner-dev/actions`
version that failed to load on a self-hosted runner.
- sbt/setup-sbt#155 (November 2026): the fix for this latest problem.
The maintainers have fixed these quickly, and verifying the sbt download is a
good goal. The concern is the size of the dependency chain for downloading one
zip file. An install step now pulls in `carabiner-dev/actions`, an ampel binary
and a policy fetched at runtime. Any of those can stop every job in our CI, and
other Pekko repos such as `pekko-http`, `pekko-connectors`, `pekko-management`
and `pekko-grpc` use the same action.
## Option 1: use `coursier/setup-action`
We already use this action in `format.yml` and `link-validator.yml`, it hasn't
been a big problem for us, and it has no ampel or carabiner dependency. Its
README shows installing sbt directly:
```yaml
- uses: coursier/setup-action@<sha> # pin as usual
with:
apps: sbt
```
Benefits:
- **Already familiar:** it's in use here, and `pekko-http`, `pekko-connectors`,
`pekko-management` and `pekko-grpc` each show a reference to it too.
- **Simpler chain:** it's a single Node action with no nested actions, so
there's no ampel binary or runtime policy fetch.
- **Dependabot still works:** updates keep coming as normal action bumps, so
there's no script to maintain.
- **Easier on version branches:** the ASF allow list has a `'*'` entry with
`keep: true` for this action, which as I read it means older pins won't expire.
`sbt/setup-sbt` versions get an `expires_at` date, which adds to the backport
churn on branches like 1.x.
Costs and things to check:
- **Still a third-party action:** we'd still depend on someone else's release
process.
- **Verification is narrower:** v3.1.0 adds an optional `requireChecksum`
input, which checks the Coursier launcher against a SHA-256 pinned in the
action, but only for the action's default Coursier version. We pin v3.0.4,
which doesn't have it. It covers the launcher rather than the apps it installs.
- **sbt version control:** the app comes from the Coursier app channels, so
we'd need to check how the sbt runner version is pinned and how that download
is verified.
- **Not yet tried for this:** our current uses install scalafmt and the default
`sbtn`, not `sbt`, so it would need testing on our sbt jobs.
## Option 2: our own install step with curl
Add a small local composite action, such as
`.github/actions/setup-sbt/action.yml`, that downloads a fixed sbt runner
version and checks it against a SHA-256 committed in our repo. sbt publishes
`.zip`, `.asc` and `.sha256` files with each release. The runner is only a
launcher, and `project/build.properties` still selects the sbt version used for
the build.
An untested sketch for Linux runners:
```yaml
name: Install sbt
description: Install a pinned sbt runner and verify its checksum
runs:
using: composite
steps:
- shell: bash
env:
SBT_VERSION: "<version>"
SBT_SHA256: "<checksum committed here>"
run: |
zip="$RUNNER_TEMP/sbt-$SBT_VERSION.zip"
curl -fsSL --retry 5 -o "$zip" \
"https://github.com/sbt/sbt/releases/download/v$SBT_VERSION/sbt-$SBT_VERSION.zip";
echo "$SBT_SHA256 $zip" | sha256sum --check --strict
unzip -q "$zip" -d "$RUNNER_TEMP/sbt-$SBT_VERSION"
echo "$RUNNER_TEMP/sbt-$SBT_VERSION/sbt/bin" >> "$GITHUB_PATH"
```
Benefits:
- **Stronger failure behaviour:** a mismatch always fails the job, and it runs
on every build rather than only on a cache miss.
- **Less to break:** no runtime policy fetch and no extra actions or binaries.
- **One place to update:** we have 62 `sbt/setup-sbt` references across 39
workflow files on `main`, which could all point at one action.
Costs:
- **We own it.** Someone has to maintain the action, and Dependabot won't bump
a version and checksum held in a script. We'd need a manual update or another
way to automate it.
- **Different trust model.** A committed checksum pins exactly what we reviewed
when we updated it. It doesn't check the sbt team's signature, so the update
itself has to confirm the new file is genuine, for example by checking the
`.asc` at that point.
- **Download reliability.** Without the action's cache, each job downloads from
GitHub releases, so we may want to keep using `actions/cache` or accept the
extra requests.
- **Checkout order.** A local action can only be used after `actions/checkout`,
so any workflow that installs sbt first would need reordering.
- **Sharing across repos.** Other Pekko repos would either copy the action or
reference it from `apache/pekko` by SHA.
## Other alternatives
1. **Keep `sbt/setup-sbt` and update faster.** This is the least work, but it
keeps the dependency chain, and a cold cache can still turn an upstream problem
into a repo-wide failure.
2. **Ask upstream for an option to turn verification off, or make it truly
non-fatal.** `fail: false` already suggests this is meant to be monitor only,
so it could help everyone using the action. We could do this alongside any
other option.
## Questions
- Which is the better fit: Coursier, our own curl step, or staying with
`sbt/setup-sbt`?
- How much verification of the sbt download do we need? For the curl step, is a
committed checksum enough, or do we also want a GPG check of the `.asc` in CI?
- Should this cover just `apache/pekko` first, or be shared across the Pekko
repos from the start?
GitHub link: https://github.com/apache/pekko/discussions/3753
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
