I experimented last week moving the CI config to Cirrus and found several benefits to making that switch. Example build in my Zeek fork:
https://cirrus-ci.com/build/6738081318699008 Notable differences from the Travis config: * Free for open-source projects, can allocate up to 8 cpus (we get 2 cpus on Travis) * Concurrent tasks up to 8 Linux, 2 FreeBSD, and 1 macOS (Travis is 8 total, doesn't offer FreeBSD, and we haven't configured it for macOS). This is also limited per _user_ not per repo, so misc. PRs will no longer backup the CI queue like they do on Travis. E.g. to quote docs: "For example, if you have 10 active contributors to a repository then you can end up with 130 tasks running in parallel for the repository" * I've set up pre-built Dockerfiles for all Linux systems to make it faster to get those up and running rather than wait to download dependencies every time (and potentially encounter transient network issues as we see sometimes on Travis). This also makes it just as easy as Travis to reproduce the CI environment locally (Travis also used docker, but didn't organize things as standard Dockerfiles) * I've set the external pcap files (~850mb) to get cached after the first successful build (or auto-reuploaded if they ever change later) and downloading from the cache takes around 1 minute rather than sometimes 15-20 on Travis * The above differences amount to getting CI results in about 12-30 minutes (macOS and memleak task are still a bit slow) rather than 50-60 minutes on Travis. So my idea going forward (let me know if there's other thoughts): I'll set up the main Zeek GitHub organization/repo to use Cirrus in parallel with existing Travis CI. If it performs well for several months, we can think about completely migrating off Travis. Don't necessarily need to move other projects to it, but might anyway in spare time. Also definitely would need to move the Coverity Scan cron somewhere else. IMO, this doesn't fit in with the CI config, it just needs to run regularly somewhere, so GitHub Actions may be a good fit for that. Other caveats: * I haven't configured Cirrus to access the external private test suite. IMO, we might just stop using that, since it's awkward for an open-source project: we don't give access to private pcaps from arbitrary PRs that trigger CI, but then what happens if the merger finds failures in the private test suite (or forget to run it before merging) ? Either need to give access to the private pcaps or else the core team basically becomes responsible (and I can imagine some new features/PRs being non-trivial to hunt down diffs ourselves). * Haven't figured out how to configure email notifications on Cirrus CI failures, but can figure that out once it's up and running. - Jon _______________________________________________ Zeek-Dev mailing list [email protected] http://mailman.icsi.berkeley.edu/mailman/listinfo/zeek-dev
