Hi all, A quick note on CI usage, since it affects everyone's build times.
ASF projects share a CI budget, and a single Pulsar CI build uses roughly 10 hours of GitHub Runner VM time. Once we hit an invisible usage limit, builds get throttled for everyone: they don't start immediately and are queued instead. Since the throttling is invisible, other ASF projects may also be spending the shared budget, so our usage isn't the only cause. We just have to make sure we don't cause overusage. A few things that help: - Keep WIP work in your fork's "Personal CI" and open the PR to apache/pulsar only when it's ready for review. - If you open a PR early, make it a draft in apache/pulsar to prevent CI from running and get CI feedback on your own fork for the same PR branch in Personal CI. - Open new PRs after your previous ones have been merged, so there are fewer WIP PRs open at once. - Prioritize your PRs so the most impactful ones get merged first. When overall activity is lower, that's a good time to push more. - Address all open review comments at once before pushing. Separate commits in the PR branch are fine, but pushing them together triggers fewer builds. - Avoid leaving PRs open for a long time. They need origin/master merged in to make sure merging doesn't introduce regressions, and that triggers another full build, which is another 10 hours of VM time. For committers: please don't approve CI runs for contributions from external contributors at every step or early in the review process. Ask them to use "Personal CI" to get CI feedback instead: https://pulsar.apache.org/contribute/personal-ci/ Reducing build times is the most lasting way to cut spending. The test time reduction PRs by void-ptr974 will help a lot with this, so please contribute to the effort by prioritizing those PRs and considering improvements in this area yourself. There are other ideas worth exploring too: - Move some tests out of the PR CI pipeline to nightly runs, since most regressions are caught in earlier phases. - Add a smarter way to determine whether integration tests need to run at all for a given change. Existing OSS tooling might be available for this. Good ideas and contributions are more than welcome! Thanks, everyone, for helping keep CI healthy! Best regards, Lari
