andygrove opened a new pull request, #2724: URL: https://github.com/apache/datafusion-ballista/pull/2724
# Which issue does this PR close? No issue yet. This proposes a release policy, so it should get comments from committers and PMC members before it merges. # Rationale for this change Ballista has released once per DataFusion major release since 43.0.0, plus one fix-only release (54.1.0). The 12 gaps between major releases average 52 days, and the longest is 83. Four weeks after 55.0.0, `main` holds 19 `feat`, 16 `fix` and 2 `perf` commits that users cannot get until 56.0.0. Nothing in the repository says what a minor or patch release may contain, beyond "cherry-pick commits as needed" in the release guide. The major version has to keep following DataFusion's: the `ballista` crate re-exports `datafusion` and returns its types, and plans cross the wire in datafusion-proto's encoding (see `ballista/core/src/version.rs`). So this PR keeps the numbering and writes down the rules for releasing more often from the release branch instead. # What changes are included in this PR? - New user guide page, **Versioning and Compatibility**: what major, minor and patch releases contain, which components work together within a major version, and what can and cannot go into a minor release. - `dev/release/README.md`: the steps for releasing from an existing release branch, based on how 54.1.0 was made. - `development.md`: a **Backporting to a release branch** section. - `.github/pull_request_template.md`: the label is `api-change`, not `api change`. - Links from the docs toctree and the upgrade guide index. Docs only, no code changes. **Points for reviewers** 1. **Features go in a minor release, not a patch release.** `55.1.0` carries features and fixes, and `55.1.1` is fixes only. This follows SemVer and what 54.1.0 already did. Cargo treats both the same for `^55`. 2. **The executor and scheduler protocol is frozen within a major version.** `BALLISTA_PROTOCOL_VERSION` does not change on a release branch, so any `55.x` scheduler works with any `55.y` executor and minor upgrades can be rolling. The cost is that features that change the wire format, such as the pipelined exchange, wait for the next major. Today `version.rs` says the value "can change in any release". 3. **Config structs may gain fields in a minor release.** `SchedulerConfig` and `ExecutorProcessConfig` implement `Default`, so adding a field is not treated as breaking, provided users build them with `..Default::default()`. The alternative, `#[non_exhaustive]`, forbids struct-update syntax outside the crate. Without an exemption, most features that add configuration would wait for a major version. 4. **"Breaking" means "needs an upgrade guide entry"**, which is the rule `development.md` already uses, rather than a narrower definition of the stable API. Narrowing it, for example by marking internals `#[doc(hidden)]`, would make more features eligible and could be a follow-up. 5. **Only the newest release branch is maintained.** **Not included, to follow once the policy is agreed** - Updating the rustdoc on `BALLISTA_PROTOCOL_VERSION` and `ballista_core::version`, which still say it can change in any release, so that it matches whatever is agreed here. - CI checks such as `cargo-semver-checks` and a protocol version check on PRs to `branch-*`. Until then the rules are enforced by review and the `api-change` label. - Backport tooling. A textual dry run (no build) of cherry-picking the `feat`, `fix` and `perf` commits merged since the 55 branch cut onto `branch-55` applied 19 of the 32 that do not carry `api-change` and conflicted on 13, after four weeks. That is why the guide asks for backports when a change merges. # Are there any user-facing changes? Documentation only. The new page is added to the Reference section of the docs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
