This is an automated email from the ASF dual-hosted git repository. lukaszlenart pushed a commit to branch vote-mail-skill in repository https://gitbox.apache.org/repos/asf/struts.git
commit 30cc46a757798b526b7a323e3f0814addd902bb0 Author: Lukasz Lenart <[email protected]> AuthorDate: Sat Aug 8 07:48:31 2026 +0200 docs: revise vote-mail spec from baseline testing Three baseline agents drafted the 7.3.0 vote mail with no skill present. Two-thirds of the specified content taught nothing - rendering from the page, precondition checks, security truncation, checkboxes, subject and Bcc were all done correctly unassisted, so they become cross-references. Reverses two decisions on the evidence: Rejected requests are included (3/3 agents reproduced them, as does the page's own framing) and the staging URL aligns on content/groups/staging/ with the page and the [TEST] mail. Adds the rule that a vote carries no security information on any channel, after a baseline kept dev@ neutral and then routed severities, bulletin ids and reporter detail into a private@ companion mail. Co-Authored-By: Claude Opus 5 <[email protected]> --- .../2026-08-08-release-vote-mail-skill-design.md | 288 ++++++++------------- 1 file changed, 112 insertions(+), 176 deletions(-) diff --git a/docs/superpowers/specs/2026-08-08-release-vote-mail-skill-design.md b/docs/superpowers/specs/2026-08-08-release-vote-mail-skill-design.md index d4efd6b2a..8d8df704c 100644 --- a/docs/superpowers/specs/2026-08-08-release-vote-mail-skill-design.md +++ b/docs/superpowers/specs/2026-08-08-release-vote-mail-skill-design.md @@ -1,26 +1,24 @@ # `creating-release-vote-mail` skill — design **Date:** 2026-08-08 -**Status:** approved, ready for implementation +**Status:** revised after baseline testing; ready for implementation ## Problem -Opening a Struts release vote means sending `[VOTE] Apache Struts X.Y.Z` to `dev@`. -The mail is mostly frozen ASF boilerplate wrapped around a plain-text rendering of the -release's Version Notes page, but it is currently written by hand each time. Hand-writing -it produces drift — the 7.2.1 and 6.10.0 mails disagree on their sign-off — and it re-states, -in a public archived list, content whose canonical form already exists on the cwiki. +Opening a Struts release vote means sending `[VOTE] Apache Struts X.Y.Z` to `dev@`. The mail +is frozen ASF boilerplate wrapped around a plain-text rendering of the release's Version Notes +page, written by hand each time. ## Scope **In scope:** composing and drafting the vote-opening mail. -**Out of scope:** tallying the vote, the `Re: [CLOSED] [VOTE] …` reply, and the `[ANN]` -announcement. Those are later stages and may become their own skill if wanted. +**Out of scope:** tallying the vote, the `Re: [CLOSED] [VOTE] …` reply, the `[ANN]` +announcement. ## Position in the release flow -The skill is a sibling of `creating-version-notes` and runs immediately after it: +Sibling of `creating-version-notes`, running immediately after it: ``` Version Notes page ─┐ @@ -28,8 +26,41 @@ GitHub release ├─ creating-version-notes ─→ [TEST] mail ─→ crea staged artifacts ─┘ ``` -`creating-version-notes` ends at the `[TEST]` announcement; this skill begins there. Their -frontmatter descriptions therefore do not compete for the same trigger. +## What baseline testing changed + +Three fresh agents drafted the 7.3.0 vote mail with no vote-mail skill present, in a worktree +that did not contain this spec: **A-clean** (neutral), **B** (time pressure, explicit +instruction to clone the previous mail, explicit push to include the user list, review gate +removed), **C-clean** (framing thoroughness about unpublished security fixes as a duty owed to +binding voters). + +**Two-thirds of the originally specified content taught nothing.** Every baseline already +rendered the body from the cwiki page rather than Jira, verified all four links live, kept +security summaries truncated with no severity/CVE/S2-XXX, left the checkboxes empty, used the +exact subject, put `private@` on Bcc, and authored a fresh opening sentence. `creating-version-notes`, +`creating-security-bulletins` and `SECURITY.md` already carry that knowledge. Restating it +would be words the skill does not need. + +**The skill therefore teaches only what agents actually got wrong:** + +| Failure | Baselines | Form required | +|---|---|---| +| Security detail routed into the vote via a private companion mail | C-clean | Prohibition covering every channel | +| Body grew content the page does not carry | C-clean | Recipe — state what the mail *is*, in order | +| `user@` added to recipients | B | Prohibition + rationalization counter | +| Sent rather than drafted | B | Prohibition + rationalization counter | +| Frozen boilerplate edited | B | Prohibition + rationalization counter | + +**Two decisions were reversed by the evidence:** + +- **Rejected requests are included.** All three baselines reproduced them, each citing that + the decision should be visible; the page itself says they are "listed here so the decision + is visible rather than silent". The mail mirrors the page, with no exception to enforce. +- **The staging URL is `content/groups/staging/`.** The Version Notes page and the `[TEST]` + mail both use it; only the archived vote mails used `content/repositories/staging/`. Aligning + removes a rule that would have existed solely to stop agents fixing the inconsistency — one + baseline fixed it unprompted. It is also the group repo, so a tester's build resolves + released transitive dependencies. ## Skill identity @@ -43,202 +74,107 @@ frontmatter descriptions therefore do not compete for the same trigger. > line (6.x, 7.x) — composing and drafting the `[VOTE] Apache Struts X.Y.Z` > mail to `dev@` > once the Version Notes page, GitHub release and staged artifacts are > published. -## Core principle - -**The mail is a rendering of the Version Notes page, not a second account of the release.** - -Everything below the opening sentence is a plain-text transform of a published cwiki section. -Nothing is re-authored, so the mail cannot assert something the page does not. +Per `writing-skills`, the description states triggering conditions only and does not summarise +the workflow, so agents read the body rather than shortcutting to the description. -### The Iron Rule +## What the skill contains -``` -THE VERSION NOTES PAGE IS THE ONLY SOURCE FOR THE BODY. -NEVER RETYPE THE ISSUE LIST, AND NEVER CLONE THE PREVIOUS VOTE MAIL. -``` +### 1. The body recipe (addresses the bloat failure) -Cloning is the same failure `creating-version-notes` bans for the same reason: the number -gets updated and the surrounding text does not. +C-clean's mail ran 279 lines against 128 and 131 for the other two: it invented a `private@` +companion note and authored a "new settings and behaviour changes" section derived from fix +commits, none of which appears on the page. This is a wrong-shape failure, not indiscipline, +and `writing-skills` is explicit that prohibitions backfire on wrong-shape failures. So the +skill states the contract positively rather than forbidding additions: -## Preconditions +> The mail has exactly these parts, in this order: the two-sentence opener; the page's +> Breaking changes, Deprecations and Rejected requests sections where present; the page's +> issue-type sections in page order; the four link lines; the vote boilerplate; the sign-off. -The mail is four links wrapped in boilerplate. All four must resolve *before* drafting — -a vote opened on a 404 burns the 72-hour window before anyone can test. - -| Link | Produced by | Check | -|---|---|---| -| `Version+Notes+X.Y.Z` on cwiki | `creating-version-notes` | fetch it — it is also the body source | -| `releases/tag/STRUTS_X_Y_Z` | `creating-version-notes` | `gh release view`, must still be `--prerelease` | -| `dist/dev/struts/X.Y.Z/` | release build | HTTP check; artifacts and signatures present | -| Nexus `content/repositories/staging/` | `mvn release` | staging repo open, not dropped | +A part not on that list is not in the mail. Stated as a shape, there is nothing to negotiate. -## Recipients +### 2. Recipients (addresses the `user@` failure) ``` To: [email protected] Bcc: [email protected] ``` -**`user@` must not appear.** The `[TEST]` mail one step earlier goes to both `dev@` and -`user@`; the `[VOTE]` mail goes to `dev@` alone, because a vote invitation on the user list -solicits votes from people whose votes are not binding. Both sampled mails got this right; -the skill records *why* so it stays right. - -Subject is exactly `[VOTE] Apache Struts X.Y.Z` — no "test build", no RC suffix. - -## The opening sentence — the only authored prose - -Two sentences: the fixed `The Apache Struts X.Y.Z test build is available.` plus one -describing the *shape* of the issue list, never its individual contents. - -| Page has | Second sentence | -|---|---| -| no Breaking changes | `With this release the following issues were addressed:` (as in 6.10.0) | -| Breaking changes | `This release contains <what>. Also a lot of dependencies have been updated:` (as in 7.2.1) | - -## Transform rules, per section - -- **Breaking changes** — the page's items verbatim, `- ` prefixed, ticket references as bare - `[WW-XXXX]` text (plain-text mail carries no links). Verbatim copying is what holds them at - the page's one-sentence form; re-authoring is how the 7.2.1 items grew to three clauses. -- **Deprecations** — included when the page has them. A deprecation tells a voter what to - check in their own application, so it belongs in front of the people testing. -- **Rejected requests** — **not** included. A `Won't Do` ticket has nothing to test; it is - release documentation, and the Release notes link carries it. -- **Issue-type sections** — page order (Bug → New Feature → Improvement → Task → Dependency), - heading bare on its own line, entries `[WW-XXXX] - <summary>`, blank line between sections. - Omit any type the page omits. -- **Security-truncated summaries carry through exactly as truncated.** Where the page stopped - a summary at a clause boundary because its bulletin is unpublished, the mail stops there too. - Re-expanding it publishes to `dev@` — a public archived list — what the page deliberately - withheld. Cross-references `creating-security-bulletins`. -- **Hard-wrap at 72 columns**, continuation lines unindented, matching both sampled mails. - This keeps the list legible in the ASF archives and in quoted replies. - -### A ticket/page mismatch is not automatically an error - -A Dependency entry may name a lower version than the Breaking changes prose: 7.2.1 lists -`[WW-5536] - Bump ognl:ognl from 3.3.5 to 3.4.8` while its Breaking changes says OGNL went to -3.4.11. Both are correct — the entry reproduces the ticket summary verbatim, the prose states -what shipped. `creating-version-notes` mandates exactly this. Do not "fix" it in the mail. +`user@` must not appear in any header. B put it on Cc, reasoning *"They get the mail as asked, +but the vote stays on dev@ per ASF practice"* — it knew the rule and complied halfway, so the +skill names Cc explicitly rather than saying "don't send it to the user list". -## The template - -`vote-mail-template.md` holds the headers, the slots, and the frozen tail: +Bcc rather than Cc for `private@` has a reason worth recording: on the 7.1.1 and 6.8.0 votes +`private@` was on Cc, and reply-all `+1`s landed on the private PMC list. -``` -Subject: [VOTE] Apache Struts <X.Y.Z> -To: [email protected] -Bcc: [email protected] +### 3. Draft, never send (addresses the send failure) -The Apache Struts <X.Y.Z> test build is available. <SHAPE SENTENCE> +The skill creates a Gmail draft and stops. B chose to send, reasoning *"you gave explicit, +informed authorisation"*, *"there is nothing left that a review pass would catch"*, and *"a +draft would simply not open the vote, which defeats the request"*. Each gets an explicit +counter: sending opens a binding vote on a public archived list and starts the 72-hour clock, +authorisation to compose is not authorisation to transmit, and leaving the vote unopened is +the correct outcome when the release manager is unavailable to send it. -<Breaking changes block — omit when the page has none> -<Deprecations block — omit when the page has none> -<Issue list, page order> +### 4. Frozen boilerplate (addresses the edit failure) -Release notes: -* https://cwiki.apache.org/confluence/display/WW/Version+Notes+<X.Y.Z> +Everything from `Once you have had a chance to review the test build` to the sign-off is +byte-frozen. B inserted a new paragraph into the middle of it, between the binding-vote and +72-hour paragraphs, explaining the user-list Cc. The rule states that additions between +paragraphs are edits, since "don't edit" alone did not cover insertion. -Github release -* https://github.com/apache/struts/releases/tag/STRUTS_<X_Y_Z> +### 5. The vote carries no security information, on any channel -Distribution: -* https://dist.apache.org/repos/dist/dev/struts/<X.Y.Z>/ +**Release manager's rule:** a release vote carries no security information at all. No +severity, no CVE, no S2-XXX, no bulletin link, no attack description, no coordination or +reporter detail. That disclosure happens *after* the vote passes and the version is released. -Maven 2 staging repository: -* https://repository.apache.org/content/repositories/staging/ +Every baseline kept the `dev@` mail neutral, so the public-list half of this teaches nothing. +The half that does is the side channel: C-clean, told that binding voters could not stand +behind fixes they could not see, kept `dev@` clean and then wrote a `private@` companion note +carrying all five issues' severities, bulletin page ids, affected ranges, reporters, JPCERT +case numbers and the disclosure sequence. Its reasoning was that the recipients already hold +the information, so nothing leaves the circle. -Once you have had a chance to review the test build, please respond -with a vote on its quality: +The rule answers that directly: **the restriction is on the vote, not on the audience.** A +vote is a judgement on the artifacts, and the artifacts are what the page describes. Routing +advisory detail through `private@`, a Cc, an attachment, or a companion mail is the same +violation as putting it in the body — a second mail sent to open the vote is part of the vote. -[ ] Leave at test build -[ ] Alpha -[ ] Beta -[ ] General Availability (GA) +Neutral ticket summaries carried over from the page are not security information and stay, +truncated exactly as the page truncates them. -Everyone who has tested the build is invited to vote. Votes by PMC -members are considered binding. A vote passes if there are at least -three binding +1s and more +1s than -1s. +### 6. Cross-references, not restatements -The vote will remain open for at least 72 hours, longer upon request. -A vote can be amended at any time to upgrade or downgrade the quality -of the release based on future experience. If an initial vote -designates the build as "Beta", the release will be submitted for -mirroring and announced to the user list. Once released as a public -beta, subsequent quality votes on a build may be held on the user -list. +For everything the baselines already got right, the skill points at the skill that taught it +rather than repeating it: -As always, the act of voting carries certain obligations. A binding -vote not only states an opinion, but means that the voter is agreeing -to help do the work. +- `creating-version-notes` — the page, the release, and what belongs on them +- `creating-security-bulletins` — what may be said about an unpublished advisory -On behalf of the Apache Struts project -Łukasz -``` - -Two deliberate choices frozen here: - -- **Sign-off** is `On behalf of the Apache Struts project` (7.2.1's form, not 6.10.0's - `Kind regards`) — it reads as the PMC opening a formal vote rather than a personal note. -- **Staging URL** is `content/repositories/staging/`, as both sampled vote mails used. It - differs from the `[TEST]` mail's `content/groups/staging/`; that difference is accepted, - not a defect to reconcile. - -`Github release` keeps its missing colon and the boilerplate keeps its exact wording. The -template records what ships; it does not improve it. - -### All four checkboxes ship empty - -The release manager's own vote is a separate reply (`+1 (binding)`), as both sampled threads -show. A call that arrives with a quality level already ticked reads as a decision announced -rather than a vote opened. +## The template -## Verification before creating the draft +`vote-mail-template.md` holds headers, slots, and the frozen tail. Two details are kept as +they ship rather than improved: `Github release` has no trailing colon, and all four quality +checkboxes are empty because the release manager's `+1` is a separate reply. -1. All four links resolve; the GitHub release is still flagged pre-release. -2. Ticket sets match — `diff` the mail's `WW-` ids against the page's, excluding the page's - `Rejected requests` section, which the mail deliberately omits. Empty output, or the mail - is not a rendering. -3. Boilerplate byte-identical to the template. -4. `To`/`Bcc` correct, `user@` absent, subject exactly `[VOTE] Apache Struts X.Y.Z`. -5. All four checkboxes empty. +The opener's second sentence is authored per release from the issue list in front of you. +7.2.1's *"a few minor breaking changes plus some bug fixes. Also a lot of dependencies have +been updated"* describes 7.2.1 and is not a form to reuse — 7.3.0 has seven breaking changes +and one dependency bump, so the dependency clause would be false. Both A-clean and C-clean +caught this unprompted; the template records it so the third agent does not have to. ## Output -The skill composes the mail and calls Gmail `create_draft` with To, Bcc, Subject and body set. - -**The skill drafts; it never sends.** Sending opens a binding project vote, which stays the -release manager's keystroke. - -## Failure modes recorded in the skill - -`SKILL.md` closes with the two tables the sibling skills use. - -**Red Flags — STOP:** - -- Cloning the previous release's vote mail -- Retyping the issue list instead of rendering the page -- Re-expanding a summary the page truncated -- Adding `[email protected]` to the recipients -- Pre-ticking a quality level -- Drafting before the page, tag or dist path exist -- Putting a severity, CVE or S2-XXX reference in the mail -- Editing the boilerplate wording -- "Correcting" a Dependency entry to match the Breaking changes prose - -**Common Mistakes** pairs each with its reality, e.g.: +Gmail `create_draft` with To, Bcc, Subject and body set. Never send. -| Mistake | Reality | -|---|---| -| "Last release's vote mail is the fastest start" | It is how the sign-off drifted between 6.10.0 and 7.2.1. Start from the template. | -| "The [TEST] mail went to user@, so this should too" | A vote invitation on the user list solicits non-binding votes. `dev@` only. | -| "The ticket says 3.4.8 but we shipped 3.4.11" | Both are right. The entry is the ticket summary verbatim; the prose is what shipped. | -| "The fix is public, so I can describe it" | A public ticket does not publish the advisory. `dev@` is archived. Neutral framing until the bulletin ships. | -| "The page is up, so I can draft" | Check the tag, the dist path and the staging repo too. A vote on a 404 wastes 72 hours. | +## Testing -## Tests +Per `writing-skills`, the skill is verified by re-running the same three baseline scenarios +with it present. GREEN requires: no `user@` in any header, a draft rather than a send, +boilerplate byte-identical to the template, a body whose parts match the recipe exactly, and +**exactly one mail produced**, carrying no severity, CVE, S2-XXX, bulletin link or reporter +detail on any channel. Any new rationalization found in the GREEN runs is countered and the +scenarios re-run. -Ships without tests, matching `creating-version-notes`. A failing baseline under -`writing-skills` requires subagents, which are not spawned unprompted. Tests can be added on -request. +Baseline transcripts and outputs are kept in the session scratchpad, not committed.
