This is an automated email from the ASF dual-hosted git repository.
chaokunyang pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/fory.git
The following commit(s) were added to refs/heads/main by this push:
new 26b6b6cef chore: improve fory release skill (#3979)
26b6b6cef is described below
commit 26b6b6cefca63aaab0195523ce817d8b3da2845b
Author: Shawn Yang <[email protected]>
AuthorDate: Tue Aug 25 19:44:40 2026 +0800
chore: improve fory release skill (#3979)
## Why?
## What does this PR do?
## Related issues
## AI Contribution Checklist
- [ ] Substantial AI assistance was used in this PR: `yes` / `no`
- [ ] If `yes`, I included a completed [AI Contribution
Checklist](https://github.com/apache/fory/blob/main/AI_POLICY.md#9-contributor-checklist-for-ai-assisted-prs)
in this PR description and the required `AI Usage Disclosure`.
- [ ] If `yes`, my PR description includes the required `ai_review`
summary and screenshot evidence or equivalent persisted links of the
final clean AI review results from both fresh reviewers described in
`AI_POLICY.md`, the Fory-guided reviewer and the independent general
reviewer, on the current PR diff or current HEAD after the latest code
changes.
## Does this PR introduce any user-facing change?
- [ ] Does this PR introduce any public API change?
- [ ] Does this PR introduce any binary protocol compatibility change?
## Benchmark
---
.agents/skills/fory-release/SKILL.md | 177 ++++++++--------
.agents/skills/fory-release/assets/vote-email.txt | 62 ++++++
.../fory-release/references/nexus-staging.md | 72 +++++++
.../fory-release/scripts/check_tag_workflows.py | 176 ++++++++++++++++
.../skills/fory-release/scripts/find_discussion.py | 113 ++++++++++
ci/release.py | 234 +++++++++++++++++++++
6 files changed, 751 insertions(+), 83 deletions(-)
diff --git a/.agents/skills/fory-release/SKILL.md
b/.agents/skills/fory-release/SKILL.md
index 1ebf5a0b7..62f4453b9 100644
--- a/.agents/skills/fory-release/SKILL.md
+++ b/.agents/skills/fory-release/SKILL.md
@@ -1,6 +1,6 @@
---
name: fory-release
-description: Prepare an Apache Fory release candidate from a clean release
branch, including the version bump, JVM staging, RC tag, ASF source artifacts,
SVN upload, and vote email. Use when creating or rerunning a Fory release
candidate.
+description: Prepare an Apache Fory release candidate from a clean release
branch, including the version bump, RC tag, JVM staging, ASF source artifacts,
SVN upload, and vote email. Use when creating or rerunning a Fory release
candidate.
---
# Apache Fory Release
@@ -14,7 +14,8 @@ Collect these values before starting:
- `release_version`: final version without `v` or an RC suffix, such as
`1.7.0`.
- `rc`: RC suffix, such as `rc3`.
- `previous_version`: previous release tag version, such as `1.6.1`.
-- Release discussion URL.
+- Release discussion URL, if already known. If it is not supplied, find the
+ exact release thread in the Fory development-list archive as described below.
Load release-manager details from `.local/fory-release.env`. If it does not
exist, ask for the following values once, create the ignored local file, and
continue. Never commit this file.
@@ -44,6 +45,26 @@ svn_wc="${FORY_DIST_DEV_WC:?missing ASF Subversion
working-copy path}"
Use the same `dist_version` in Subversion and the vote email.
+### Find the release discussion
+
+If the release discussion URL was not supplied, search the
+[Fory development-list
archive](https://lists.apache.org/[email protected])
+for the exact release version and a `[DISCUSS]` subject. Use the deterministic
+helper, which accepts exactly one root discussion thread:
+
+```bash
+discussion_url="$(
+ python3 .agents/skills/fory-release/scripts/find_discussion.py \
+ "$release_version"
+)"
+```
+
+Open the resulting URL and verify that its subject and body discuss the exact
+`release_version`. If the automated search finds zero or multiple roots, use
+the archive UI to search the same exact version and `[DISCUSS]`; ask the
release
+manager only if the result remains absent or ambiguous. When a URL is supplied,
+open and verify it instead of assuming it matches this release.
+
## Release Workflow
### 1. Create a clean release branch
@@ -81,15 +102,7 @@ release_commit="$(git rev-parse HEAD)"
Stage only the version changes produced by the release script.
-### 4. Publish JVM artifacts
-
-```bash
-python3 ci/release.py publish_jvm
-```
-
-The command publishes from a temporary worktree at the committed `HEAD` and
removes that worktree afterward. Record the Java/Kotlin and Scala Nexus staging
repository IDs from the output. In Nexus, close both repositories and confirm
their public URLs are readable before starting the vote. Do not release them
until the vote passes; drop a failed candidate instead.
-
-### 5. Create and push the RC tag
+### 4. Create and push the RC tag
Confirm that neither the local nor remote tag already exists. An invalid RC
gets a new RC number; never move or reuse an RC tag.
@@ -101,9 +114,59 @@ git tag "$rc_tag" && git push apache "$rc_tag"
test "$(git rev-parse "${rc_tag}^{commit}")" = "$release_commit"
```
-The tag starts the ecosystem package-release workflows. Check that the
tag-triggered workflows complete before sending the vote.
+The tag starts the ecosystem package-release workflows. Do not wait for them
+here: start JVM publication immediately so the remote workflows and JVM staging
+run in parallel. Once the tag has been pushed, any JVM or later release failure
+invalidates this RC and requires a higher RC number; never move or reuse the
+tag.
+
+### 5. Publish JVM artifacts
+
+```bash
+python3 ci/release.py publish_jvm
+```
+
+The command publishes from a temporary worktree at the committed `HEAD` and
+removes that worktree afterward. Record the distinct Java/Kotlin and Scala
+Nexus staging repository IDs from the output:
+
+```bash
+java_kotlin_staging_id="orgapachefory-..."
+scala_staging_id="orgapachefory-..."
+test -n "$java_kotlin_staging_id"
+test -n "$scala_staging_id"
+test "$java_kotlin_staging_id" != "$scala_staging_id"
+```
+
+After recording both IDs, read and follow
+[Nexus staging closure](references/nexus-staging.md). It contains the
+credential rules, authenticated state checks, `/bulk/close` request, HTTP 201
+gate, close polling, failure inspection, and anonymous artifact checks. Do not
+close any repository ID that was not recorded from this publication. Keep both
+repositories closed during the vote; do not promote them until the vote passes.
+
+### 6. Check the tag-triggered workflows
+
+After JVM publication and Nexus closure, inspect the workflows that have been
+running since the tag was pushed. Filter by the tag rather than only by commit
+SHA so main-branch runs at the same commit are not mixed into the result. By
+default, wait for every tag-triggered run and require successful conclusions:
+
+```bash
+python3 .agents/skills/fory-release/scripts/check_tag_workflows.py \
+ --repo apache/fory \
+ --tag "$rc_tag" \
+ --commit "$release_commit" \
+ --watch
+```
+
+The helper re-queries by tag after waiting to catch later-created runs. If the
+release manager explicitly waives workflow monitoring for a particular RC, run
+the same command with `--allow-incomplete` instead of `--watch`, and record the
+snapshot IDs, states, and reason. Do not cancel the remote workflows or report
+incomplete runs as successful.
-### 6. Build the ASF source release
+### 7. Build the ASF source release
Start from the clean release branch. The build temporarily commits
release-archive changes and resets them, so verify that it restores the
original commit and clean tree.
@@ -120,7 +183,7 @@ test -f
"dist/apache-fory-${release_version}-src.tar.gz.sha512"
The build command verifies the generated PGP signature and SHA-512 checksum.
-### 7. Commit the source release to ASF Subversion
+### 8. Commit the source release to ASF Subversion
Use a clean, updated working copy of the ASF development distribution
repository.
@@ -132,87 +195,35 @@ cp dist/* "$svn_wc/$dist_version/"
svn add --force "$svn_wc/$dist_version"
svn status "$svn_wc/$dist_version"
svn commit "$svn_wc/$dist_version" -m "Prepare Apache Fory ${rc_tag}"
+test -z "$(svn status "$svn_wc/$dist_version")"
+svn log -l 1 "$svn_wc/$dist_version"
svn ls "https://dist.apache.org/repos/dist/dev/fory/${dist_version}/";
```
Inspect `svn status` before committing. The upload is complete only after `svn
commit` returns a revision and the remote `svn ls` shows the three release
files; local `A` status alone is not an upload.
-### 8. Draft the vote email
-
-Produce a complete, copyable email from this template. Fill every placeholder
from verified output, use an explicit UTC deadline at least 72 hours after
sending, and do not send the email unless requested.
-
-```text
-Subject: [VOTE] Release Apache Fory v${release_version}-${rc}
-
-Hello, Apache Fory Community:
-
-This is a call for vote to release Apache Fory version v${release_version}.
-
-Apache Fory - A blazingly fast multi-language serialization framework
-for idiomatic domain objects, schema IDL, and cross-language data exchange.
-
-The discussion thread:
-${discussion_url}
-
-The change list for this release:
-https://github.com/apache/fory/compare/v${previous_version}...${rc_tag}
-
-The release candidate:
-https://dist.apache.org/repos/dist/dev/fory/${dist_version}/
+### 9. Draft the vote email
-The Maven staging repositories for this release:
-https://repository.apache.org/content/repositories/${java_kotlin_staging_id}/
- (Java/Kotlin artifacts)
-https://repository.apache.org/content/repositories/${scala_staging_id}/
- (Scala artifacts)
-
-Git tag for the release candidate:
-https://github.com/apache/fory/releases/tag/${rc_tag}
-
-Git commit for the release candidate:
-https://github.com/apache/fory/commit/${release_commit}
-
-The artifacts are signed with PGP key fingerprint
-${gpg_fingerprint}, corresponding to ${apache_email}, which can be found in
-the KEYS file:
-https://downloads.apache.org/fory/KEYS
-
-The vote will remain open for at least 72 hours and close at
-${vote_deadline_utc}.
-
-Please vote accordingly:
-
-[ ] +1 approve
-[ ] +0 no opinion
-[ ] -1 disapprove with the reason
-
-To learn more about Fory, please see:
-https://fory.apache.org/
-
-Checklist for reference:
-
-[ ] The Fory source archive downloads successfully.
-[ ] Checksums and PGP signatures are valid.
-[ ] Source distribution names match the release version.
-[ ] LICENSE and NOTICE files are correct.
-[ ] Files have license headers where required.
-[ ] No compiled archives are bundled in the source archive.
-[ ] The release can be built from source.
-
-How to build and test:
-https://github.com/apache/fory/blob/${rc_tag}/docs/development/index.md
-
-Thanks,
-${release_manager_name}
-```
+Read [the vote email template](assets/vote-email.txt) and produce a complete,
+copyable email. Replace every placeholder from verified output, confirm that no
+`${...}` placeholder remains, use an explicit UTC deadline at least 72 hours
+after sending, and do not send the email unless requested.
Before sending, verify the tag and commit, all URLs, both closed Maven staging
repositories, the remote Subversion files, PGP fingerprint, and UTC deadline
against the actual release outputs.
## Stop Conditions
-Stop before the next irreversible step if the Git tree is dirty, a command
fails, an RC tag already exists, the tag and commit differ, either staging
repository is not closed and public, or the Subversion commit is not remotely
visible. Fix the issue and create a higher RC number instead of mutating a
published candidate.
+Before pushing the tag, stop if the Git tree is dirty, a command fails, the RC
+tag already exists, or the tag target would differ from the release commit.
+After pushing the immutable tag, any failed JVM publication, workflow,
+artifact verification, or Subversion publication invalidates that candidate;
+fix the issue and create a higher RC instead of moving or reusing the tag.
+Before sending the vote, require both staging repositories to be closed and
+public, the Subversion commit to be remotely visible, and the tag workflows to
+be successful unless the release manager explicitly waived monitoring.
## References
- [Apache Fory release
guide](https://fory.apache.org/docs/community/how_to_release)
-- [Apache OpenDAL release
skill](https://github.com/apache/opendal/blob/main/.agents/skills/opendal-release/SKILL.md),
used only as secondary operational guidance
+- [Fory development-list
archive](https://lists.apache.org/[email protected])
+- [Sonatype Nexus 2 staging REST
example](https://support.sonatype.com/hc/en-us/articles/213465448-Automatically-dropping-old-staging-repositories)
diff --git a/.agents/skills/fory-release/assets/vote-email.txt
b/.agents/skills/fory-release/assets/vote-email.txt
new file mode 100644
index 000000000..87feab758
--- /dev/null
+++ b/.agents/skills/fory-release/assets/vote-email.txt
@@ -0,0 +1,62 @@
+Subject: [VOTE] Release Apache Fory v${release_version}-${rc}
+
+Hello, Apache Fory Community:
+
+This is a call for vote to release Apache Fory ${release_version}.
+
+Apache Fory is a blazingly fast multi-language serialization framework
+for idiomatic domain objects, schema IDL, and cross-language data exchange.
+
+The discussion thread:
+${discussion_url}
+
+The change list since Apache Fory ${previous_version}:
+https://github.com/apache/fory/compare/v${previous_version}...${rc_tag}
+
+The release candidate artifacts:
+https://dist.apache.org/repos/dist/dev/fory/${dist_version}/
+
+The Maven staging repositories:
+Java and Kotlin:
+https://repository.apache.org/content/repositories/${java_kotlin_staging_id}/
+Scala:
+https://repository.apache.org/content/repositories/${scala_staging_id}/
+
+The release tag:
+https://github.com/apache/fory/releases/tag/${rc_tag}
+
+The release commit:
+https://github.com/apache/fory/commit/${release_commit}
+
+The artifacts are signed with PGP key fingerprint
+${gpg_fingerprint}, corresponding to ${apache_email}. The key can be found in
+the KEYS file:
+https://downloads.apache.org/fory/KEYS
+
+The vote will remain open for at least 72 hours and close at
+${vote_deadline_utc}.
+
+Please vote accordingly:
+
+[ ] +1 approve
+[ ] +0 no opinion
+[ ] -1 disapprove (please explain why)
+
+To learn more about Fory, please see:
+https://fory.apache.org/
+
+Checklist for reference:
+
+[ ] The Fory source archive downloads successfully.
+[ ] Checksums and PGP signatures are valid.
+[ ] Source distribution names match the release version.
+[ ] LICENSE and NOTICE files are correct.
+[ ] Files have license headers where required.
+[ ] No compiled archives are bundled in the source archive.
+[ ] The release can be built from source.
+
+How to build and test:
+https://github.com/apache/fory/blob/${rc_tag}/docs/development/index.md
+
+Thanks,
+${release_manager_name}
diff --git a/.agents/skills/fory-release/references/nexus-staging.md
b/.agents/skills/fory-release/references/nexus-staging.md
new file mode 100644
index 000000000..c7341898c
--- /dev/null
+++ b/.agents/skills/fory-release/references/nexus-staging.md
@@ -0,0 +1,72 @@
+# Apache Nexus Staging Closure
+
+Read this reference after `ci/release.py publish_jvm` creates the distinct
+Java/Kotlin and Scala staging repositories and their IDs have been recorded in
+`java_kotlin_staging_id` and `scala_staging_id`.
+
+## Nexus Repository 2 API
+
+Apache `repository.apache.org` currently exposes the Nexus Repository 2
+staging API. The
+[Sonatype Nexus 2 staging REST
example](https://support.sonatype.com/hc/en-us/articles/213465448-Automatically-dropping-old-staging-repositories)
+documents the repository-list response and bulk-action payload. Its
+`/bulk/drop` operation deletes a staging repository; it is not the close
+operation used here. Close both Fory repositories with `/bulk/close`.
+
+Use the same credentials as the Maven server `apache.releases.https`. Load them
+into `NEXUS_USERNAME` and `NEXUS_PASSWORD` from the release manager's secret
+store. Do not print them, put them in `.local/fory-release.env`, or write them
+to task logs.
+
+```bash
+: "${NEXUS_USERNAME:?export the Apache Nexus username}"
+: "${NEXUS_PASSWORD:?export the Apache Nexus password}"
+```
+
+## Close and Verify the Repositories
+
+Run the repository-owned command with both IDs from the current
+`publish_jvm` output:
+
+```bash
+python3 ci/release.py close_jvm_staging \
+ -v "$release_version" \
+ --rc-tag "$rc_tag" \
+ --java-kotlin-id "$java_kotlin_staging_id" \
+ --scala-id "$scala_staging_id"
+```
+
+The command requires two distinct `orgapachefory-*` IDs and an RC tag matching
+the final release version. Before mutation, it requires both recorded
+repositories to exist and be `open`. It then sends exactly one authenticated
+`/bulk/close` request, requires HTTP 201, polls both repositories until they
are
+`closed`, and verifies anonymous HTTP 200 access to representative Java,
+Kotlin, Scala 2.13, and Scala 3 artifacts.
+
+If closure times out, the command retrieves and logs the activity for both
+repositories before stopping. Do not repeatedly submit close requests or
+proceed to the vote while either repository is open or transitioning.
+
+## Read-Only Verification
+
+For recovery or audit work on repositories that are already closed, add
+`--verify-only`:
+
+```bash
+python3 ci/release.py close_jvm_staging \
+ -v "$release_version" \
+ --rc-tag "$rc_tag" \
+ --java-kotlin-id "$java_kotlin_staging_id" \
+ --scala-id "$scala_staging_id" \
+ --verify-only
+```
+
+This path never submits a close request. It requires both repositories to
+already be `closed` and repeats the anonymous artifact checks. HTTP 200 from an
+authenticated API request is not a substitute for these public download
+checks.
+
+Record both repository IDs, their `closed` states, and the public verification
+results. Keep the repositories closed during the vote. Do not promote/release
+them until the vote passes. Drop a failed candidate only after the release
+manager decides to abandon that RC.
diff --git a/.agents/skills/fory-release/scripts/check_tag_workflows.py
b/.agents/skills/fory-release/scripts/check_tag_workflows.py
new file mode 100644
index 000000000..2a5056f1f
--- /dev/null
+++ b/.agents/skills/fory-release/scripts/check_tag_workflows.py
@@ -0,0 +1,176 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import shutil
+import subprocess
+
+RUN_FIELDS = "databaseId,workflowName,status,conclusion,headSha,headBranch,url"
+RC_TAG_PATTERN = re.compile(r"v\d+\.\d+\.\d+-rc\d+")
+COMMIT_PATTERN = re.compile(r"[0-9a-f]{40}")
+
+
+def _parse_args():
+ parser = argparse.ArgumentParser(
+ description="Validate GitHub Actions runs triggered by a Fory RC tag"
+ )
+ parser.add_argument("--repo", default="apache/fory", help="GitHub
repository")
+ parser.add_argument("--tag", required=True, help="immutable RC tag")
+ parser.add_argument("--commit", required=True, help="full release commit
SHA")
+ action = parser.add_mutually_exclusive_group(required=True)
+ action.add_argument(
+ "--watch",
+ action="store_true",
+ help="wait for every tag-triggered run and require success",
+ )
+ action.add_argument(
+ "--allow-incomplete",
+ action="store_true",
+ help="record a read-only snapshot without waiting for incomplete runs",
+ )
+ return parser.parse_args()
+
+
+def _run_gh(arguments):
+ result = subprocess.run(
+ ["gh", *arguments],
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE,
+ text=True,
+ check=False,
+ )
+ if result.returncode != 0:
+ detail = result.stderr.strip() or result.stdout.strip()
+ raise RuntimeError(f"gh {' '.join(arguments)} failed: {detail}")
+ return result.stdout
+
+
+def _watch_run(repo, database_id):
+ result = subprocess.run(
+ [
+ "gh",
+ "run",
+ "watch",
+ str(database_id),
+ "--repo",
+ repo,
+ "--exit-status",
+ ],
+ check=False,
+ )
+ if result.returncode != 0:
+ raise RuntimeError(f"GitHub workflow run {database_id} did not
succeed")
+
+
+def _list_runs(repo, tag):
+ output = _run_gh(
+ [
+ "run",
+ "list",
+ "--repo",
+ repo,
+ "--branch",
+ tag,
+ "--event",
+ "push",
+ "--limit",
+ "100",
+ "--json",
+ RUN_FIELDS,
+ ]
+ )
+ try:
+ runs = json.loads(output)
+ except json.JSONDecodeError as exc:
+ raise RuntimeError(f"gh returned invalid workflow JSON: {exc}") from
None
+ if not isinstance(runs, list) or not runs:
+ raise RuntimeError(f"No tag-triggered workflows found for {tag}")
+ return runs
+
+
+def _validate_runs(runs, tag, commit):
+ wrong_revision = [
+ run
+ for run in runs
+ if run.get("headSha") != commit or run.get("headBranch") != tag
+ ]
+ if wrong_revision:
+ raise RuntimeError(f"Unexpected workflow revisions: {wrong_revision}")
+ for run in sorted(
+ runs, key=lambda item: (str(item.get("workflowName")),
item.get("databaseId"))
+ ):
+ print(
+ run.get("databaseId"),
+ run.get("workflowName"),
+ run.get("status"),
+ run.get("conclusion") or "-",
+ run.get("url"),
+ )
+ failed = [
+ run
+ for run in runs
+ if run.get("status") == "completed" and run.get("conclusion") !=
"success"
+ ]
+ if failed:
+ raise RuntimeError(f"Failed tag-triggered workflows: {failed}")
+
+
+def _incomplete_runs(runs):
+ return [run for run in runs if run.get("status") != "completed"]
+
+
+def _watch_runs(repo, tag, commit):
+ while True:
+ runs = _list_runs(repo, tag)
+ _validate_runs(runs, tag, commit)
+ incomplete = _incomplete_runs(runs)
+ if not incomplete:
+ return
+ for run in incomplete:
+ database_id = run.get("databaseId")
+ if database_id is None:
+ raise RuntimeError(f"Workflow run has no databaseId: {run}")
+ _watch_run(repo, database_id)
+
+
+def main():
+ args = _parse_args()
+ if not shutil.which("gh"):
+ raise SystemExit("gh is required")
+ if not RC_TAG_PATTERN.fullmatch(args.tag):
+ raise SystemExit("--tag must be an RC tag such as v1.7.0-rc3")
+ if not COMMIT_PATTERN.fullmatch(args.commit):
+ raise SystemExit("--commit must be a full 40-character lowercase SHA")
+ try:
+ if args.watch:
+ _watch_runs(args.repo, args.tag, args.commit)
+ return
+ runs = _list_runs(args.repo, args.tag)
+ _validate_runs(runs, args.tag, args.commit)
+ incomplete = _incomplete_runs(runs)
+ if incomplete:
+ print(f"Monitoring waived: {len(incomplete)} workflow run(s)
incomplete")
+ except RuntimeError as exc:
+ raise SystemExit(str(exc)) from None
+
+
+if __name__ == "__main__":
+ main()
diff --git a/.agents/skills/fory-release/scripts/find_discussion.py
b/.agents/skills/fory-release/scripts/find_discussion.py
new file mode 100644
index 000000000..cc088badd
--- /dev/null
+++ b/.agents/skills/fory-release/scripts/find_discussion.py
@@ -0,0 +1,113 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import urllib.error
+import urllib.parse
+import urllib.request
+
+PONY_MAIL_SEARCH_URL = "https://lists.apache.org/api/stats.lua";
+FORY_LIST_DOMAIN = "fory.apache.org"
+RELEASE_VERSION_PATTERN = re.compile(r"\d+\.\d+\.\d+")
+
+
+def _parse_args():
+ parser = argparse.ArgumentParser(
+ description="Find the root Fory release discussion thread"
+ )
+ parser.add_argument(
+ "release_version",
+ help="final release version without a v prefix or RC suffix",
+ )
+ parser.add_argument(
+ "--years",
+ type=int,
+ default=1,
+ help="number of years of mailing-list history to search",
+ )
+ return parser.parse_args()
+
+
+def _search_messages(release_version, years):
+ query = urllib.parse.urlencode(
+ {
+ "d": f"lte={years}y",
+ "list": "dev",
+ "domain": FORY_LIST_DOMAIN,
+ "q": release_version,
+ "header_subject": "DISCUSS",
+ }
+ )
+ request = urllib.request.Request(
+ f"{PONY_MAIL_SEARCH_URL}?{query}",
+ headers={
+ "Accept": "application/json",
+ "User-Agent": "apache-fory-release-helper/1",
+ },
+ )
+ try:
+ with urllib.request.urlopen(request, timeout=30) as response:
+ payload = json.load(response)
+ except (urllib.error.HTTPError, urllib.error.URLError,
json.JSONDecodeError) as exc:
+ raise RuntimeError(f"Apache mailing-list search failed: {exc}") from
None
+ if not isinstance(payload, dict):
+ raise RuntimeError("Apache mailing-list search returned invalid JSON")
+ messages = payload.get("emails")
+ if not isinstance(messages, list):
+ raise RuntimeError("Apache mailing-list search returned no email list")
+ return messages
+
+
+def _discussion_url(release_version, messages):
+ version = re.compile(rf"(?<![\d.]){re.escape(release_version)}(?![\d.])")
+ matches = []
+ for message in messages:
+ subject = str(message.get("subject", ""))
+ if message.get("in-reply-to"):
+ continue
+ if "[DISCUSS]" not in subject.upper() or not version.search(subject):
+ continue
+ thread_id = message.get("id")
+ if thread_id:
+ matches.append(str(thread_id))
+ if len(matches) != 1:
+ raise RuntimeError(
+ f"Expected one root [DISCUSS] thread for {release_version}, "
+ f"found {len(matches)}"
+ )
+ return f"https://lists.apache.org/thread/{matches[0]}";
+
+
+def main():
+ args = _parse_args()
+ if not RELEASE_VERSION_PATTERN.fullmatch(args.release_version):
+ raise SystemExit("release_version must be a final version such as
1.7.0")
+ if args.years < 1:
+ raise SystemExit("--years must be positive")
+ try:
+ messages = _search_messages(args.release_version, args.years)
+ print(_discussion_url(args.release_version, messages))
+ except RuntimeError as exc:
+ raise SystemExit(str(exc)) from None
+
+
+if __name__ == "__main__":
+ main()
diff --git a/ci/release.py b/ci/release.py
index 128dfbc90..6f9f4df24 100644
--- a/ci/release.py
+++ b/ci/release.py
@@ -18,6 +18,7 @@
from __future__ import annotations
import argparse
+import base64
import json
import logging
import os
@@ -26,6 +27,9 @@ import shutil
import subprocess
import sys
import tempfile
+import time
+import urllib.error
+import urllib.request
import xml.etree.ElementTree as ET
import zipfile
@@ -132,6 +136,10 @@ RELEASE_DOC_ROOTS = (
RELEASE_DOC_EXTS = (".md", ".example")
VERSION_SUFFIX_PATTERN =
r"(?i:-snapshot|-dev\d*|\.dev\d+|-(?:alpha|beta|rc)\.\d+)"
VERSION_PATTERN = rf"\d+\.\d+\.\d+(?:{VERSION_SUFFIX_PATTERN})?"
+NEXUS_BASE_URL = "https://repository.apache.org";
+NEXUS_TIMEOUT_SECONDS = 30
+NEXUS_CLOSE_ATTEMPTS = 30
+NEXUS_CLOSE_INTERVAL_SECONDS = 10
def prepare(v: str):
@@ -272,6 +280,198 @@ def publish_jvm(languages="all", mode="release"):
raise NotImplementedError(f"Unsupported JVM release language:
{lang}")
+def close_jvm_staging(
+ v,
+ rc_tag,
+ java_kotlin_staging_id,
+ scala_staging_id,
+ verify_only=False,
+):
+ """Close and verify the two Nexus repositories created by publish_jvm."""
+ _check_release_version(v)
+ if not re.fullmatch(r"\d+\.\d+\.\d+", v):
+ raise ValueError(f"Invalid final release version: {v}")
+ if not re.fullmatch(rf"v{re.escape(v)}-rc\d+", rc_tag):
+ raise ValueError(f"RC tag {rc_tag} does not match release version {v}")
+ staging_ids = [java_kotlin_staging_id, scala_staging_id]
+ for staging_id in staging_ids:
+ if not re.fullmatch(r"orgapachefory-\d+", staging_id):
+ raise ValueError(f"Invalid Apache Fory staging repository ID:
{staging_id}")
+ if len(set(staging_ids)) != len(staging_ids):
+ raise ValueError("Java/Kotlin and Scala staging repository IDs must
differ")
+
+ authorization = _nexus_authorization()
+ expected_state = "closed" if verify_only else "open"
+ states = _nexus_states(staging_ids, authorization)
+ _require_nexus_state(states, expected_state)
+ if not verify_only:
+ payload = {
+ "data": {
+ "stagedRepositoryIds": staging_ids,
+ "description": f"Close Apache Fory {rc_tag} staging
repositories",
+ }
+ }
+ status, _ = _nexus_request(
+ "/service/local/staging/bulk/close",
+ authorization,
+ method="POST",
+ payload=payload,
+ )
+ if status != 201:
+ raise RuntimeError(f"Nexus bulk close returned HTTP {status},
expected 201")
+ logger.info("Submitted one Nexus bulk close request for %s",
staging_ids)
+ _wait_for_nexus_close(staging_ids, authorization)
+
+ _verify_nexus_downloads(v, java_kotlin_staging_id, scala_staging_id)
+
+
+def _nexus_authorization():
+ username = os.environ.get("NEXUS_USERNAME")
+ password = os.environ.get("NEXUS_PASSWORD")
+ if not username or not password:
+ raise RuntimeError(
+ "NEXUS_USERNAME and NEXUS_PASSWORD are required for Nexus staging"
+ )
+ credentials = base64.b64encode(f"{username}:{password}".encode()).decode()
+ return f"Basic {credentials}"
+
+
+def _nexus_request(path, authorization, method="GET", payload=None):
+ headers = {
+ "Accept": "application/json",
+ "Authorization": authorization,
+ "User-Agent": "apache-fory-release-helper/1",
+ }
+ body = None
+ if payload is not None:
+ headers["Content-Type"] = "application/json"
+ body = json.dumps(payload).encode()
+ request = urllib.request.Request(
+ f"{NEXUS_BASE_URL}{path}",
+ data=body,
+ headers=headers,
+ method=method,
+ )
+ try:
+ with urllib.request.urlopen(request, timeout=NEXUS_TIMEOUT_SECONDS) as
response:
+ return response.status, response.read()
+ except urllib.error.HTTPError as exc:
+ detail = exc.read(4096).decode(errors="replace").strip()
+ raise RuntimeError(
+ f"Nexus {method} {path} failed with HTTP {exc.code}: {detail}"
+ ) from None
+ except urllib.error.URLError as exc:
+ raise RuntimeError(f"Nexus {method} {path} failed: {exc.reason}") from
None
+
+
+def _nexus_states(staging_ids, authorization):
+ status, body = _nexus_request(
+ "/service/local/staging/profile_repositories", authorization
+ )
+ if status != 200:
+ raise RuntimeError(
+ f"Nexus repository list returned HTTP {status}, expected 200"
+ )
+ try:
+ payload = json.loads(body)
+ except json.JSONDecodeError as exc:
+ raise RuntimeError(f"Invalid Nexus repository-list response: {exc}")
from None
+ data = payload.get("data") if isinstance(payload, dict) else None
+ if not isinstance(data, list):
+ raise RuntimeError("Nexus repository-list response has no data list")
+ repositories = {
+ item.get("repositoryId"): item for item in data if isinstance(item,
dict)
+ }
+ missing = [
+ staging_id for staging_id in staging_ids if staging_id not in
repositories
+ ]
+ if missing:
+ raise RuntimeError(f"Missing Nexus staging repositories: {missing}")
+ states = {
+ staging_id: repositories[staging_id].get("type") for staging_id in
staging_ids
+ }
+ for staging_id, state in states.items():
+ logger.info("Nexus staging repository %s: %s", staging_id, state)
+ return states
+
+
+def _require_nexus_state(states, expected_state):
+ unexpected = {
+ staging_id: state
+ for staging_id, state in states.items()
+ if state != expected_state
+ }
+ if unexpected:
+ raise RuntimeError(
+ f"Nexus repositories must be {expected_state}; found {unexpected}"
+ )
+
+
+def _wait_for_nexus_close(staging_ids, authorization):
+ for attempt in range(1, NEXUS_CLOSE_ATTEMPTS + 1):
+ states = _nexus_states(staging_ids, authorization)
+ if all(state == "closed" for state in states.values()):
+ return
+ if attempt < NEXUS_CLOSE_ATTEMPTS:
+ time.sleep(NEXUS_CLOSE_INTERVAL_SECONDS)
+ _log_nexus_activity(staging_ids, authorization)
+ raise RuntimeError(
+ f"Nexus repositories did not close after {NEXUS_CLOSE_ATTEMPTS} checks"
+ )
+
+
+def _log_nexus_activity(staging_ids, authorization):
+ for staging_id in staging_ids:
+ path = f"/service/local/staging/repository/{staging_id}/activity"
+ try:
+ status, body = _nexus_request(path, authorization)
+ if status != 200:
+ logger.error(
+ "Nexus activity for %s returned HTTP %s", staging_id,
status
+ )
+ continue
+ try:
+ activity = json.dumps(json.loads(body), indent=2,
sort_keys=True)
+ except json.JSONDecodeError:
+ activity = body[:8192].decode(errors="replace")
+ logger.error("Nexus activity for %s:\n%s", staging_id, activity)
+ except RuntimeError as exc:
+ logger.error("Unable to read Nexus activity for %s: %s",
staging_id, exc)
+
+
+def _verify_nexus_downloads(v, java_kotlin_staging_id, scala_staging_id):
+ java_kotlin_url =
f"{NEXUS_BASE_URL}/content/repositories/{java_kotlin_staging_id}/"
+ scala_url = f"{NEXUS_BASE_URL}/content/repositories/{scala_staging_id}/"
+ artifact_urls = [
+ java_kotlin_url,
+ f"{java_kotlin_url}org/apache/fory/fory-core/{v}/fory-core-{v}.jar",
+
f"{java_kotlin_url}org/apache/fory/fory-kotlin/{v}/fory-kotlin-{v}.jar",
+ scala_url,
+
f"{scala_url}org/apache/fory/fory-scala_2.13/{v}/fory-scala_2.13-{v}.jar",
+
f"{scala_url}org/apache/fory/fory-json-scala_3/{v}/fory-json-scala_3-{v}.jar",
+ ]
+ for url in artifact_urls:
+ request = urllib.request.Request(
+ url, headers={"User-Agent": "apache-fory-release-helper/1"}
+ )
+ try:
+ with urllib.request.urlopen(
+ request, timeout=NEXUS_TIMEOUT_SECONDS
+ ) as response:
+ status = response.status
+ except urllib.error.HTTPError as exc:
+ status = exc.code
+ except urllib.error.URLError as exc:
+ raise RuntimeError(
+ f"Anonymous Nexus download failed for {url}: {exc.reason}"
+ )
+ logger.info("Anonymous Nexus download HTTP %s: %s", status, url)
+ if status != 200:
+ raise RuntimeError(
+ f"Anonymous Nexus download returned HTTP {status}: {url}"
+ )
+
+
def _jvm_release_langs(languages):
if languages in (None, "", "all"):
return list(JVM_RELEASE_LANGS)
@@ -1804,6 +2004,40 @@ def _parse_args():
)
publish_jvm_parser.set_defaults(func=publish_jvm)
+ close_jvm_parser = subparsers.add_parser(
+ "close_jvm_staging",
+ description="Close and verify the two Nexus repositories from
publish_jvm",
+ )
+ close_jvm_parser.add_argument(
+ "-v",
+ dest="v",
+ required=True,
+ help="final release version without a v prefix or RC suffix",
+ )
+ close_jvm_parser.add_argument(
+ "--rc-tag",
+ required=True,
+ help="immutable release-candidate tag",
+ )
+ close_jvm_parser.add_argument(
+ "--java-kotlin-id",
+ dest="java_kotlin_staging_id",
+ required=True,
+ help="staging repository ID produced by Java and Kotlin publication",
+ )
+ close_jvm_parser.add_argument(
+ "--scala-id",
+ dest="scala_staging_id",
+ required=True,
+ help="staging repository ID produced by Scala publication",
+ )
+ close_jvm_parser.add_argument(
+ "--verify-only",
+ action="store_true",
+ help="require closed state and verify downloads without submitting a
close",
+ )
+ close_jvm_parser.set_defaults(func=close_jvm_staging)
+
verify_java_parser = subparsers.add_parser(
"verify_java_artifacts",
description="Verify Java binary and source-release artifacts",
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
