This is an automated email from the ASF dual-hosted git repository. andrewmusselman pushed a commit to branch adding-pypi-trusted-publishing in repository https://gitbox.apache.org/repos/asf/mahout.git
commit 61e8c9e32ef628c0efca757f28ae738a5a7c6657 Author: Andrew Musselman <[email protected]> AuthorDate: Sun May 31 13:45:02 2026 -0700 Add Trusted Publishing release workflow for qumat and qumat-qdp - release.yml: builds qumat (pure Python) and qumat-qdp (Rust/CUDA) - qumat-qdp builds with CUDA 12.5 toolkit and gcc-13 for real GPU kernels - Publishes to PyPI via OIDC Trusted Publishing (no API tokens) - .asf.yaml: adds pypi environment with reviewer gates --- .asf.yaml | 17 +++++++ .github/workflows/release.yml | 116 ++++++++++++++++++++++++++++++++++++++++++ dev/release.md | 48 ++++++++++++----- 3 files changed, 169 insertions(+), 12 deletions(-) diff --git a/.asf.yaml b/.asf.yaml index 7e82c5b2f..fbc1e54a1 100644 --- a/.asf.yaml +++ b/.asf.yaml @@ -59,6 +59,23 @@ github: main: required_linear_history: true required_signatures: false + environments: + pypi: + required_reviewers: + - id: andrewmusselman + type: User + - id: guan404ming + type: User + - id: 400Ping + type: User + - id: rawkintrevo + type: User + wait_timer: 5 + deployment_branch_policy: + protected_branches: false + policies: + - name: "v*" + type: tag notifications: commits: [email protected] diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml new file mode 100644 index 000000000..2f155ac28 --- /dev/null +++ b/.github/workflows/release.yml @@ -0,0 +1,116 @@ +name: Release +on: + push: + tags: + - v* + - mahout-qumat-* + +jobs: + # ------------------------------------------------------------------ + # qumat (pure Python — one wheel for all Python versions) + # ------------------------------------------------------------------ + build-qumat: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + - name: Build qumat + run: | + pip install build + python -m build --sdist --wheel + - uses: actions/upload-artifact@v5 + with: + name: qumat-dist + path: dist/ + + # ------------------------------------------------------------------ + # qumat-qdp (Rust/CUDA via maturin) + # Builds with CUDA toolkit for real GPU kernel compilation. + # ------------------------------------------------------------------ + build-qdp-linux: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + + - name: Install CUDA toolkit + uses: Jimver/cuda-toolkit@3d45d157f327c09c04b50ee6ccdea2d9d017ec76 + with: + cuda: "12.5.0" + method: "network" + + - name: Install compatible GCC (nvcc requires gcc <= 13) + run: | + sudo apt-get update + sudo apt-get install -y gcc-13 g++-13 + sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-13 100 + sudo update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-13 100 + sudo update-alternatives --install /usr/bin/cc cc /usr/bin/gcc-13 100 + sudo update-alternatives --install /usr/bin/c++ c++ /usr/bin/g++-13 100 + gcc --version + + - uses: dtolnay/rust-toolchain@29eef336d9b2848a0b548edc03f92a220660cdb8 + with: + toolchain: stable + + - uses: actions/setup-python@v5 + with: + python-version: | + 3.10 + 3.11 + 3.12 + + - name: Install maturin and patchelf + run: pip install maturin patchelf + + - name: Build qumat-qdp wheels + working-directory: qdp/qdp-python + run: | + maturin build --release --out dist \ + -i python3.10 -i python3.11 -i python3.12 \ + --features pyo3/extension-module \ + --compatibility manylinux_2_34 + + - uses: actions/upload-artifact@v5 + with: + name: qdp-dist-linux + path: qdp/qdp-python/dist/ + + build-qdp-sdist: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + - name: Build qumat-qdp sdist + uses: PyO3/maturin-action@v1 + with: + working-directory: qdp/qdp-python + command: sdist + args: --out dist + - uses: actions/upload-artifact@v5 + with: + name: qdp-dist-sdist + path: qdp/qdp-python/dist/ + + # ------------------------------------------------------------------ + # Publish all packages to PyPI + # ------------------------------------------------------------------ + publish: + name: Publish to PyPI + runs-on: ubuntu-latest + needs: [build-qumat, build-qdp-linux, build-qdp-sdist] + environment: + name: pypi + url: https://pypi.org/p/qumat + permissions: + id-token: write + steps: + - uses: actions/download-artifact@v5 + with: + pattern: "*-dist*" + merge-multiple: true + path: dist + - name: List distributions + run: ls -la dist/ + - name: Publish to PyPI + uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e diff --git a/dev/release.md b/dev/release.md index 825c91612..7ccdb8056 100644 --- a/dev/release.md +++ b/dev/release.md @@ -266,18 +266,42 @@ Once the vote passes: 2. Click **"Promote to Release"** (or **"Publish"**). - This moves the signed source artifacts to the Apache release SVN at `https://dist.apache.org/repos/dist/release/mahout/`. -### 3.2 Publish Final Version to PyPI -After the PMC vote passes, publish the final version to PyPI. The artifacts must be built from the voted source in Apache SVN to ensure they match what the PMC approved. - -**Automated Process:** -Set up a GitHub Actions workflow that: -- Is triggered manually via "Workflow Dispatch" with the release version as input. -- Downloads the approved source tarball from Apache SVN (`https://dist.apache.org/repos/dist/release/mahout/`). -- Verifies GPG signatures and checksums. -- Builds wheels from the verified source. -- Uploads to PyPI using Trusted Publisher configuration. - -**Important:** The PyPI release is built from the exact source code stored in Apache SVN that the PMC voted on, ensuring consistency between the Apache release and PyPI. +### 3.2 Publish Final Version to PyPI (Trusted Publishing) + +After the PMC vote passes, publish to PyPI by tagging the release: + +```bash +# Tag the voted release on the stable branch +git checkout v0.6-stable +git tag -a v0.6.0 -m "Release 0.6.0" +git push upstream v0.6.0 +``` + +This triggers the `release.yml` GitHub Actions workflow which: + +1. Builds `qumat` (pure Python wheel + sdist) +2. Builds `qumat-qdp` (Rust/maturin wheels for Python 3.10/3.11/3.12 + sdist) +3. Waits for reviewer approval (any one of the configured reviewers) +4. Publishes all artifacts to PyPI via Trusted Publishing (OIDC — no API tokens) + +**To approve the deploy:** + +1. Go to **GitHub Actions** → the release workflow run +2. The publish job shows **"Waiting for review"** +3. Click **"Review deployments"** → check **`pypi`** → **"Approve and deploy"** + +**Verify the release:** + +```bash +pip install qumat==0.6.0 +pip install qumat-qdp==0.2.0 +python -c "import qumat; print(qumat.__version__)" +``` + +**Note:** The `release.yml` workflow uses [PyPI Trusted Publishing](https://docs.pypi.org/trusted-publishers/) — no API tokens or `.pypirc` files needed. Authentication is handled via OIDC between GitHub Actions and PyPI. The trusted publishers are configured at: + +- https://pypi.org/manage/project/qumat/settings/publishing/ +- https://pypi.org/manage/project/qumat-qdp/settings/publishing/ ### 3.3 Post-Release Actions - **Tag the release** in Git:
