This is an automated email from the ASF dual-hosted git repository. XiaoHongbo-Hope pushed a commit to branch release-0.1 in repository https://gitbox.apache.org/repos/asf/paimon-mosaic.git
commit 2dd78cc090b3b9b7db6311f278dd1fdfdb43fa89 Author: QuakeWang <[email protected]> AuthorDate: Sun May 24 16:14:34 2026 +0800 ci: publish Python release after Rust and Java (#42) Signed-off-by: QuakeWang <[email protected]> --- .github/workflows/release-java.yml | 9 +-- .github/workflows/release-python-publish.yml | 89 ++++++++++++++++++++++ .github/workflows/release-python.yml | 57 +++----------- .github/workflows/release-rust.yml | 13 ++-- .../workflows/{release-rust.yml => release.yml} | 59 +++++++------- docs/creating-a-release.html | 24 +++--- docs/verifying-a-release-candidate.html | 2 +- 7 files changed, 156 insertions(+), 97 deletions(-) diff --git a/.github/workflows/release-java.yml b/.github/workflows/release-java.yml index 519b5b2..961ba7d 100644 --- a/.github/workflows/release-java.yml +++ b/.github/workflows/release-java.yml @@ -19,18 +19,15 @@ name: Release Java on: - push: - tags: - - "v[0-9]+.[0-9]+.[0-9]+" - - "v[0-9]+.[0-9]+.[0-9]+-rc[0-9]+" + workflow_call: workflow_dispatch: env: JDK_VERSION: 8 concurrency: - group: ${{ github.workflow }}-${{ github.ref }} - cancel-in-progress: true + group: release-java-${{ github.ref }} + cancel-in-progress: false jobs: build-native: diff --git a/.github/workflows/release-python-publish.yml b/.github/workflows/release-python-publish.yml new file mode 100644 index 0000000..34cab74 --- /dev/null +++ b/.github/workflows/release-python-publish.yml @@ -0,0 +1,89 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# Publish Python wheels after the orchestrator has confirmed Rust and Java +# release jobs completed successfully. + +name: Release Python Publish + +on: + workflow_call: + +concurrency: + group: release-python-publish-${{ github.ref }} + cancel-in-progress: false + +permissions: + actions: read + contents: read + +jobs: + publish: + name: Publish to PyPI + if: github.repository == 'apache/paimon-mosaic' && startsWith(github.ref, 'refs/tags/') + runs-on: ubuntu-latest + steps: + - uses: actions/download-artifact@v5 + with: + pattern: wheels-* + merge-multiple: true + path: dist + + - name: Verify wheel versions + env: + TAG_NAME: ${{ github.ref_name }} + shell: bash + run: | + set -euo pipefail + + expected_version="${TAG_NAME#v}" + expected_version="${expected_version/-rc/rc}" + shopt -s nullglob + wheels=(dist/*.whl) + + if [[ "${#wheels[@]}" -eq 0 ]]; then + echo "No wheels found in dist" >&2 + exit 1 + fi + + for wheel in "${wheels[@]}"; do + base="$(basename "$wheel")" + case "$base" in + paimon_mosaic-"${expected_version}"-*.whl) ;; + *) + echo "Unexpected wheel for ${TAG_NAME}: ${base}" >&2 + exit 1 + ;; + esac + done + + - name: Publish to TestPyPI + if: contains(github.ref_name, '-rc') + uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e + with: + repository-url: https://test.pypi.org/legacy/ + skip-existing: true + packages-dir: dist + password: ${{ secrets.TEST_PYPI_API_TOKEN }} + + - name: Publish to PyPI + if: ${{ !contains(github.ref_name, '-') }} + uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e + with: + skip-existing: true + packages-dir: dist + password: ${{ secrets.PYPI_API_TOKEN }} diff --git a/.github/workflows/release-python.yml b/.github/workflows/release-python.yml index 1b103d0..257f04d 100644 --- a/.github/workflows/release-python.yml +++ b/.github/workflows/release-python.yml @@ -15,25 +15,21 @@ # specific language governing permissions and limitations # under the License. -# Publish the paimon-mosaic Python package to PyPI. +# Build the paimon-mosaic Python release wheels. # -# Trigger: push a version tag (e.g. v0.1.0, v0.1.0-rc1). -# Pre-release tags (containing '-') publish to TestPyPI; release tags publish to PyPI. -# -# Token auth: add secrets PYPI_API_TOKEN / TEST_PYPI_API_TOKEN for publishing. +# Trigger: called by release.yml or manually dispatched. +# Publishing is handled by release-python-publish.yml after Rust and Java +# release jobs complete successfully. -name: Release Python +name: Release Python Wheels on: - push: - tags: - - "v[0-9]+.[0-9]+.[0-9]+" - - "v[0-9]+.[0-9]+.[0-9]+-rc[0-9]+" + workflow_call: workflow_dispatch: concurrency: - group: ${{ github.workflow }}-${{ github.ref }}-${{ github.event_name }} - cancel-in-progress: true + group: release-python-${{ github.ref }} + cancel-in-progress: false permissions: contents: read @@ -53,7 +49,7 @@ jobs: - uses: actions/checkout@v6 - name: Inject RC version into pyproject.toml - if: contains(github.ref, '-rc') + if: startsWith(github.ref, 'refs/tags/') && contains(github.ref_name, '-rc') run: | TAG="${GITHUB_REF#refs/tags/v}" # Convert 0.1.0-rc1 to PEP 440: 0.1.0rc1 @@ -99,7 +95,7 @@ jobs: - uses: actions/checkout@v6 - name: Inject RC version into pyproject.toml - if: contains(github.ref, '-rc') + if: startsWith(github.ref, 'refs/tags/') && contains(github.ref_name, '-rc') run: | TAG="${GITHUB_REF#refs/tags/v}" PEP440_VERSION=$(echo "$TAG" | sed 's/-rc/rc/') @@ -150,7 +146,7 @@ jobs: - uses: actions/checkout@v6 - name: Inject RC version into pyproject.toml - if: contains(github.ref, '-rc') + if: startsWith(github.ref, 'refs/tags/') && contains(github.ref_name, '-rc') shell: bash run: | TAG="${GITHUB_REF#refs/tags/v}" @@ -186,34 +182,3 @@ jobs: with: name: wheels-windows-x86_64 path: python/dist/*.whl - - release: - name: Publish to PyPI - runs-on: ubuntu-latest - permissions: - contents: read - needs: [wheels-linux, wheels-macos, wheels-windows] - if: startsWith(github.ref, 'refs/tags/') - steps: - - uses: actions/download-artifact@v5 - with: - pattern: wheels-* - merge-multiple: true - path: dist - - - name: Publish to TestPyPI - if: contains(github.ref, '-') - uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e - with: - repository-url: https://test.pypi.org/legacy/ - skip-existing: true - packages-dir: dist - password: ${{ secrets.TEST_PYPI_API_TOKEN }} - - - name: Publish to PyPI - if: ${{ !contains(github.ref, '-') }} - uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e - with: - skip-existing: true - packages-dir: dist - password: ${{ secrets.PYPI_API_TOKEN }} diff --git a/.github/workflows/release-rust.yml b/.github/workflows/release-rust.yml index 7585019..7a74ef6 100644 --- a/.github/workflows/release-rust.yml +++ b/.github/workflows/release-rust.yml @@ -17,7 +17,7 @@ # Publish paimon-mosaic-core crate to crates.io. # -# Trigger: push a version tag (e.g. v0.1.0, v0.1.0-rc1). +# Trigger: called by release.yml or manually dispatched. # Pre-release tags (containing '-') only run dry-run checks without publishing. # # Token auth: add secret CARGO_REGISTRY_TOKEN for crates.io publishing. @@ -25,12 +25,13 @@ name: Release Rust on: - push: - tags: - - "v[0-9]+.[0-9]+.[0-9]+" - - "v[0-9]+.[0-9]+.[0-9]+-rc[0-9]+" + workflow_call: workflow_dispatch: +concurrency: + group: release-rust-${{ github.ref }} + cancel-in-progress: false + jobs: publish: runs-on: ubuntu-latest @@ -48,7 +49,7 @@ jobs: run: cargo publish -p paimon-mosaic-core --dry-run - name: Publish paimon-mosaic-core to crates.io - if: startsWith(github.ref, 'refs/tags/') && !contains(github.ref, '-') + if: github.repository == 'apache/paimon-mosaic' && startsWith(github.ref, 'refs/tags/') && !contains(github.ref_name, '-') run: cargo publish -p paimon-mosaic-core env: CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }} diff --git a/.github/workflows/release-rust.yml b/.github/workflows/release.yml similarity index 51% copy from .github/workflows/release-rust.yml copy to .github/workflows/release.yml index 7585019..7c1f516 100644 --- a/.github/workflows/release-rust.yml +++ b/.github/workflows/release.yml @@ -15,14 +15,10 @@ # specific language governing permissions and limitations # under the License. -# Publish paimon-mosaic-core crate to crates.io. -# -# Trigger: push a version tag (e.g. v0.1.0, v0.1.0-rc1). -# Pre-release tags (containing '-') only run dry-run checks without publishing. -# -# Token auth: add secret CARGO_REGISTRY_TOKEN for crates.io publishing. +# Orchestrate the full release while keeping each language release workflow +# independently runnable through workflow_dispatch. -name: Release Rust +name: Release on: push: @@ -31,24 +27,33 @@ on: - "v[0-9]+.[0-9]+.[0-9]+-rc[0-9]+" workflow_dispatch: +concurrency: + group: release-${{ github.ref }} + cancel-in-progress: false + +permissions: + actions: read + contents: read + jobs: - publish: - runs-on: ubuntu-latest - permissions: - contents: read - steps: - - uses: actions/checkout@v6 - - - name: Setup Rust toolchain - run: | - rustup update stable - rustup default stable - - - name: Dry run - run: cargo publish -p paimon-mosaic-core --dry-run - - - name: Publish paimon-mosaic-core to crates.io - if: startsWith(github.ref, 'refs/tags/') && !contains(github.ref, '-') - run: cargo publish -p paimon-mosaic-core - env: - CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }} + rust: + name: Rust release + uses: ./.github/workflows/release-rust.yml + secrets: inherit + + java: + name: Java release + uses: ./.github/workflows/release-java.yml + secrets: inherit + + python-wheels: + name: Python wheels + uses: ./.github/workflows/release-python.yml + secrets: inherit + + python-publish: + name: Python publish + needs: [rust, java, python-wheels] + if: startsWith(github.ref, 'refs/tags/') + uses: ./.github/workflows/release-python-publish.yml + secrets: inherit diff --git a/docs/creating-a-release.html b/docs/creating-a-release.html index 7895d08..8b69a4d 100644 --- a/docs/creating-a-release.html +++ b/docs/creating-a-release.html @@ -69,15 +69,15 @@ </ol> <h3>Automated Publishing</h3> - <p>When a version tag is pushed, GitHub Actions automatically publishes language-specific artifacts:</p> + <p>When a version tag is pushed, the <code>Release</code> workflow orchestrates language-specific release jobs:</p> <table> <thead> - <tr><th>Component</th><th>Tag Pattern</th><th>Published To</th><th>Pre-release (<code>-rc</code>) Behavior</th></tr> + <tr><th>Component</th><th>Tag Pattern</th><th>Published To</th><th>Pre-release (<code>-rc</code>) Behavior</th><th>Ordering</th></tr> </thead> <tbody> - <tr><td>Rust crate</td><td><code>v0.1.0</code></td><td>crates.io</td><td>Dry-run only</td></tr> - <tr><td>Java binding</td><td><code>v0.1.0</code></td><td>Apache Nexus staging</td><td>Deploys to staging</td></tr> - <tr><td>Python binding</td><td><code>v0.1.0</code></td><td>PyPI</td><td>Publishes to TestPyPI</td></tr> + <tr><td>Rust crate</td><td><code>v0.1.0</code></td><td>crates.io</td><td>Dry-run only</td><td>Runs in parallel</td></tr> + <tr><td>Java binding</td><td><code>v0.1.0</code></td><td>Apache Nexus staging</td><td>Deploys to staging</td><td>Runs in parallel</td></tr> + <tr><td>Python binding</td><td><code>v0.1.0</code></td><td>PyPI</td><td>Publishes to TestPyPI</td><td>Publishes only after Rust and Java jobs succeed</td></tr> </tbody> </table> <p>The Release Manager's primary responsibility is managing the <strong>source release</strong> (tarball + signature) and coordinating the community vote. Language artifact publishing is handled by CI once the tag is pushed.</p> @@ -215,12 +215,14 @@ cd ..</code></pre> git tag -v ${RC_TAG} git push origin ${RELEASE_BRANCH} git push origin ${RC_TAG}</code></pre> - <p>After pushing, verify in <a href="https://github.com/apache/paimon-mosaic/actions";>GitHub Actions</a> that all release workflows succeed:</p> + <p>After pushing, verify in <a href="https://github.com/apache/paimon-mosaic/actions";>GitHub Actions</a> that the <code>Release</code> workflow succeeds:</p> <ul> - <li><strong>Release Rust</strong> — dry-run check (does not publish for RC tags)</li> - <li><strong>Release Java</strong> — builds native JNI libraries for 4 platforms, deploys JAR to Apache Nexus staging</li> - <li><strong>Release Python</strong> — builds wheels for 4 platforms, publishes to TestPyPI</li> + <li><strong>Rust release</strong> — dry-run check (does not publish for RC tags)</li> + <li><strong>Java release</strong> — builds native JNI libraries for 4 platforms, deploys JAR to Apache Nexus staging</li> + <li><strong>Python wheels</strong> — builds wheels for 4 platforms</li> + <li><strong>Python publish</strong> — publishes to TestPyPI only after the Rust, Java, and Python wheel jobs succeed</li> </ul> + <p>If only the Python publish job fails due to a transient PyPI or TestPyPI issue, use <strong>Re-run failed jobs</strong> on the <code>Release</code> workflow. The successful Rust, Java, and Python wheel jobs do not need to be repeated.</p> <h3>Create Source Release Artifacts</h3> <p>Create source release artifacts from the same commit as the RC tag:</p> @@ -245,7 +247,7 @@ svn commit -m "Add paimon-mosaic ${RELEASE_VERSION} RC${RC_NUM}"</code></pre> <p><strong>Checklist:</strong></p> <ul> - <li>RC tag pushed and CI workflows succeeded</li> + <li>RC tag pushed and the CI release workflow succeeded</li> <li>Source tarball, signature, and checksum staged to <a href="https://dist.apache.org/repos/dist/dev/paimon/";>dist.apache.org dev</a></li> <li>Java artifacts deployed to Nexus staging repository</li> <li>Python wheels published to TestPyPI</li> @@ -308,7 +310,7 @@ svn commit -m "Remove paimon-mosaic ${RELEASE_VERSION} RC${RC_NUM} (superseded)" <h2 id="finalize-the-release">Finalize the Release</h2> <h3>Push the Release Tag</h3> - <p>Once the vote passes, create and push the final release tag. This triggers CI to publish to crates.io and PyPI automatically.</p> + <p>Once the vote passes, create and push the final release tag. This triggers CI to publish to crates.io and PyPI automatically; PyPI publishing runs only after the Rust and Java release jobs succeed.</p> <pre><code>git checkout ${RC_TAG} git tag -s ${RELEASE_TAG} -m "Release Apache Paimon Mosaic ${RELEASE_VERSION}" git tag -v ${RELEASE_TAG} diff --git a/docs/verifying-a-release-candidate.html b/docs/verifying-a-release-candidate.html index 1ec53e0..6f620de 100644 --- a/docs/verifying-a-release-candidate.html +++ b/docs/verifying-a-release-candidate.html @@ -170,7 +170,7 @@ paimon-mosaic-core = { git = "https://github.com/apache/paimon-mosaic";, tag = "v </ol> <h3>Python (TestPyPI)</h3> - <p>The RC tag publishes wheels to TestPyPI. Install and verify:</p> + <p>The RC tag publishes wheels to TestPyPI after the Rust and Java release jobs succeed. Install and verify:</p> <pre><code>pip install -i https://test.pypi.org/simple/ paimon-mosaic==${RELEASE_VERSION}rc${RC_NUM} python -c "import mosaic; print('OK')"</code></pre> <p>Verify wheels are available for: Linux x86_64, Linux aarch64, macOS aarch64, Windows x86_64.</p>
