TalaatHarb opened a new issue, #4038:
URL: https://github.com/apache/iggy/issues/4038
### Description
## Motivation
Apache Iggy currently publishes compiled binaries (`iggy-server`, `iggy`,
`iggy-bench`, `iggy-connectors`) and connector plugin `.so` shared libraries
**only** to the rolling `:edge` GitHub pre-release on master commits
(`.github/workflows/edge-release.yml`).
When official/versioned releases are published via
`.github/workflows/publish.yml`, artifacts are distributed to package
registries:
- Rust crates -> crates.io
- Container images -> Docker Hub (`apache/iggy`, `apache/iggy-connect`, etc.)
- Python SDK -> PyPI
- Node SDK -> npm
- Java SDK -> Maven Central / ASF Nexus
- C# SDK -> NuGet
- Go SDK -> Git tag
However, **GitHub Releases for versioned tags (e.g. `server-0.8.0`,
`cli-0.8.0`, etc.) do not include downloadable tarballs / binary assets**.
Users who want standalone binaries without Docker, `cargo install`, or building
from source cannot currently download pre-compiled tarballs from GitHub
Releases. (AI Disclosure: GitHub copilot helped in the investigation and
drafting the issue)
Providing downloadable binary tarballs and connector plugins on GitHub
Releases significantly improves developer adoption, simplifies CI integration
in downstream projects, and provides a quick onboarding path for users
evaluating Iggy without Docker.
---
## Current Architecture & Existing Components
The repository already has all the required building blocks in place:
1. **Reusable Build Workflow
(`.github/workflows/_build_rust_artifacts.yml`)**:
- Compiles release binaries across cross-platform matrix:
- `x86_64-unknown-linux-gnu` (glibc)
- `x86_64-unknown-linux-musl` (musl static)
- `aarch64-unknown-linux-gnu` (glibc)
- `aarch64-unknown-linux-musl` (musl static)
- Compiles connector plugins (`.so` shared libraries) for:
- `x86_64-unknown-linux-gnu`
- `aarch64-unknown-linux-gnu`
- Generates tarballs:
- `iggy-${target}-${version}.tar.gz` (contains `iggy-server`, `iggy`,
`iggy-bench`, `iggy-connectors`)
- `iggy-connectors-${target}-${version}.tar.gz` (contains plugin `.so`
files)
- Packages and uploads a combined artifact `rust-artifacts-all`.
2. **Edge Release Workflow (`.github/workflows/edge-release.yml`)**:
- Calls `_build_rust_artifacts.yml` with `version: edge`.
- Downloads `rust-artifacts-all`.
- Uses `softprops/[email protected]` to upload
`artifacts/*.tar.gz` to the rolling `edge` pre-release.
3. **Release Publisher (`.github/workflows/publish.yml`)**:
- Central entry point for manual (`workflow_dispatch`) and automated
post-merge releases.
- Pushes Docker manifests, publishes packages to crates.io / PyPI / npm /
NuGet / Maven, and creates corresponding Git tags using
`.github/actions/utils/create-git-tag`.
- **Missing step**: It does not build or upload the compiled Rust
binary/plugin tarballs as GitHub Release assets.
---
## What is Needed to Enable Release Binary Artifacts
To attach downloadable tarballs to tagged releases, the following additions
are needed:
### 1. Update `.github/workflows/publish.yml`
- When `publish_dockerhub` includes `rust-server` or when server/CLI
releases are published:
- Invoke `_build_rust_artifacts.yml` passing `version: ${{
steps.ver.outputs.version }}` and `commit: ${{ needs.validate.outputs.commit
}}`.
- Add a release step (using `gh release create` / `gh release upload` or
`softprops/action-gh-release`) to create/update the GitHub Release for the
server/CLI tag (e.g. `server-vX.Y.Z` or `vX.Y.Z`) and attach the generated
tarballs (`iggy-*.tar.gz`, `iggy-connectors-*.tar.gz`).
- Generate and attach SHA256 checksums (`SHA256SUMS.txt`) for all tarballs
to adhere to ASF release integrity recommendations.
### 2. Optional: Add Windows and macOS Binary Targets (when support matures)
- Once Windows native compilation blockers
(#<insert-windows-build-issue-number>) and macOS targets are addressed:
- Add `x86_64-pc-windows-msvc` and `x86_64-apple-darwin` /
`aarch64-apple-darwin` matrix rows to `_build_rust_artifacts.yml`.
- Package `.zip` for Windows and `.tar.gz` for macOS/Linux.
### 3. Checksum & Provenance
- Include a checksum generation step:
```bash
sha256sum *.tar.gz > SHA256SUMS.txt
```
- Optionally enable GitHub Artifact Attestations
(`actions/attest-build-provenance`) for supply chain security.
---
## Suggested Acceptance Criteria
- [ ] Publishing a release via `publish.yml` creates a GitHub Release (or
updates the tag's release) containing downloadable `.tar.gz` tarballs for Linux
x86_64 and aarch64 (glibc and musl).
- [ ] Each release includes `SHA256SUMS.txt` for integrity verification.
- [ ] Tarballs contain all primary binaries (`iggy-server`, `iggy`,
`iggy-bench`, `iggy-connectors`) and connector plugins.
- [ ] Download links are documented or visible on the GitHub Releases page
for easy end-user consumption.
### Affected area / component
_No response_
### Proposed solution
_No response_
### Alternatives considered
_No response_
### Contribution
- [ ] I'm willing to submit a pull request to implement this feature
### Good first issue
- [ ] I think this could be a good first issue for a new contributor
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
