andygrove opened a new issue, #5899: URL: https://github.com/apache/datafusion-comet/issues/5899
### What is the problem the feature request solves? Anyone who wants to try a fix or feature before the next Comet release has to build from source, including the Rust native library, which needs a Rust toolchain, protoc and a JDK. Downstream projects that want to test against Comet's main branch have the same problem. Other ASF projects such as Spark and Iceberg publish nightly SNAPSHOT artifacts to https://repository.apache.org/content/repositories/snapshots/ from a scheduled GitHub Actions workflow, using the Nexus credentials that ASF Infra provisions as repository secrets. The repository previously had a tag-triggered workflow that published Docker images to GHCR. It was removed in #4241 after failing for a long time: first within a minute of starting, later as `startup_failure` because it used third-party actions that were not pinned to a commit SHA, and it also built arm64 under QEMU with caching disabled. ### Describe the potential solution Add a scheduled workflow that: - builds `libcomet.so` for linux/amd64 and linux/aarch64 on native runners, inside an Ubuntu 20.04 container so the glibc baseline matches the release builder - assembles the shaded `comet-spark` jars for Spark 3.4 (Scala 2.12), 3.5 (Scala 2.12), 4.0 and 4.1 (Scala 2.13) with both native libraries, the same way `dev/release/build-release-comet.sh` does - deploys them with `./mvnw deploy` using the `NEXUS_USER` / `NEXUS_PW` repository secrets; the `org.apache:apache` parent pom already routes SNAPSHOT deploys to `apache.snapshots.https` - skips nights where `main` has not changed - supports a `dry_run` dispatch that builds and verifies the jars without touching Nexus, so the workflow can be exercised on a fork The installation guide should document where the snapshots live, which artifacts exist, and that they are unreleased builds for testing only. macOS libraries are out of scope: the x86_64 build needs the Apple SDK cross-compile path, which CI cannot use. ### Additional context If the `NEXUS_USER` / `NEXUS_PW` secrets are not yet configured on this repository, an INFRA Jira ticket is needed before the first real publish succeeds. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
