This is an automated email from the ASF dual-hosted git repository. sciabarracom pushed a commit to branch 0.9.0 in repository https://gitbox.apache.org/repos/asf/openserverless.git
commit 834b390be5296d3ad8fb3ce77e847fa4709be337 Author: Michele Sciabarra <[email protected]> AuthorDate: Fri Sep 4 12:11:34 2026 +0200 fixing builds using new folders --- BUILD.md | 86 +++++++++----- build-step.md | 356 ---------------------------------------------------------- runtimes | 2 +- 3 files changed, 60 insertions(+), 384 deletions(-) diff --git a/BUILD.md b/BUILD.md index 61a1c07..2c227ab 100644 --- a/BUILD.md +++ b/BUILD.md @@ -121,31 +121,53 @@ Components carry their own base version prefix where applicable (e.g. the CLI produces `v0.9.0-<timestamp>.SNAPSHOT`). The documentation must show one real tag per case rather than describing the format abstractly. -### 2.3 Where images go +### 2.3 Where images go — no configuration required -The destination registry is decided by **who owns the repository the tag was -pushed to**, not by anything in the tag: +The destination registry and the credentials used to push are **derived +automatically from the repository the tag was pushed to**. Nothing has to be +configured for the normal case, and the documentation must lead with that fact +rather than with a configuration table. -- Tag pushed to the Apache repository → Apache official namespace. -- Tag pushed to a fork → the fork owner's namespace, using the fork's secrets. +The rule: -The documentation must be explicit that a contributor without Apache -credentials is not blocked — forking is the supported path, not a workaround. +| Tag pushed to | Registry | Namespace | Credentials | +|---|---|---|---| +| the Apache repository | `docker.io` | `apache` | secrets `DOCKERHUB_USER` / `DOCKERHUB_TOKEN` | +| any fork | `ghcr.io` | the fork owner's GitHub user | the current GitHub actor + the built-in `GITHUB_TOKEN` | -### 2.4 Registry configuration +So a contributor who forks and pushes a tag gets images at +`ghcr.io/<their-user>/…` with **no secrets to create and no `.env` to write** — +the workflow authenticates as the user running it, using the token GitHub +provides to every Actions run. This must be stated as the default path, not as a +fallback. -Where credentials and namespace come from differs by case, and this is the -single largest source of confusion. The documentation must present it as a -table rather than prose: +### 2.4 Overriding the registry (optional) -| Case | Configured via | Notes | -|---|---|---| -| A — single image | `.env` in the component dir: `REGISTRY`, `NAMESPACE` | `REGISTRY` is `ghcr`, `dockerhub`, or `apache`; the task derives the full image name from it | -| B — runtimes | Repository secrets: `DOCKERHUB_REGISTRY`, `DOCKERHUB_USER`, `DOCKERHUB_TOKEN` | Read as env vars by the Taskfile; must be set in the fork's GitHub settings | -| C — cli | None | No registry involved | +An explicit registry can be selected by setting `DOCKERHUB_REGISTRY` together +with the secrets `DOCKERHUB_USER` and `DOCKERHUB_TOKEN`. The documentation must +present this as an override for people who want their images somewhere other +than their own GHCR namespace — **not** as a required setup step. + +### 2.5 The one exception: runtimes + +Case B is the single place where the automatic path is not sufficient, and the +documentation must call this out explicitly rather than leaving it to be +discovered from a failed build. + +The runtimes build publishes images that do **not** belong to the current +repository. The built-in `GITHUB_TOKEN` is scoped to packages owned by the +repository running the workflow, so it does not carry enough permission for +those pushes. + +Therefore, for runtimes only: + +- `DOCKERHUB_TOKEN` **must** be provided explicitly, and +- it must be a token with **write permission on the target images**, not merely + on the packages of the current repository. -The documentation must say plainly: **for Case B, forking is not enough — the -three secrets must be configured in the fork before any CI build can succeed.** +The documentation must state the symptom this produces when skipped — a +permission/denied error at push time, after a long and otherwise successful +build — so the cause is recognisable from the error alone. --- @@ -163,8 +185,9 @@ The documentation must cover: is tagged with the timestamp and exists only in the local Docker daemon. - **CI flow** — `task ci` pushes the tag; the `image` workflow builds multi-arch (`linux/amd64`, `linux/arm64`) and pushes. -- **Fork setup** — set `REGISTRY` and `NAMESPACE` in `.env`. Show the two common - values (`ghcr` → `ghcr.io/<user>/…`, `dockerhub` → `docker.io/<user>/…`). +- **Fork setup** — nothing to do. Per section 2.3 the images go to + `ghcr.io/<your-user>/…` automatically, authenticated with the built-in + `GITHUB_TOKEN`. Mention the `DOCKERHUB_REGISTRY` override only as an aside. - **What CI does beyond building** — license check (skywalking-eyes) and unit tests run before the image is built. A build can fail for reasons unrelated to the image; the documentation must say so, so failures are not mysterious. @@ -193,10 +216,15 @@ documentation must call out as the *entire* reason this case is separate: <registry>/<namespace>/openserverless-runtime-<rt>:<ver>-<tag> ``` -3. **Secrets are mandatory.** Unlike Case A, registry configuration is not in a - local `.env` — it comes from repository secrets. The documentation must list - `DOCKERHUB_REGISTRY`, `DOCKERHUB_USER`, `DOCKERHUB_TOKEN` and say where to - set them (fork → Settings → Secrets and variables → Actions). +3. **A token must be supplied.** This is the one case where the automatic + credentials of section 2.3 are not enough: the images published are not the + current repository's own packages, so the built-in `GITHUB_TOKEN` lacks the + permission. A `DOCKERHUB_TOKEN` with write access to the target images must + be set in the fork (Settings → Secrets and variables → Actions), optionally + alongside `DOCKERHUB_REGISTRY` and `DOCKERHUB_USER` to change the + destination. The documentation must present this as *the* prerequisite for + Case B, and as the only place in the whole build system where a secret is + mandatory. The documentation must also explain the relationship between the **common base image** and the language runtimes: the base is built and tagged separately, and @@ -313,9 +341,13 @@ the documentation cannot be written without them: 1. **One tag encoding or two.** Either every component uses the letter-encoded timestamp, or the plain numeric one. The current mix must not be documented as if it were intentional. -2. **Where registry configuration lives.** Case A reads `.env`, Case B reads - repository secrets. Either this difference is justified and documented as a - rule, or the two are unified. +2. **Convergence on the automatic registry rule.** Section 2.3 is the target: + registry, namespace, and credentials derived from the repository owner, with + no configuration in the normal case. The single-image Taskfiles currently + require `REGISTRY` and `NAMESPACE` in a local `.env` and fail without them; + they must be changed to fall back to the automatic behaviour and treat + `DOCKERHUB_REGISTRY` / `DOCKERHUB_USER` / `DOCKERHUB_TOKEN` as an override. + The runtimes exception (2.5) stays. Until these are settled, the documentation must not describe the current behaviour as stable. diff --git a/build-step.md b/build-step.md deleted file mode 100644 index e2d2d4d..0000000 --- a/build-step.md +++ /dev/null @@ -1,356 +0,0 @@ -# Build Session Notes - -Date: 2026-06-27 - -## Scope - -Work was done locally only. No git push and no Docker Hub push were performed. - -## Branches And Submodules - -- Root repository branch: `build` -- `build` submodule branch: `build-local` -- `build/openwhisk` submodule branch: `openwhisk-build-fixes` - -## OpenWhisk Source - -The nested `build/openwhisk` submodule was moved to the Nuvoplugins commit: - -```text -a456219d35bfabe4a3458fcce82cd71497a8524e -``` - -`build/.gitmodules` was updated to use: - -```text -https://github.com/nuvoplugins/openwhisk.git -``` - -## Build Devcontainer - -The correct build devcontainer is: - -```text -build/.devcontainer/Dockerfile -``` - -The top-level `devcontainer` submodule was touched by mistake, then reverted. It is not part of the final build path. - -The build devcontainer was updated to be multi-arch and OpenWhisk-ready: - -- Java 11 via Temurin is the default `JAVA_HOME` -- GraalVM Java 17 is still available for native-image -- Docker CLI and buildx are installed -- Docker repo setup uses the Ubuntu codename instead of a hardcoded `bionic` - -Verified inside the container: - -```text -java 11.0.17 -javac 11.0.17 -native-image 17.0.9 -docker CLI available -task available -``` - -## Gradle Fixes - -`task distdocker` initially failed with dependency resolution errors in the old OpenWhisk commit. - -Backported minimal Gradle fixes in `build/openwhisk`: - -- `build.gradle`: use `gradle.plugin.cz.alenkacz:gradle-scalafmt` -- `settings.gradle`: `scalafmt` version `1.5.1` -- `core/standalone/build.gradle`: `gradle-git-properties` version `2.4.2` - -These fixes are committed locally in: - -```text -f2cba906 Fix OpenWhisk distDocker dependency resolution -``` - -## Java 17 Host Error - -Running OpenWhisk Gradle directly on the host failed with: - -```text -Unsupported class file major version 61 -``` - -Reason: host Java was 17, but this OpenWhisk/Gradle 6.9.1 build must run with Java 11. - -Fix: `build/Taskfile.yml` now wraps `distdocker` and `build-all` with `ops ide devcontainer`, so the host uses the same devcontainer path expected by OPS instead of a hand-written `docker run`. - -The wrapper also sets a temporary empty `DOCKER_CONFIG` inside the devcontainer. -This avoids failures from a devcontainer-generated Docker credential helper, for -example: - -```text -error getting credentials - err: exit status 255 -``` - -## Local Docker Build Tasks - -`build/Taskfile.yml` was changed so: - -- `task build` is the main local flow: build images, update OPS root, load kind, roll out controller -- `task distdocker` runs through `ops ide devcontainer` -- `task build-all` runs through `ops ide devcontainer` -- `task build-all:local` checks that `openwhisk/common/scala` exists and prints a submodule init hint if it does not -- `distdocker` no longer passes a Docker registry by default -- `build-all` no longer uses `--push` for `ACT=build` -- Push remains explicit via `buildx-and-push` / `ACT=buildx-push` - -Verified: - -```text -cd /home/msciab/openserverless/build -task build -``` - -This completed successfully locally through `ops ide devcontainer`. It built the -OpenWhisk images, loaded controller/invoker/standalone into kind, and rolled out -`controller-0`. - -Latest verified tag: - -```text -2.0.0-incubating.2606291403 -``` - -Latest verified pod image: - -```text -controller-0 registry.hub.docker.com/apache/openserverless-wsk-controller:2.0.0-incubating.2606291403 -``` - -## OpenWhisk Directory - -`build/openwhisk` was converted from a nested Git submodule into a regular -tracked directory inside the `build-local` branch. - -Reason: on a fresh server, `task build` failed when Docker tried to build: - -```text -openwhisk/common/scala -``` - -The directory was missing because the nested OpenWhisk submodule had not been -initialized. The `build-local` branch now vendors the OpenWhisk tree from commit -`f2cba906`, including the local Gradle fixes, so a checkout of `build-local` -contains `openwhisk/common/scala` directly. - -## Images Built Locally - -The OpenWhisk build produced these local Docker images: - -```text -registry.hub.docker.com/apache/openserverless-wsk-scala:2.0.0-incubating.2506080813 -registry.hub.docker.com/apache/openserverless-wsk-controller:2.0.0-incubating.2506080813 -registry.hub.docker.com/apache/openserverless-wsk-invoker:2.0.0-incubating.2506080813 -registry.hub.docker.com/apache/openserverless-wsk-scheduler:2.0.0-incubating.2506080813 -registry.hub.docker.com/apache/openserverless-wsk-standalone:2.0.0-incubating.2506080813 -``` - -Despite the `registry.hub.docker.com/apache` prefix, these were local tags only. They were not pushed. - -## OPS Root Image Override - -The installed OPS root was updated here: - -```text -/home/msciab/.ops/0.1.0/oplugins/opsroot.json -``` - -Changed keys: - -```json -{ - "standalone": "registry.hub.docker.com/apache/openserverless-wsk-standalone:2.0.0-incubating.2506080813", - "controller": "registry.hub.docker.com/apache/openserverless-wsk-controller:2.0.0-incubating.2506080813", - "invoker": "registry.hub.docker.com/apache/openserverless-wsk-invoker:2.0.0-incubating.2506080813" -} -``` - -Important: in the current standalone OpenWhisk deployment path, OPS/operator uses `controller.image` for the controller pod. The `standalone` key is present in `opsroot.json`, but the observed `Whisk` resource uses `spec.controller.image`. - -## Kubeconfig Repair - -The host kubeconfig was stale: - -```text -https://127.0.0.1:35889 -``` - -The running kind control plane was exposed on: - -```text -https://127.0.0.1:34101 -``` - -Backed up kubeconfig: - -```text -/home/msciab/.kube/config.bak.20260627085813 -``` - -Updated context: - -```text -kind-nuvoplugins -``` - -Verified: - -```text -kubectl cluster-info -kubectl get nodes -``` - -## Loading Images Into Kind - -This is now automated by the build repository task: - -```bash -cd /home/msciab/openserverless/build -task build -``` - -That task runs: - -1. `task build-all` -2. update of `/home/msciab/.ops/0.1.0/oplugins/opsroot.json` -3. import of local OpenWhisk images into the kind node -4. rollout of the local OpenWhisk controller StatefulSet -5. final status output - -The locally built images were not automatically present in the kind node. - -The underlying import command is: - -```bash -for img in \ - registry.hub.docker.com/apache/openserverless-wsk-controller:2.0.0-incubating.2506080813 \ - registry.hub.docker.com/apache/openserverless-wsk-invoker:2.0.0-incubating.2506080813 \ - registry.hub.docker.com/apache/openserverless-wsk-standalone:2.0.0-incubating.2506080813 -do - docker save "$img" | docker exec -i nuvoplugins-control-plane ctr --namespace=k8s.io images import - -done -``` - -Verified in kind: - -```text -registry.hub.docker.com/apache/openserverless-wsk-controller 2.0.0-incubating.2506080813 -registry.hub.docker.com/apache/openserverless-wsk-invoker 2.0.0-incubating.2506080813 -registry.hub.docker.com/apache/openserverless-wsk-standalone 2.0.0-incubating.2506080813 -``` - -## Current Pod Image - -The `Whisk` CR had the new image: - -```text -spec.controller.image = registry.hub.docker.com/apache/openserverless-wsk-controller:2.0.0-incubating.2506080813 -``` - -But the actual StatefulSet/pod initially still used: - -```text -ghcr.io/nuvoplugins/openwhisk-controller:0.3.0-morpheus.22122609 -``` - -This is now automated by: - -```bash -cd /home/msciab/openserverless/build -task build -``` - -The underlying patch command is: - -```bash -kubectl -n nuvoplugins set image statefulset/controller \ - controller=registry.hub.docker.com/apache/openserverless-wsk-controller:2.0.0-incubating.2506080813 -``` - -Rollout completed and `controller-0` now uses: - -```text -registry.hub.docker.com/apache/openserverless-wsk-controller:2.0.0-incubating.2506080813 -``` - -## Why `task build` / `task setup` Did Not Do This Automatically - -There are two separate build systems: - -1. Root `Taskfile.yml` - - Builds CLI, operator, runtimes, streamer, admin-api. - - Its `opsroot` task updates only: - - `operator` - - `streamer` - - `systemapi` - - It does not build or update OpenWhisk `controller` / `invoker` / `standalone`. - - It saves some images into `~/.ops/<os>-<arch>/images/kind`, but not the OpenWhisk images produced by `build/Taskfile.yml`. - -2. `build/Taskfile.yml` - - Builds OpenWhisk images: - - controller - - invoker - - scheduler - - standalone - - scala base - - Before this session, it did not update OPS `opsroot.json`. - - Before manual loading, it did not load these images into kind. - -Also, the operator/standalone image replacement appears to miss the new image when rendering the StatefulSet. - -Observed behavior: - -- The `Whisk` CR contains `spec.controller.image` with the new image. -- The generated StatefulSet still had the old template image. - -Likely technical cause: - -- `oplugins-op/nuvoplugins/openwhisk_standalone.py` calls `kus.image(whisk_image, newTag=whisk_tag)`. -- Kustomize image replacement matches the image name already present in the template. -- The template contains `ghcr.io/nuvoplugins/openwhisk-controller`. -- The generated kustomization uses the new image as the match name, so it does not match the old template image. - -Proper code-level fix should replace the template image name with `newName`, for example: - -```text -name: ghcr.io/nuvoplugins/openwhisk-controller -newName: registry.hub.docker.com/apache/openserverless-wsk-controller -newTag: 2.0.0-incubating.2506080813 -``` - -## Local Commits Created - -Root repo: - -```text -bd8b2b4 Update build-all local workflow -1796e13 Update build distdocker container wrapper -35ae719 Update build distdocker local workflow -d4d269d Update build submodule devcontainer toolchain -d781ddb Update build submodule for Nuvoplugins openwhisk source -d758eab Point build submodule to local openwhisk update -``` - -Build submodule: - -```text -3326ff6 Make build-all local and containerized -f591434 Run distdocker through build container -13ef155 Make distdocker local-only by default -3492964 Make build devcontainer multi-arch OpenWhisk ready -2c53f0f Use Nuvoplugins openwhisk submodule source -2fd1cf8 Point openwhisk to Nuvoplugins action size change -``` - -OpenWhisk submodule: - -```text -f2cba906 Fix OpenWhisk distDocker dependency resolution -``` diff --git a/runtimes b/runtimes index d86fcf0..4095752 160000 --- a/runtimes +++ b/runtimes @@ -1 +1 @@ -Subproject commit d86fcf08f71037f908dbcd8da0e24dc4df7a754a +Subproject commit 40957528ee8cfc7855b6e4ca1ef43b4243c32073
