tanishqgandhi1908 opened a new pull request, #8475:
URL: https://github.com/apache/texera/pull/8475
### What changes were proposed in this PR?
Every computing unit runs the same image, fixed when the cluster is
installed. ML Models needing a different Python version, a system package, or
a library built from source cannot run, because a Python virtual environment
only holds pip packages.
This lets an administrator register an image reference from a public
registry, and a computing unit can then be started from it. **Off by default**
(`curatedImages.enabled: false`) until the UI to manage these ships in #8470
and #8471.
**How it works.** Texera reads the image's manifest and config blob — a few
kilobytes, never the layers — to check its start command runs
`computing-unit-master`, which means it was built `FROM` the Texera
computing-unit image, and to resolve the digest behind the reference. A
misspelled, private or unsuitable image is refused in seconds, in front of the
administrator, rather than when a user's unit will not start.
The row records `owner/name@sha256:…`, and that is what units run, so a tag
its owner moves later cannot change what already ran. **Nothing is copied and
no registry is added** — units pull the reference the same way the deployment's
own image is already pulled.
**Uniqueness is enforced by the database**, not only checked in the service:
two administrators registering the same link at the same moment both pass a
read-then-write check and produce two rows for one image.
**Non-root, for curated images only.** A curated image was supplied by an
administrator and reviewed by nobody, so a unit started from one runs as a
non-root user with no privilege escalation and no capabilities. The
deployment's own image is untouched — it is its operator's choice, and a
deployment that has replaced it with an image needing root would break on
upgrade.
**Known limitation.** The first unit on each node waits for the image to
download — about 80 seconds for a 3 GB one — while later units on that node
start at once. Pre-pulling ready images onto nodes is #8469.
### Any related issues, documentation, discussions?
Closes #8468
Part of #8466
### How was this PR tested?
Unit tests, chart rendering, and a deployment to minikube exercising both
states.
```
sbt "ComputingUnitManagingService/testOnly
org.apache.texera.service.resource.CuratedImageResourceSpec"
"ComputingUnitManagingService/testOnly
org.apache.texera.service.util.KubernetesClientSpec"
"Config/testOnly org.apache.texera.common.config.KubernetesConfigSpec"
scalafmtCheckAll
```
CuratedImageResourceSpec 27 passed
KubernetesClientSpec 17 passed
KubernetesConfigSpec 6 passed
scalafmtCheckAll clean
`helm template` renders with the feature off and on; the manager Role gains
`jobs` and `pods/log` only.
**Deployed to minikube, feature off:**
```
GET /api/cu-image 503 "Curated images are not enabled on this deployment."
POST /api/cu-image 503
create unit with iid 403 "Image 1 is not available..."
create unit without iid 200 deployment's own image, no security context
```
**Feature on:**
register a good image READY in 8s, pinned to @sha256:bdeadc3c...
duplicate reference 400 names the existing row
duplicate name 400
empty name 400
tag that does not exist FAILED, log names the tag and what to do instead
alpine (not a CU image) FAILED, "Its start command is: [/bin/sh]"
unit from a curated image
image tagandhi19/texera-cu-sklearn@sha256:bdeadc3c...
security {allowPrivilegeEscalation:false, capabilities:{drop:[ALL]},
runAsNonRoot:true, runAsUser:1001}
id uid=1001(texera)
delete the image while a unit runs on it
204, pod stays Running, unit still reports imageName 'Python ML'
a new unit from it 403
### Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
