zgxme opened a new issue, #70816: URL: https://github.com/apache/doris/issues/70816
### Background External-regression fixtures need images from Docker Hub and other registries. CI environments often use a local registry, such as Harbor, for caching or mirrored images. ### Current behavior Image references are embedded in Compose templates, the Kerberos Dockerfile and auxiliary container calls. A CI adapter must rewrite these references to route all image requests through its registry. Changing only Compose images misses build bases and filesystem/baseline helpers. ### Expected behavior Operators can configure image sources without rewriting templates. Default settings preserve existing references; an explicit mapping can resolve `mysql:5.7.36` to `harbor.example.com:443/dockerhub/library/mysql:5.7.36`, including helpers and build bases under the same policy. ### Proposed design Add three settings in `docker/thirdparties/custom_settings.env`, allowing environment overrides: ```bash export THIRDPARTY_IMAGE_MODE=mapped export THIRDPARTY_IMAGE_REGISTRY=harbor.example.com:443 export THIRDPARTY_IMAGE_REGISTRY_MAP='docker.io=dockerhub,ghcr.io=ghcr,quay.io=quay,icr.io=icr,cgr.dev=cgr,mcr.microsoft.com=mcr' ``` Use an explicit component image catalog and a shared resolver. `native` is the default and preserves references. `mapped` applies registry/project mappings after user and platform defaults, preserves tags/digests, rejects missing mappings, and keeps local build outputs local. References already under the target registry are explicit overrides and stay unchanged. Compose consumes exported image variables; Dockerfile bases use build arguments; standalone helper tools load the same configuration. Provide `--list-images`, `--pull-images` and `--image-config-version`. Mapped starts fetch missing images before modifying fixtures; stopping requires no registry probes. A Harbor-specific mode name would unnecessarily couple the interface to one product. Continuing runtime source rewriting would leave image dependencies implicit and require separate handling for build and helper paths. Generic explicit mapping addresses both concerns. ### Impact This affects developers and CI operators deploying `docker/thirdparties`. It does not change SQL behavior or FE/BE execution in either shared-nothing or cloud deployments. ### Reproduction or evidence Relevant entry points are `run-thirdparties-docker.sh`, `docker-command.sh`, `docker-compose/kerberos/Dockerfile` and `docker-compose/hive/scripts/snapshot-hive-baseline.sh`. The implementation's default inventory contains 51 declarations, including 49 distinct remote references. Local contract tests and a live Harbor experiment verified manifest/config access for all 49 references and 13 representative complete pulls. The live Docker daemon was rootful; rootless fixture startup and full external regression remain unverified. ### Scope and non-goals Cover service images, build bases and auxiliary containers under `docker/thirdparties`. Cloud-data access, artifact repositories, host aliases, Docker provisioning and external CI adapters remain separately managed. IPv6 registry literals are unsupported in the proposed configuration. ### Acceptance criteria - Native mode preserves existing image defaults and platform-specific selection. - Mapped mode covers Compose, build-base and helper image references, with tags/digests preserved. - Missing mappings and pull failures are reported before fixture mutation. - Image inventory and interface-version queries require no Docker daemon; stop requires no registry probes. - CI tests cover configuration errors, component selection, explicit target overrides, template coverage and Compose interpolation. ### Risks and compatibility No FE/BE protocols or persisted data formats change. Existing Docker authentication and certificate configuration remain necessary. CI adapters must detect the new interface before disabling rewriting for new packages and retain support for older packages. Direct consumers of generated Compose files need to initialize image variables through the launcher. ### Related The implementation PR will link this issue. No earlier public design discussion is known. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
