zgxme opened a new issue, #70816:
URL: https://github.com/apache/doris/issues/70816

   ### Background
   
   External-regression fixtures need images from Docker Hub and other 
registries. CI environments often use a local registry, such as Harbor, for 
caching or mirrored images.
   
   ### Current behavior
   
   Image references are embedded in Compose templates, the Kerberos Dockerfile 
and auxiliary container calls. A CI adapter must rewrite these references to 
route all image requests through its registry. Changing only Compose images 
misses build bases and filesystem/baseline helpers.
   
   ### Expected behavior
   
   Operators can configure image sources without rewriting templates. Default 
settings preserve existing references; an explicit mapping can resolve 
`mysql:5.7.36` to `harbor.example.com:443/dockerhub/library/mysql:5.7.36`, 
including helpers and build bases under the same policy.
   
   ### Proposed design
   
   Add three settings in `docker/thirdparties/custom_settings.env`, allowing 
environment overrides:
   
   ```bash
   export THIRDPARTY_IMAGE_MODE=mapped
   export THIRDPARTY_IMAGE_REGISTRY=harbor.example.com:443
   export 
THIRDPARTY_IMAGE_REGISTRY_MAP='docker.io=dockerhub,ghcr.io=ghcr,quay.io=quay,icr.io=icr,cgr.dev=cgr,mcr.microsoft.com=mcr'
   ```
   
   Use an explicit component image catalog and a shared resolver. `native` is 
the default and preserves references. `mapped` applies registry/project 
mappings after user and platform defaults, preserves tags/digests, rejects 
missing mappings, and keeps local build outputs local. References already under 
the target registry are explicit overrides and stay unchanged.
   
   Compose consumes exported image variables; Dockerfile bases use build 
arguments; standalone helper tools load the same configuration. Provide 
`--list-images`, `--pull-images` and `--image-config-version`. Mapped starts 
fetch missing images before modifying fixtures; stopping requires no registry 
probes.
   
   A Harbor-specific mode name would unnecessarily couple the interface to one 
product. Continuing runtime source rewriting would leave image dependencies 
implicit and require separate handling for build and helper paths. Generic 
explicit mapping addresses both concerns.
   
   ### Impact
   
   This affects developers and CI operators deploying `docker/thirdparties`. It 
does not change SQL behavior or FE/BE execution in either shared-nothing or 
cloud deployments.
   
   ### Reproduction or evidence
   
   Relevant entry points are `run-thirdparties-docker.sh`, `docker-command.sh`, 
`docker-compose/kerberos/Dockerfile` and 
`docker-compose/hive/scripts/snapshot-hive-baseline.sh`.
   
   The implementation's default inventory contains 51 declarations, including 
49 distinct remote references. Local contract tests and a live Harbor 
experiment verified manifest/config access for all 49 references and 13 
representative complete pulls. The live Docker daemon was rootful; rootless 
fixture startup and full external regression remain unverified.
   
   ### Scope and non-goals
   
   Cover service images, build bases and auxiliary containers under 
`docker/thirdparties`. Cloud-data access, artifact repositories, host aliases, 
Docker provisioning and external CI adapters remain separately managed. IPv6 
registry literals are unsupported in the proposed configuration.
   
   ### Acceptance criteria
   
   - Native mode preserves existing image defaults and platform-specific 
selection.
   - Mapped mode covers Compose, build-base and helper image references, with 
tags/digests preserved.
   - Missing mappings and pull failures are reported before fixture mutation.
   - Image inventory and interface-version queries require no Docker daemon; 
stop requires no registry probes.
   - CI tests cover configuration errors, component selection, explicit target 
overrides, template coverage and Compose interpolation.
   
   ### Risks and compatibility
   
   No FE/BE protocols or persisted data formats change. Existing Docker 
authentication and certificate configuration remain necessary. CI adapters must 
detect the new interface before disabling rewriting for new packages and retain 
support for older packages. Direct consumers of generated Compose files need to 
initialize image variables through the launcher.
   
   ### Related
   
   The implementation PR will link this issue. No earlier public design 
discussion is known.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to