shunping commented on PR #40690: URL: https://github.com/apache/beam/pull/40690#issuecomment-6298836891
> sdks/python/apache_beam/options/pipeline_options.py (line 1280): 🟡 Medium: When --prebuild_sdk_container_engine is used on Dataflow without --sdk_container_image, sdk_container_builder._get_cloud_build_id_and_image_name (in sdks/python/apache_beam/runners/portability/sdk_container_builder.py) resolves the base image via environments.DockerEnvironment.default_docker_image() and only rewrites the apache/beam prefix to names.DATAFLOW_CONTAINER_IMAGE_REPOSITORY when sdk_location == 'default'. As a result, prebuilding a container with --dataflow_container_base_repository set will still pull the base image from gcr.io/cloud-dataflow/v1beta3 (or apache/beam), which will fail in restricted environments where only the custom mirror repository is accessible. Consider using apiclient.get_container_image_from_options(pipeline_options) (or get_container_base_repository_from_options) when determining the default base image in sdk_container_builder.py. Good catch on the gap, though the details differ: the prebuild base image comes from `SdkContainerImageBuilder.__init__` (`sdk_container_image` or `apache/beam_python3.X_sdk:<ver>` on Docker Hub), not from `gcr.io` or a non-existent `_get_cloud_build_id_and_image_name` helper. Fixed in dataflow_runner.py > runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/options/DataflowPipelineWorkerPoolOptions.java (line 150): 🟢 Low: For consistency with other optional string getters in this interface (and since getDataflowContainerBaseRepository() returns null when unset, as checked in DataflowRunner.getContainerImageForJob), consider annotating the return type with @Nullable. Done: annotated the getter and setter with @Nullable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
