akashorabek commented on code in PR #34250:
URL: https://github.com/apache/beam/pull/34250#discussion_r2000999932
##########
.github/workflows/beam_PreCommit_Flink_Container.yml:
##########
@@ -77,6 +77,10 @@ env:
HARNESS_IMAGES_TO_PULL:
gcr.io/apache-beam-testing/beam-sdk/beam_go_sdk:latest
JOB_SERVER_IMAGE:
gcr.io/apache-beam-testing/beam_portability/beam_flink1.17_job_server:latest
ARTIFACTS_DIR: gs://beam-flink-cluster/beam-precommit-flink-container-${{
github.run_id }}
+ DOCKER_REGISTRY: gcr.io
+ DOCKER_REPOSITORY_ROOT: ${{ github.event_name == 'pull_request_target' &&
'gcr.io/apache-beam-testing/beam-sdk-pr' ||
'gcr.io/apache-beam-testing/beam-sdk' }}
Review Comment:
When changing the repository from public gcr.io to private us.gcr.io, there
is an error:
`RuntimeError: Pipeline
flink-tests-python-0317062609_6454c5c2-eb5f-4fed-9c51-3d7e2df3e1bf failed in
state FAILED: java.io.IOException: Received exit code 125 for command 'docker
run -d --mount
type=bind,src=/var/lib/hadoop-yarn/.config/gcloud,dst=/root/.config/gcloud
--network=host --env=DOCKER_MAC_CONTAINER=null
us.gcr.io/apache-beam-testing/github-actions/beam_python3.9_sdk:2.64.0-SNAPSHOT
--id=1-1 --provision_endpoint=localhost:40945'. stderr: Unable to find image
'us.gcr.io/apache-beam-testing/github-actions/beam_python3.9_sdk:2.64.0-SNAPSHOT'
locallydocker: Error response from daemon: Head
"https://us.gcr.io/v2/apache-beam-testing/github-actions/beam_python3.9_sdk/manifests/2.64.0-SNAPSHOT":
denied: Unauthenticated request. Unauthenticated requests do not have
permission "artifactregistry.repositories.downloadArtifacts" on resource
"projects/apache-beam-testing/locations/us/repositories/us.gcr.io" (or it may
not exist). See 'docker run --help'.`
Tried different methods, including the command `gcloud auth configure-docker
us.gcr.io --quiet.` The same error occurred when trying to push a snapshot to
the private repository, but adding that command to the workflow YAML helped.
But now the error happens in the test itself when it tries to pull the image. I
ran that command on all Dataproc workers in the `flink_cluster.sh` file,
including for the `yarn` user, because based on
`src=/var/lib/hadoop-yarn/.config/gcloud` in the error message, it's `yarn`
that runs the test. But that didn't help. It's strange because when I connect
to these Dataproc workers locally, they can do a docker pull from `us.gcr.io`,
but when the workflow runs on GitHub Actions, the test fails. Can we, for now,
keep using gcr.io(we use beam-sdk-pr instead of beam-sdk and delete the
container after tests finish) and create an issue for further investigation of
this since it might take longer than expected?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
