Alan Myrvold created BEAM-3584:
----------------------------------
Summary: Java dataflow job fails with 2.3.0 RC1, due to missing
worker image
Key: BEAM-3584
URL: https://issues.apache.org/jira/browse/BEAM-3584
Project: Beam
Issue Type: Bug
Components: examples-java
Affects Versions: 2.3.0
Reporter: Alan Myrvold
Assignee: Alan Myrvold
Dataflow java jobs currently fail due to a missing docker image when using
2.3.0 RC1. This is not a bug in the SDK, the worker image needs to be published
by google. I will be coordinating the worker image publication.
# Update to your own project and bucket.
GCS_BUCKET=my-cloud-storage-bucket
GCP_PROJECT=my-cloud-project
mkdir -p ~/try2.3.0/.m2/repository
cd ~/try2.3.0/.m2/repository
wget
[https://repository.apache.org/content/repositories/orgapachebeam-1026/archetype-catalog.xml]
cd ~/try2.3.0
echo
"<settings><localRepository>$PWD/.m2/repository</localRepository><profiles><profile><id>testrel</id><repositories><repository><id>testrel</id><url>[https://repository.apache.org/content/repositories/orgapachebeam-1026]</url></repository></repositories></profile></profiles></settings>"
> settings.xml
mvn archetype:generate -DarchetypeCatalog=local
-DarchetypeGroupId=org.apache.beam
-DarchetypeArtifactId=beam-sdks-java-maven-archetypes-examples
-DarchetypeVersion=2.3.0 -DgroupId=org.example
-DartifactId=word-count-beam -Dversion="0.1"
-Dpackage=org.apache.beam.examples -s settings.xml
# answer yes
cd word-count-beam
mvn compile exec:java -Dexec.mainClass=org.apache.beam.examples.WordCount
-Dexec.args="--runner=DataflowRunner --project=alan-jenkins-test
--gcpTempLocation=gs://${GCS_BUCKET}/tmp
--inputFile=gs://apache-beam-samples/shakespeare/*
--output=gs://${GCP_PROJECT}e/counts" -s ../settings.xml
-Ptestrel,dataflow-runner
Dataflow logs contain:
E Handler for GET
/v1.27/images/dataflow.gcr.io/v1beta3/beam-java-batch:beam-2.3.0/json returned
error: No such image: dataflow.gcr.io/v1beta3/beam-java-batch:beam-2.3.0
E PullImage "dataflow.gcr.io/v1beta3/beam-java-batch:beam-2.3.0" from image
service failed: rpc error: code = 2 desc = Error: Status 405 trying to pull
repository v1beta3/beam-java-batch: "v1 Registry API is disabled. If you are
not explicitly using the v1 Registry API, it is possible your v2 image could
not be found. Verify that your image is available, or retry with `dockerd
--disable-legacy-registry`. See
https://cloud.google.com/container-registry/docs/support/deprecation-notices";
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
