See
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2_java17/338/display/redirect?page=changes>
Changes:
[dhuntsperger] documented maven-to-gradle conversion for Dataflow; refactored
java
[dhuntsperger] adding a list of example pipelines
[dhuntsperger] removing unnecessary `ls` command from maven project generation
[dhuntsperger] fixing filename formatting in response to feedback
[dhuntsperger] adding extra step emphasizing runner setupt
[dhuntsperger] reorganized instructions to emphasize setup steps for runners
[noreply] Merge pull request #17052 from [BEAM-13818] [SnowflakeIO] Add support
[noreply] Adding pydoc for StateHandler (#17091)
[noreply] BEAM-3165 Bypass split if numSplit is zero (#17084)
------------------------------------------
[...truncated 44.57 KB...]
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 502: Bad Gateway
ERROR:root:Invalid url for slf4j-jdk14-1.7.30:
http://www.opensource.org/licenses/mit-license.php. Retrying...
Traceback (most recent call last):
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2_java17/ws/src/sdks/java/container/license_scripts/pull_licenses_java.py";,>
line 58, in pull_from_url
url_read = urlopen(url)
File "/usr/lib/python3.8/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 563, in error
result = self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 755, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 563, in error
result = self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 755, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/lib/python3.8/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
response = self.parent.error(
File "/usr/lib/python3.8/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
result = func(*args)
File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 502: Bad Gateway
ERROR:root:Invalid url for slf4j-jdk14-1.7.30:
http://www.opensource.org/licenses/mit-license.php. Retrying...
INFO:root:Replaced local file URL with
file://<https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2_java17/ws/src/sdks/java/container/license_scripts/manual_licenses/xz/COPYING>
for xz-1.5
INFO:root:pull_licenses_java.py succeed. It took 64.016197 seconds with 16
threads.
Copying licenses from
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2_java17/ws/src/sdks/java/container/build/target/java_third_party_licenses>
to
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2_java17/ws/src/sdks/java/container/build/target/third_party_licenses.>
Finished license_scripts.sh
> Task :sdks:java:container:java17:copyJavaThirdPartyLicenses
> Task :release:go-licenses:java:dockerRun
+ go-licenses save github.com/apache/beam/sdks/java/container
--save_path=/output/licenses
+ go-licenses csv github.com/apache/beam/sdks/java/container
+ tee /output/licenses/list.csv
github.com/apache/beam/sdks/go/pkg/beam,https://github.com/apache/beam/blob/master/sdks/go/README.md,Apache-2.0
golang.org/x/net,https://go.googlesource.com/net/+/refs/heads/master/LICENSE,BSD-3-Clause
golang.org/x/sys,https://go.googlesource.com/sys/+/refs/heads/master/LICENSE,BSD-3-Clause
github.com/golang/protobuf,https://github.com/golang/protobuf/blob/master/LICENSE,BSD-3-Clause
google.golang.org/protobuf,https://go.googlesource.com/protobuf/+/refs/heads/master/LICENSE,BSD-3-Clause
github.com/apache/beam/sdks/java/container,https://github.com/apache/beam/blob/master/LICENSE,Apache-2.0
google.golang.org/grpc,https://github.com/grpc/grpc-go/blob/master/LICENSE,Apache-2.0
google.golang.org/genproto/googleapis/rpc/status,https://github.com/googleapis/go-genproto/blob/master/LICENSE,Apache-2.0
golang.org/x/text,https://go.googlesource.com/text/+/refs/heads/master/LICENSE,BSD-3-Clause
+ chmod -R a+w /output/licenses
> Task :release:go-licenses:java:createLicenses
> Task :sdks:java:container:java17:copyGolangLicenses
> Task :sdks:java:container:java17:dockerPrepare
> Task :sdks:java:container:java17:docker
> Task :runners:google-cloud-dataflow-java:buildAndPushDockerJavaContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above
18.03.
As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.
See:
https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker
The push refers to repository
[us.gcr.io/apache-beam-testing/java-postcommit-it/java]
a30df45a9638: Preparing
e172eb4c74dd: Preparing
abc8061a936b: Preparing
78515daaf8c1: Preparing
bead25860eb2: Preparing
a8c7b369ae42: Preparing
258b99bc8601: Preparing
f217d2eb7cef: Preparing
59f0533288cf: Preparing
012718046681: Preparing
4085d472d0ea: Preparing
e79e80c16e07: Preparing
95a2611f2f63: Preparing
a32c2183f0b3: Preparing
c23736825228: Preparing
e3f84a8cee1f: Preparing
48144a6f44ae: Preparing
26d5108b2cba: Preparing
89fda00479fc: Preparing
f217d2eb7cef: Waiting
59f0533288cf: Waiting
012718046681: Waiting
a8c7b369ae42: Waiting
258b99bc8601: Waiting
4085d472d0ea: Waiting
e79e80c16e07: Waiting
c23736825228: Waiting
e3f84a8cee1f: Waiting
48144a6f44ae: Waiting
a32c2183f0b3: Waiting
95a2611f2f63: Waiting
26d5108b2cba: Waiting
89fda00479fc: Waiting
abc8061a936b: Pushed
bead25860eb2: Pushed
e172eb4c74dd: Pushed
a30df45a9638: Pushed
78515daaf8c1: Pushed
f217d2eb7cef: Pushed
a8c7b369ae42: Pushed
012718046681: Pushed
59f0533288cf: Pushed
258b99bc8601: Pushed
a32c2183f0b3: Layer already exists
c23736825228: Layer already exists
e3f84a8cee1f: Layer already exists
48144a6f44ae: Layer already exists
26d5108b2cba: Layer already exists
89fda00479fc: Layer already exists
95a2611f2f63: Pushed
e79e80c16e07: Pushed
4085d472d0ea: Pushed
20220316000117: digest:
sha256:1d14e05829370d6a6ec9e6965c92864b1eed14acf78cd1085a916c3bb9cbedae size:
4314
> Task :runners:google-cloud-dataflow-java:examplesJavaRunnerV2IntegrationTest
org.apache.beam.examples.cookbook.DistinctExampleIT > testE2EDistinctExample
FAILED
java.lang.RuntimeException at DistinctExampleIT.java:81
org.apache.beam.examples.cookbook.BigQueryTornadoesIT >
testE2EBigQueryTornadoesWithExport FAILED
java.lang.RuntimeException at BigQueryTornadoesIT.java:68
org.apache.beam.examples.cookbook.CombinePerKeyExamplesIT >
testE2ECombinePerKey FAILED
java.lang.RuntimeException at CombinePerKeyExamplesIT.java:73
org.apache.beam.examples.cookbook.FilterExamplesIT > testE2EFilterExamples
FAILED
java.lang.RuntimeException at FilterExamplesIT.java:73
***********************************************************
***********************************************************
**********************Set Up Pubsub************************
The Pub/Sub topic has been set up for this example:
projects/apache-beam-testing/topics/testpipeline-jenkins-0316000506-a82bff0b
The Pub/Sub subscription has been set up for this example:
projects/apache-beam-testing/subscriptions/testpipeline-jenkins-0316000506-a82bff0b
*************************Tear Down*************************
The Pub/Sub topic has been deleted:
projects/apache-beam-testing/topics/testpipeline-jenkins-0316000506-a82bff0b
The Pub/Sub subscription has been deleted:
projects/apache-beam-testing/subscriptions/testpipeline-jenkins-0316000506-a82bff0b
***********************************************************
***********************************************************
org.apache.beam.examples.snippets.transforms.io.gcp.bigquery.BigQuerySamplesIT
> testTableIO FAILED
java.lang.RuntimeException at BigQuerySamplesIT.java:108
***********************************************************
***********************************************************
**********************Set Up Pubsub************************
The Pub/Sub topic has been set up for this example:
projects/apache-beam-testing/topics/testpipeline-jenkins-0316000506-92ad7da2
The Pub/Sub subscription has been set up for this example:
projects/apache-beam-testing/subscriptions/testpipeline-jenkins-0316000506-92ad7da2
*************************Tear Down*************************
The Pub/Sub topic has been deleted:
projects/apache-beam-testing/topics/testpipeline-jenkins-0316000506-92ad7da2
The Pub/Sub subscription has been deleted:
projects/apache-beam-testing/subscriptions/testpipeline-jenkins-0316000506-92ad7da2
***********************************************************
***********************************************************
org.apache.beam.examples.cookbook.BigQueryTornadoesIT >
testE2eBigQueryTornadoesWithStorageApi FAILED
java.lang.RuntimeException at BigQueryTornadoesIT.java:68
org.apache.beam.examples.cookbook.MaxPerKeyExamplesIT >
testE2EMaxPerKeyExamples FAILED
java.lang.RuntimeException at MaxPerKeyExamplesIT.java:75
org.apache.beam.examples.cookbook.BigQueryTornadoesIT >
testE2eBigQueryTornadoesWithStorageApiUsingQuery FAILED
java.lang.RuntimeException at BigQueryTornadoesIT.java:68
org.apache.beam.examples.cookbook.JoinExamplesIT > testE2EJoinExamples FAILED
java.lang.RuntimeException at JoinExamplesIT.java:63
13 tests completed, 9 failed
> Task :runners:google-cloud-dataflow-java:examplesJavaRunnerV2IntegrationTest
> FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220316000117
Untagged:
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:1d14e05829370d6a6ec9e6965c92864b1eed14acf78cd1085a916c3bb9cbedae
ERROR: (gcloud.container.images.untag) Image could not be found:
[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220316000117]
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':runners:google-cloud-dataflow-java:examplesJavaRunnerV2IntegrationTest'.
> There were failing tests. See the report at:
> file://<https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2_java17/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/examplesJavaRunnerV2IntegrationTest/index.html>
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Build file
'<https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Dataflow_V2_java17/ws/src/runners/google-cloud-dataflow-java/build.gradle'>
line: 294
* What went wrong:
Execution failed for task
':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'.
> Process 'command 'gcloud'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 45m 7s
88 actionable tasks: 53 executed, 33 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/3z5itvonovidc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
