See
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_PythonUsingJava_Dataflow/78/display/redirect?page=changes>
Changes:
[noreply] Update 2.36.0 blog post to mention ARM64 support
[noreply] Minor: Disable checker framework in nightly snapshot (#16829)
[noreply] [BEAM-13860] Make `DoFn.infer_output_type` return element type
(#16788)
------------------------------------------
[...truncated 60.88 KB...]
> Task :sdks:python:installGcpTest
Collecting google-crc32c<2.0dev,>=1.0
Using cached
google_crc32c-1.3.0-cp36-cp36m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
(37 kB)
Requirement already satisfied: zipp>=0.5 in
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_PythonUsingJava_Dataflow/src/build/gradleenv/1922375555/lib/python3.6/site-packages
(from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.38.0.dev0)
(3.6.0)
Collecting typing-inspect>=0.4.0
Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)
Collecting requests-oauthlib>=0.5.0
Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting isodate>=0.6.0
Using cached isodate-0.6.1-py2.py3-none-any.whl (41 kB)
Collecting typing-utils>=0.0.3
Using cached typing_utils-0.1.0-py3-none-any.whl (10 kB)
Collecting websocket-client>=0.32.0
Using cached websocket_client-1.2.3-py3-none-any.whl (53 kB)
Collecting oauthlib>=3.0.0
Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Collecting mypy-extensions>=0.3.0
Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam:
filename=apache_beam-2.38.0.dev0-py3-none-any.whl size=2659514
sha256=449e1e780eacb612ef23ba5d8c88075c19375d3cc82aa8e926a5d9638036b517
Stored in directory:
/home/jenkins/.cache/pip/wheels/ae/83/60/8ffa598c4b9bac2ca773fed6f402d79ce042675f87beed8970
Successfully built apache-beam
Installing collected packages: pyasn1, urllib3, rsa, pyparsing, pyasn1-modules,
idna, charset-normalizer, certifi, cachetools, typing-extensions, requests,
pytz, mypy-extensions, googleapis-common-protos, google-auth, websocket-client,
wcwidth, typing-inspect, pyyaml, python-dateutil, pycparser, pluggy, oauthlib,
more-itertools, jmespath, google-api-core, dataclasses, attrs, atomicwrites,
wrapt, typing-utils, requests-oauthlib, pytest, proto-plus, numpy, libcst,
isodate, httplib2, grpcio-gcp, grpc-google-iam-v1, greenlet, google-crc32c,
docopt, docker, deprecation, cffi, botocore, testcontainers, sqlalchemy,
s3transfer, pytest-forked, pymysql, pymongo, pydot, pyarrow, pbr, overrides,
orjson, oauth2client, msrest, hdfs, grpcio-status, google-resumable-media,
google-cloud-pubsub, google-cloud-core, fasteners, fastavro, execnet, dill,
cryptography, crcmod, cloudpickle, azure-core, tenacity, requests-mock,
pytest-xdist, pytest-timeout, pyhamcrest, psycopg2-binary, parameterized,
pandas, mock, google-cloud-vision, google-cloud-videointelligence,
google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsublite,
google-cloud-language, google-cloud-dlp, google-cloud-datastore,
google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery,
google-apitools, freezegun, boto3, azure-storage-blob, apache-beam
Attempting uninstall: pyparsing
Found existing installation: pyparsing 3.0.7
Uninstalling pyparsing-3.0.7:
Successfully uninstalled pyparsing-3.0.7
Attempting uninstall: pluggy
Found existing installation: pluggy 1.0.0
Uninstalling pluggy-1.0.0:
Successfully uninstalled pluggy-1.0.0
Successfully installed apache-beam-2.38.0.dev0 atomicwrites-1.4.0 attrs-21.4.0
azure-core-1.22.1 azure-storage-blob-12.9.0 boto3-1.20.54 botocore-1.23.54
cachetools-4.2.4 certifi-2021.10.8 cffi-1.15.0 charset-normalizer-2.0.12
cloudpickle-2.0.0 crcmod-1.7 cryptography-36.0.1 dataclasses-0.8
deprecation-2.1.0 dill-0.3.1.1 docker-5.0.3 docopt-0.6.2 execnet-1.9.0
fastavro-1.4.7 fasteners-0.17.3 freezegun-1.1.0 google-api-core-1.31.5
google-apitools-0.5.31 google-auth-1.35.0 google-cloud-bigquery-2.32.0
google-cloud-bigquery-storage-2.11.0 google-cloud-bigtable-1.7.0
google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-3.6.0
google-cloud-language-1.3.0 google-cloud-pubsub-2.9.0
google-cloud-pubsublite-1.3.0 google-cloud-recommendations-ai-0.2.0
google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1
google-cloud-vision-1.0.0 google-crc32c-1.3.0 google-resumable-media-2.2.1
googleapis-common-protos-1.54.0 greenlet-1.1.2 grpc-google-iam-v1-0.12.3
grpcio-gcp-0.2.2 grpcio-status-1.43.0 hdfs-2.6.0 httplib2-0.19.1 idna-3.3
isodate-0.6.1 jmespath-0.10.0 libcst-0.4.1 mock-2.0.0 more-itertools-8.12.0
msrest-0.6.21 mypy-extensions-0.4.3 numpy-1.19.5 oauth2client-4.1.3
oauthlib-3.2.0 orjson-3.6.1 overrides-6.1.0 pandas-1.1.5 parameterized-0.7.5
pbr-5.8.1 pluggy-0.13.1 proto-plus-1.20.0 psycopg2-binary-2.9.3 pyarrow-6.0.1
pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.21 pydot-1.4.2 pyhamcrest-1.10.1
pymongo-3.12.3 pymysql-1.0.2 pyparsing-2.4.7 pytest-4.6.11 pytest-forked-1.4.0
pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.3
pyyaml-6.0 requests-2.27.1 requests-mock-1.9.3 requests-oauthlib-1.3.1 rsa-4.8
s3transfer-0.5.1 sqlalchemy-1.4.31 tenacity-5.1.5 testcontainers-3.4.2
typing-extensions-4.1.1 typing-inspect-0.7.1 typing-utils-0.1.0 urllib3-1.26.8
wcwidth-0.2.5 websocket-client-1.2.3 wrapt-1.13.3
> Task :release:go-licenses:py:dockerRun
+ go-licenses save github.com/apache/beam/sdks/python/container
--save_path=/output/licenses
> Task :release:go-licenses:java:dockerRun
+ go-licenses save github.com/apache/beam/sdks/java/container
--save_path=/output/licenses
+ tee /output/licenses/list.csv
+ go-licenses csv github.com/apache/beam/sdks/java/container
github.com/golang/protobuf,https://github.com/golang/protobuf/blob/master/LICENSE,BSD-3-Clause
google.golang.org/protobuf,https://go.googlesource.com/protobuf/+/refs/heads/master/LICENSE,BSD-3-Clause
golang.org/x/text,https://go.googlesource.com/text/+/refs/heads/master/LICENSE,BSD-3-Clause
golang.org/x/net,https://go.googlesource.com/net/+/refs/heads/master/LICENSE,BSD-3-Clause
google.golang.org/genproto/googleapis/rpc/status,https://github.com/googleapis/go-genproto/blob/master/LICENSE,Apache-2.0
golang.org/x/sys,https://go.googlesource.com/sys/+/refs/heads/master/LICENSE,BSD-3-Clause
github.com/apache/beam/sdks/java/container,https://github.com/apache/beam/blob/master/LICENSE,Apache-2.0
github.com/apache/beam/sdks/go/pkg/beam,https://github.com/apache/beam/blob/master/sdks/go/README.md,Apache-2.0
google.golang.org/grpc,https://github.com/grpc/grpc-go/blob/master/LICENSE,Apache-2.0
+ chmod -R a+w /output/licenses
> Task :release:go-licenses:py:dockerRun
+ go-licenses csv github.com/apache/beam/sdks/python/container
+ tee /output/licenses/list.csv
github.com/golang/protobuf,https://github.com/golang/protobuf/blob/master/LICENSE,BSD-3-Clause
google.golang.org/genproto/googleapis/rpc/status,https://github.com/googleapis/go-genproto/blob/master/LICENSE,Apache-2.0
golang.org/x/sys,https://go.googlesource.com/sys/+/refs/heads/master/LICENSE,BSD-3-Clause
github.com/nightlyone/lockfile,https://github.com/nightlyone/lockfile/blob/master/LICENSE,MIT
github.com/apache/beam/sdks/python/container,https://github.com/apache/beam/blob/master/LICENSE,Apache-2.0
google.golang.org/grpc,https://github.com/grpc/grpc-go/blob/master/LICENSE,Apache-2.0
golang.org/x/net,https://go.googlesource.com/net/+/refs/heads/master/LICENSE,BSD-3-Clause
github.com/apache/beam/sdks/go/pkg/beam,https://github.com/apache/beam/blob/master/sdks/go/README.md,Apache-2.0
google.golang.org/protobuf,https://go.googlesource.com/protobuf/+/refs/heads/master/LICENSE,BSD-3-Clause
golang.org/x/text,https://go.googlesource.com/text/+/refs/heads/master/LICENSE,BSD-3-Clause
+ chmod -R a+w /output/licenses
> Task :release:go-licenses:py:createLicenses
> Task :sdks:python:container:py36:copyGolangLicenses
> Task :sdks:python:container:py36:dockerPrepare
> Task :release:go-licenses:java:createLicenses
> Task :sdks:java:container:java8:copyGolangLicenses
> Task :sdks:java:container:java8:dockerPrepare
> Task :sdks:python:container:py36:docker
> Task :sdks:java:container:java8:docker
> Task :runners:google-cloud-dataflow-java:buildAndPushDockerJavaContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above
18.03.
As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.
See:
https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker
> Task :runners:google-cloud-dataflow-java:buildAndPushDockerPythonContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above
18.03.
As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.
See:
https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker
The push refers to repository
[us.gcr.io/apache-beam-testing/java-postcommit-it/python]
8ce72a4cb247: Preparing
e6fbbef9fe33: Preparing
37802aab1829: Preparing
1491cbe28038: Preparing
89c4716f0c44: Preparing
653a2f52258d: Preparing
75447a12f685: Preparing
217cff8341ec: Preparing
c8d4300ecb77: Preparing
50513fff389d: Preparing
2077c2dc9ca5: Preparing
a6720796c2a9: Preparing
2e050f4863ad: Preparing
10c9ac0985bc: Preparing
f6c77da3659f: Preparing
aa4c808c19f6: Preparing
8ba9f690e8ba: Preparing
3e607d59ef9f: Preparing
1e18e7e1fcc2: Preparing
c3a0d593ed24: Preparing
26a504e63be4: Preparing
8bf42db0de72: Preparing
31892cc314cb: Preparing
11936051f93b: Preparing
653a2f52258d: Waiting
75447a12f685: Waiting
217cff8341ec: Waiting
8ba9f690e8ba: Waiting
2e050f4863ad: Waiting
3e607d59ef9f: Waiting
c8d4300ecb77: Waiting
1e18e7e1fcc2: Waiting
50513fff389d: Waiting
aa4c808c19f6: Waiting
c3a0d593ed24: Waiting
f6c77da3659f: Waiting
2077c2dc9ca5: Waiting
10c9ac0985bc: Waiting
26a504e63be4: Waiting
a6720796c2a9: Waiting
31892cc314cb: Waiting
8bf42db0de72: Waiting
37802aab1829: Pushed
1491cbe28038: Pushed
8ce72a4cb247: Pushed
e6fbbef9fe33: Pushed
89c4716f0c44: Pushed
75447a12f685: Pushed
c8d4300ecb77: Pushed
653a2f52258d: Pushed
50513fff389d: Pushed
2077c2dc9ca5: Pushed
10c9ac0985bc: Pushed
aa4c808c19f6: Layer already exists
8ba9f690e8ba: Layer already exists
3e607d59ef9f: Layer already exists
1e18e7e1fcc2: Layer already exists
c3a0d593ed24: Layer already exists
26a504e63be4: Layer already exists
8bf42db0de72: Layer already exists
31892cc314cb: Layer already exists
11936051f93b: Layer already exists
f6c77da3659f: Pushed
217cff8341ec: Pushed
a6720796c2a9: Pushed
2e050f4863ad: Pushed
20220214180311: digest:
sha256:16cb227b90ea30eea8026046da9d8cb3ab23ff6db428fa660ded9f61eff775e4 size:
5361
> Task :runners:google-cloud-dataflow-java:setupXVR
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerSetup
Launching Java expansion service @ 37381
Launching Python expansion service @ 36053
> Task
> :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerPythonUsingJava
>>> RUNNING integration tests with pipeline options:
>>> --runner=TestDataflowRunner --project=apache-beam-testing
>>> --region=us-central1
>>> --sdk_harness_container_image_overrides=.*java.*,us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220214180311
>>> --experiments=use_runner_v2 --experiments=shuffle_mode=appliance
>>> --sdk_location=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_PythonUsingJava_Dataflow/src/sdks/python/build/apache-beam.tar.gz
>>> pytest options: --capture=no --numprocesses=8 --timeout=4500
>>> --log-cli-level=INFO
>>> collect markers: -m=xlang_transforms
============================= test session starts ==============================
platform linux -- Python 3.6.15, pytest-4.6.11, py-1.11.0, pluggy-0.13.1
rootdir:
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_XVR_PythonUsingJava_Dataflow/src/sdks/python,
inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.4.0, requests-mock-1.9.3
timeout: 4500.0s
timeout method: signal
timeout func_only: False
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I
[gw0] Python 3.6.15 (default, Jan 15 2022, 19:47:13) -- [GCC 9.3.0]
[gw1] Python 3.6.15 (default, Jan 15 2022, 19:47:13) -- [GCC 9.3.0]
[gw2] Python 3.6.15 (default, Jan 15 2022, 19:47:13) -- [GCC 9.3.0]
[gw4] Python 3.6.15 (default, Jan 15 2022, 19:47:13) -- [GCC 9.3.0]
[gw3] Python 3.6.15 (default, Jan 15 2022, 19:47:13) -- [GCC 9.3.0]
[gw6] Python 3.6.15 (default, Jan 15 2022, 19:47:13) -- [GCC 9.3.0]
[gw5] Python 3.6.15 (default, Jan 15 2022, 19:47:13) -- [GCC 9.3.0]
[gw7] Python 3.6.15 (default, Jan 15 2022, 19:47:13) -- [GCC 9.3.0]
gw0 [10] / gw1 [10] / gw2 [10] / gw3 [10] / gw4 [10] / gw5 [10] / gw6 [10] /
gw7 [10]
scheduling tests via LoadScheduling
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_combine_per_key
apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence_java_class_lookup_payload_builder
apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence_java_external_transform
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_combine_globally
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_group_by_key
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_cogroup_by_key
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_multi_input_output_with_sideinput
apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence
[gw5] PASSED
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_group_by_key
[gw4] PASSED
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_cogroup_by_key
[gw1] PASSED
apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence_java_class_lookup_payload_builder
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_prefix
[gw2] PASSED
apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence_java_external_transform
[gw7] PASSED
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_multi_input_output_with_sideinput
[gw0] PASSED
apache_beam/io/external/generate_sequence_test.py::XlangGenerateSequenceTest::test_generate_sequence
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_partition
[gw3] PASSED
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_combine_globally
[gw6] PASSED
apache_beam/transforms/validate_runner_xlang_test.py::ValidateRunnerXlangTest::test_combine_per_key
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-8' is disconnected.
at
hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:216)
at
hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy133.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1211)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1203)
at hudson.Launcher$ProcStarter.join(Launcher.java:523)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:803)
at hudson.model.Build$BuildExecution.build(Build.java:197)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:513)
at hudson.model.Run.execute(Run.java:1906)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:429)
Caused by: java.io.IOException: Unexpected termination of the channel
at
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:75)
Caused by: java.io.EOFException
at
java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2799)
at
java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3274)
at
java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:934)
at java.io.ObjectInputStream.<init>(ObjectInputStream.java:396)
at
hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49)
at hudson.remoting.Command.readFrom(Command.java:142)
at hudson.remoting.Command.readFrom(Command.java:128)
at
hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
at
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-8 is offline; cannot locate jdk_1.8_latest
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
