See
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_JavaUsingPython_Dataflow/1539/display/redirect?page=changes>
Changes:
[noreply] Fix interface{} in iter& emit type of DoFn in Go (#25203)
[noreply] Task #25064: Python SDK Data sampling implementation (#25093)
[noreply] Add support for all Java based portable runners to consume elements
[noreply] Swap Java SDK container to use eclipse-temurin as the base instead of
[noreply] Update BigQueryIO documentation with details on how to override the
[noreply] Support Avro GenericRecord as a valid format for StorageWrite API on
------------------------------------------
[...truncated 1.07 MB...]
termcolor==2.2.0
testcontainers==3.7.1
threadpoolctl==3.1.0
tomli==2.0.1
tqdm==4.64.1
typing_extensions==4.4.0
uritemplate==4.1.1
urllib3==1.26.14
websocket-client==1.5.1
Werkzeug==2.2.3
wheel==0.38.4
wrapt==1.14.1
zstandard==0.19.0
Removing intermediate container 7c077d65a0dc
---> fc9e96bc9f4e
Step 17/32 : COPY target/LICENSE /opt/apache/beam/
---> 076d7649f4b4
Step 18/32 : COPY target/LICENSE.python /opt/apache/beam/
---> 72c8b7210c3a
Step 19/32 : COPY target/NOTICE /opt/apache/beam/
---> 5de804e32d91
Step 20/32 : COPY target/launcher/linux_amd64/boot /opt/apache/beam/
---> 628b6bc2ae01
Step 21/32 : ENTRYPOINT ["/opt/apache/beam/boot"]
---> Running in a8708367dc35
Removing intermediate container a8708367dc35
---> f38ee1b2d00e
Step 22/32 : FROM beam as third_party_licenses
---> f38ee1b2d00e
Step 23/32 : ARG pull_licenses
---> Running in 4b6e83bf1682
Removing intermediate container 4b6e83bf1682
---> 1c19fad5d6d8
Step 24/32 : COPY target/license_scripts /tmp/license_scripts/
---> cc2f26e2647c
Step 25/32 : COPY target/LICENSE target/go-licenses/*
/opt/apache/beam/third_party_licenses/golang/
---> 7b67e20c626f
Step 26/32 : RUN rm /opt/apache/beam/third_party_licenses/golang/LICENSE
---> Running in 8e3b5a0c6e8d
Removing intermediate container 8e3b5a0c6e8d
---> e1647587e241
Step 27/32 : COPY target/license_scripts /tmp/license_scripts/
---> 063ca9085e70
Step 28/32 : RUN if [ "$pull_licenses" = "true" ] ; then pip install
'pip-licenses<4.0.0' pyyaml tenacity && python
/tmp/license_scripts/pull_licenses_py.py ; fi
---> Running in 91f1c34fe6b9
Collecting pip-licenses<4.0.0
Downloading pip_licenses-3.5.5-py3-none-any.whl (17 kB)
Requirement already satisfied: pyyaml in
/usr/local/lib/python3.10/site-packages (6.0)
Requirement already satisfied: tenacity in
/usr/local/lib/python3.10/site-packages (5.1.5)
Collecting PTable
Downloading PTable-0.9.2.tar.gz (31 kB)
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: six>=1.9.0 in
/usr/local/lib/python3.10/site-packages (from tenacity) (1.16.0)
Building wheels for collected packages: PTable
Building wheel for PTable (setup.py): started
Building wheel for PTable (setup.py): finished with status 'done'
Created wheel for PTable: filename=PTable-0.9.2-py3-none-any.whl size=22908
sha256=f783102a1332d68d411af1bd4c74b942772dee461726ddb593953bd146b2bac1
Stored in directory:
/root/.cache/pip/wheels/bc/88/52/f2e9fc70f3a657cf256e9b01a8a42938c4c5ee69118d51ed90
Successfully built PTable
Installing collected packages: PTable, pip-licenses
Successfully installed PTable-0.9.2 pip-licenses-3.5.5
[91mWARNING: Running pip as the 'root' user can result in broken permissions
and conflicting behaviour with the system package manager. It is recommended to
use a virtual environment instead: https://pip.pypa.io/warnings/venv
[0m[91mINFO:root:Successfully pulled licenses for 134 dependencies
[0mSkip pulling license for bs4
Removing intermediate container 91f1c34fe6b9
---> 3a4322761768
Step 29/32 : FROM beam
---> f38ee1b2d00e
Step 30/32 : ARG pull_licenses
---> Running in b40c45e37f05
Removing intermediate container b40c45e37f05
---> c65384b62ed5
Step 31/32 : COPY --from=third_party_licenses
/opt/apache/beam/third_party_licenses /opt/apache/beam/third_party_licenses
---> 6f8a6b31819a
Step 32/32 : RUN if [ "$pull_licenses" != "true" ] ; then rm -rf
/opt/apache/beam/third_party_licenses ; fi
---> Running in 57b3586232ac
Removing intermediate container 57b3586232ac
---> 5cff0384de5e
Successfully built 5cff0384de5e
Successfully tagged apache/beam_python3.10_sdk:2.46.0.dev
> Task :runners:google-cloud-dataflow-java:buildAndPushDockerPythonContainer
WARNING: `gcloud docker` will not be supported for Docker client versions above
18.03.
As an alternative, use `gcloud auth configure-docker` to configure `docker` to
use `gcloud` as a credential helper, then use `docker` as you would for non-GCR
registries, e.g. `docker pull gcr.io/project-id/my-image`. Add
`--verbosity=error` to silence this warning: `gcloud docker
--verbosity=error -- pull gcr.io/project-id/my-image`.
See:
https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker
The push refers to repository
[us.gcr.io/apache-beam-testing/java-postcommit-it/python]
f4ab44b141a3: Preparing
f304219c7e75: Preparing
e609293e75f9: Preparing
50a14fa238ab: Preparing
34aba3b081a7: Preparing
939f060899f0: Preparing
145c065488fd: Preparing
18b4b5887b19: Preparing
fb6c82629dd7: Preparing
1615ac2bd39c: Preparing
f3092caa375e: Preparing
a6c914324f8c: Preparing
5a2b2065e71d: Preparing
84093bc29af2: Preparing
8243d6a3557a: Preparing
94058978c148: Preparing
1b45dbbd13b7: Preparing
17efc79ac0fe: Preparing
0b6859e9fff1: Preparing
11829b3be9c0: Preparing
dc8e1d8b53e9: Preparing
9d49e0bc68a4: Preparing
145c065488fd: Waiting
8e396a1aad50: Preparing
f3092caa375e: Waiting
18b4b5887b19: Waiting
11829b3be9c0: Waiting
94058978c148: Waiting
a6c914324f8c: Waiting
dc8e1d8b53e9: Waiting
9d49e0bc68a4: Waiting
1b45dbbd13b7: Waiting
5a2b2065e71d: Waiting
fb6c82629dd7: Waiting
8243d6a3557a: Waiting
17efc79ac0fe: Waiting
8e396a1aad50: Waiting
1615ac2bd39c: Waiting
0b6859e9fff1: Waiting
939f060899f0: Waiting
e609293e75f9: Pushed
f4ab44b141a3: Pushed
50a14fa238ab: Pushed
34aba3b081a7: Pushed
f304219c7e75: Pushed
18b4b5887b19: Pushed
145c065488fd: Pushed
fb6c82629dd7: Pushed
5a2b2065e71d: Pushed
f3092caa375e: Pushed
8243d6a3557a: Layer already exists
84093bc29af2: Pushed
94058978c148: Layer already exists
1b45dbbd13b7: Layer already exists
17efc79ac0fe: Layer already exists
0b6859e9fff1: Layer already exists
11829b3be9c0: Layer already exists
dc8e1d8b53e9: Layer already exists
9d49e0bc68a4: Layer already exists
8e396a1aad50: Layer already exists
939f060899f0: Pushed
1615ac2bd39c: Pushed
a6c914324f8c: Pushed
20230217034814: digest:
sha256:3c6626b3bfee5a01199afc08fc8147195d273c78ef57c8ca3719788369a39635 size:
5156
> Task :runners:google-cloud-dataflow-java:setupXVR
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerSetup
Launching Java expansion service @ 36455
Launching Python expansion service @ 44215
> Task
> :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerJavaUsingPython
INFO:root:Default Python SDK image for environment is
apache/beam_python3.8_sdk:2.45.0
INFO:root:No image given, using default Python SDK image
INFO:root:Default Python SDK image for environment is
apache/beam_python3.8_sdk:2.45.0
INFO:root:Python SDK container image set to "apache/beam_python3.8_sdk:2.45.0"
for Docker environment
INFO:__main__:Listening for expansion requests at 32979
INFO:root:Missing pipeline option (runner). Executing pipeline using the
default runner: DirectRunner.
INFO:__main__:Shutting down expansion service.
INFO:apache_beam.runners.portability.stager:Executing command:
['/home/jenkins/.apache_beam/cache/venvs/py-3.8-beam-2.45.0-743bdccef67087512a7a190820e727aab7bdcb14/bin/python',
'-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r',
'/tmp/tmpo8ja12gk/tmp_requirements.txt', '--exists-action', 'i', '--no-deps',
'--implementation', 'cp', '--abi', 'cp38', '--platform', 'manylinux2014_x86_64']
INFO:root:Default Python SDK image for environment is
apache/beam_python3.8_sdk:2.45.0
INFO:root:No image given, using default Python SDK image
INFO:root:Default Python SDK image for environment is
apache/beam_python3.8_sdk:2.45.0
INFO:root:Python SDK container image set to "apache/beam_python3.8_sdk:2.45.0"
for Docker environment
INFO:__main__:Listening for expansion requests at 39541
INFO:root:Missing pipeline option (runner). Executing pipeline using the
default runner: DirectRunner.
INFO:__main__:Shutting down expansion service.
org.apache.beam.sdk.extensions.python.transforms.DataframeTransformTest >
testDataframeSum FAILED
java.lang.RuntimeException at TestDataflowRunner.java:141
16 tests completed, 1 failed
> Task
> :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerJavaUsingPython
> FAILED
> Task :runners:google-cloud-dataflow-java:cleanupXVR UP-TO-DATE
> Task :runners:google-cloud-dataflow-java:cleanUpDockerPythonImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/python:20230217034814
Untagged:
us.gcr.io/apache-beam-testing/java-postcommit-it/python@sha256:3c6626b3bfee5a01199afc08fc8147195d273c78ef57c8ca3719788369a39635
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/python:20230217034814]
- referencing digest:
[us.gcr.io/apache-beam-testing/java-postcommit-it/python@sha256:3c6626b3bfee5a01199afc08fc8147195d273c78ef57c8ca3719788369a39635]
Deleted
[[us.gcr.io/apache-beam-testing/java-postcommit-it/python:20230217034814]
(referencing
[us.gcr.io/apache-beam-testing/java-postcommit-it/python@sha256:3c6626b3bfee5a01199afc08fc8147195d273c78ef57c8ca3719788369a39635])].
Removing untagged image
us.gcr.io/apache-beam-testing/java-postcommit-it/python@sha256:3c6626b3bfee5a01199afc08fc8147195d273c78ef57c8ca3719788369a39635
Digests:
-
us.gcr.io/apache-beam-testing/java-postcommit-it/python@sha256:3c6626b3bfee5a01199afc08fc8147195d273c78ef57c8ca3719788369a39635
Deleted
[us.gcr.io/apache-beam-testing/java-postcommit-it/python@sha256:3c6626b3bfee5a01199afc08fc8147195d273c78ef57c8ca3719788369a39635].
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20230217034814
Untagged:
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:99d75ef400740b2fd3ea866c612e6dcb703b474e5ccc0150db86654469ec8609
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20230217034814]
- referencing digest:
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:99d75ef400740b2fd3ea866c612e6dcb703b474e5ccc0150db86654469ec8609]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20230217034814]
(referencing
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:99d75ef400740b2fd3ea866c612e6dcb703b474e5ccc0150db86654469ec8609])].
Removing untagged image
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:99d75ef400740b2fd3ea866c612e6dcb703b474e5ccc0150db86654469ec8609
Digests:
-
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:99d75ef400740b2fd3ea866c612e6dcb703b474e5ccc0150db86654469ec8609
Deleted
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:99d75ef400740b2fd3ea866c612e6dcb703b474e5ccc0150db86654469ec8609].
> Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerCleanup
Stopping expansion service pid: 899263.
Stopping expansion service pid: 899264.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at:
> file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_JavaUsingPython_Dataflow/ws/src/runners/google-cloud-dataflow-java/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 36m 37s
175 actionable tasks: 25 executed, 150 up-to-date
Publishing build scan...
https://gradle.com/s/fzlkumh3ts4z6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
