See
<https://ci-beam.apache.org/job/beam_PostCommit_Python_Xlang_IO_Dataflow/619/display/redirect>
Changes:
------------------------------------------
[...truncated 298.06 KB...]
d44c5fd40519b8bbf58d931c13869fa282edbe413645cd6a97d118ecb7b94a0f
+ /bin/true
+ mkdir -p /project
✓ 9.21s
Copying project into container...
+ mkdir -p /project
✓ 0.25s
Building cp311-manylinux_x86_64 wheel
CPython 3.11 manylinux x86_64
Setting up build environment...
+ /opt/python/cp38-cp38/bin/python -c 'import sys, json, os;
json.dump(os.environ.copy(), sys.stdout)'
+ which python
+ which pip
✓ 0.34s
Building wheel...
+ rm -rf /tmp/cibuildwheel/built_wheel
+ mkdir -p /tmp/cibuildwheel/built_wheel
+ python -m pip wheel /project --wheel-dir=/tmp/cibuildwheel/built_wheel
--no-deps
[notice] A new release of pip available: 22.2.2 -> 23.3.2
[notice] To update, run: pip install --upgrade pip
Processing /project
Installing build dependencies: started
Installing build dependencies: finished with status 'done'
Getting requirements to build wheel: started
Getting requirements to build wheel: finished with status 'done'
Preparing metadata (pyproject.toml): started
Preparing metadata (pyproject.toml): finished with status 'done'
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (pyproject.toml): started
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): still running...
Building wheel for apache-beam (pyproject.toml): finished with status 'done'
Created wheel for apache-beam:
filename=apache_beam-2.55.0.dev0-cp311-cp311-linux_x86_64.whl size=15853286
sha256=7b08a9db13ed4011ec34fb764f912a0c2c2920cb4299634b5a4a047aff993720
Stored in directory:
/tmp/pip-ephem-wheel-cache-gxihyh6f/wheels/aa/0c/49/7c4d39428e78274ff9f1db5b9c539aa0d6bd6becda70207fa7
Successfully built apache-beam
+ /opt/python/cp38-cp38/bin/python -c 'import sys, json, glob;
json.dump(glob.glob('"'"'/tmp/cibuildwheel/built_wheel/*.whl'"'"'), sys.stdout)'
+ rm -rf /tmp/cibuildwheel/repaired_wheel
+ mkdir -p /tmp/cibuildwheel/repaired_wheel
✓ 232.93s
Repairing wheel...
+ sh -c 'auditwheel repair -w /tmp/cibuildwheel/repaired_wheel
/tmp/cibuildwheel/built_wheel/apache_beam-2.55.0.dev0-cp311-cp311-linux_x86_64.whl'
INFO:auditwheel.main_repair:Repairing
apache_beam-2.55.0.dev0-cp311-cp311-linux_x86_64.whl
INFO:auditwheel.wheeltools:Previous filename tags: linux_x86_64
INFO:auditwheel.wheeltools:New filename tags: manylinux_2_17_x86_64,
manylinux2014_x86_64
INFO:auditwheel.wheeltools:Previous WHEEL info tags: cp311-cp311-linux_x86_64
INFO:auditwheel.wheeltools:New WHEEL info tags:
cp311-cp311-manylinux_2_17_x86_64, cp311-cp311-manylinux2014_x86_64
INFO:auditwheel.main_repair:
Fixed-up wheel written to
/tmp/cibuildwheel/repaired_wheel/apache_beam-2.55.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
+ /opt/python/cp38-cp38/bin/python -c 'import sys, json, glob;
json.dump(glob.glob('"'"'/tmp/cibuildwheel/repaired_wheel/*.whl'"'"'),
sys.stdout)'
+ mkdir -p /output
+ mv
/tmp/cibuildwheel/repaired_wheel/apache_beam-2.55.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
/output
✓ 6.90s
✓ cp311-manylinux_x86_64 finished in 240.27s
Copying wheels back to host...
✓ 0.13s
1 wheels produced in 4 minutes:
apache_beam-2.55.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
15,499 kB
> Task :sdks:python:test-suites:dataflow:py311:initializeForDataflowJob
> Task :sdks:python:test-suites:dataflow:py311:ioCrossLanguageSetup
Launching Java expansion service @ 44061
Launching Python expansion service @
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Xlang_IO_Dataflow/src/sdks/python/scripts/run_expansion_services.sh:
line 128: /bin/activate: No such file or directory
> Task :sdks:python:bdistPy38linux
Requirement already satisfied: cibuildwheel==2.9.0 in
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Xlang_IO_Dataflow/src/build/gradleenv/1922375555/lib/python3.8/site-packages
(2.9.0)
Requirement already satisfied: bashlex!=0.13 in
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Xlang_IO_Dataflow/src/build/gradleenv/1922375555/lib/python3.8/site-packages
(from cibuildwheel==2.9.0) (0.18)
Requirement already satisfied: bracex in
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Xlang_IO_Dataflow/src/build/gradleenv/1922375555/lib/python3.8/site-packages
(from cibuildwheel==2.9.0) (2.4)
Requirement already satisfied: certifi in
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Xlang_IO_Dataflow/src/build/gradleenv/1922375555/lib/python3.8/site-packages
(from cibuildwheel==2.9.0) (2023.11.17)
Requirement already satisfied: filelock in
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Xlang_IO_Dataflow/src/build/gradleenv/1922375555/lib/python3.8/site-packages
(from cibuildwheel==2.9.0) (3.13.1)
Requirement already satisfied: packaging>=20.9 in
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Xlang_IO_Dataflow/src/build/gradleenv/1922375555/lib/python3.8/site-packages
(from cibuildwheel==2.9.0) (23.2)
Requirement already satisfied: platformdirs in
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Xlang_IO_Dataflow/src/build/gradleenv/1922375555/lib/python3.8/site-packages
(from cibuildwheel==2.9.0) (4.1.0)
Requirement already satisfied: tomli in
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Xlang_IO_Dataflow/src/build/gradleenv/1922375555/lib/python3.8/site-packages
(from cibuildwheel==2.9.0) (2.0.1)
cp38-manylinux_x86_64
_ _ _ _ _ _ _
___|_| |_ _ _|_| |_| |_ _ _| |_ ___ ___| |
| _| | . | | | | | . | | | | | -_| -_| |
|___|_|___|___|_|_|___|_____|_|_|___|___|_|
cibuildwheel version 2.9.0
Build options:
platform: 'linux'
architectures: {<Architecture.i686: 'i686'>, <Architecture.x86_64: 'x86_64'>}
build_selector: {'build_config': 'cp38-manylinux_*64*', 'skip_config': '',
'requires_python': None, 'prerelease_pythons': False}
container_engine: 'docker'
output_dir:
PosixPath('/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Xlang_IO_Dataflow/src/sdks/python/build')
package_dir:
PosixPath('/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Xlang_IO_Dataflow/src/sdks/python')
test_selector: {'skip_config': ''}
before_all: ''
before_build: ''
before_test: ''
build_frontend: 'pip'
build_verbosity: 0
dependency_constraints:
DependencyConstraints(PosixPath('/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Xlang_IO_Dataflow/src/build/gradleenv/1922375555/lib/python3.8/site-packages/cibuildwheel/resources/constraints.txt'))
environment: {'assignments': [SETUPTOOLS_USE_DISTUTILS=stdlib]}
cp38-manylinux_x86_64:
ParsedEnvironment(['SETUPTOOLS_USE_DISTUTILS=stdlib'])
manylinux_images: {'x86_64':
'quay.io/pypa/manylinux2014_x86_64:2022-08-09-51a01be', 'i686':
'quay.io/pypa/manylinux2014_i686:2022-08-09-51a01be', 'pypy_x86_64':
'quay.io/pypa/manylinux2014_x86_64:2022-08-09-51a01be', 'aarch64':
'quay.io/pypa/manylinux2014_aarch64:2022-08-09-51a01be', 'ppc64le':
'quay.io/pypa/manylinux2014_ppc64le:2022-08-09-51a01be', 's390x':
'quay.io/pypa/manylinux2014_s390x:2022-08-09-51a01be', 'pypy_aarch64':
'quay.io/pypa/manylinux2014_aarch64:2022-08-09-51a01be', 'pypy_i686':
'quay.io/pypa/manylinux2014_i686:2022-08-09-51a01be'}
musllinux_images: {'x86_64':
'quay.io/pypa/musllinux_1_1_x86_64:2022-08-09-51a01be', 'i686':
'quay.io/pypa/musllinux_1_1_i686:2022-08-09-51a01be', 'aarch64':
'quay.io/pypa/musllinux_1_1_aarch64:2022-08-09-51a01be', 'ppc64le':
'quay.io/pypa/musllinux_1_1_ppc64le:2022-08-09-51a01be', 's390x':
'quay.io/pypa/musllinux_1_1_s390x:2022-08-09-51a01be'}
repair_command: 'auditwheel repair -w {dest_dir} {wheel}'
test_command: ''
test_extras: ''
test_requires: []
Cache folder: /home/jenkins/.cache/cibuildwheel
Here we go!
Starting container image quay.io/pypa/manylinux2014_x86_64:2022-08-09-51a01be...
info: This container will host the build for cp38-manylinux_x86_64...
8d150088aa9cefe445f06787d3b587c54b7acd5161361ae004891a60fb4c3b36
+ /bin/true
+ mkdir -p /project
✓ 0.48s
Copying project into container...
+ mkdir -p /project
✓ 0.27s
Building cp38-manylinux_x86_64 wheel
CPython 3.8 manylinux x86_64
Setting up build environment...
+ /opt/python/cp38-cp38/bin/python -c 'import sys, json, os;
json.dump(os.environ.copy(), sys.stdout)'
+ which python
+ which pip
✓ 0.13s
Building wheel...
+ rm -rf /tmp/cibuildwheel/built_wheel
+ mkdir -p /tmp/cibuildwheel/built_wheel
+ python -m pip wheel /project --wheel-dir=/tmp/cibuildwheel/built_wheel
--no-deps
> Task :sdks:python:test-suites:dataflow:py311:ioCrossLanguagePythonUsingJava
>>> RUNNING integration tests with pipeline options:
>>> --runner=TestDataflowRunner --project=apache-beam-testing
>>> --region=us-central1
>>> --sdk_container_image=gcr.io/apache-beam-testing/beam-sdk/beam_python3.11_sdk:latest
>>>
>>> --sdk_harness_container_image_overrides=.*java.*,gcr.io/apache-beam-testing/beam-sdk/beam_java8_sdk:latest
>>>
>>> --sdk_location=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Xlang_IO_Dataflow/src/sdks/python/build/apache_beam-2.55.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
>>> pytest options: --capture=no --timeout=4500 --color=yes
>>> --log-cli-level=INFO
>>> collect markers: -m=uses_io_java_expansion_service
[1m============================= test session starts
==============================[0m
platform linux -- Python 3.11.2, pytest-7.4.4, pluggy-1.4.0
rootdir:
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Xlang_IO_Dataflow/src/sdks/python
configfile: pytest.ini
plugins: hypothesis-6.97.0, timeout-2.2.0, requests-mock-1.11.0, xdist-3.5.0
timeout: 4500.0s
timeout method: signal
timeout func_only: False
[1m----------------------------- live log collection
------------------------------[0m
[33mWARNING [0m
apache_beam.runners.interactive.interactive_environment:interactive_environment.py:189
Dependencies required for Interactive Beam PCollection visualization are not
available, please use: `pip install apache-beam[interactive]` to install
necessary dependencies to enable all data visualization features.
[33mWARNING [0m
apache_beam.runners.interactive.interactive_environment:interactive_environment.py:198
You cannot use Interactive Beam features when you are not in an interactive
environment such as a Jupyter notebook or ipython terminal.
[33mWARNING [0m root:avroio_test.py:54 python-snappy is not installed; some
tests will be skipped.
[33mWARNING [0m root:tfrecordio_test.py:55 Tensorflow is not installed, so
skipping some tests.
[32mINFO [0m apache_beam.internal.gcp.auth:auth.py:134 Setting socket
default timeout to 60 seconds.
[32mINFO [0m apache_beam.internal.gcp.auth:auth.py:136 socket default
timeout is 60.0 seconds.
collected 7318 items / 7314 deselected / 17 skipped / 4 selected
apache_beam/io/external/xlang_kafkaio_it_test.py::CrossLanguageKafkaIOTest::test_hosted_kafkaio_null_key
[1m-------------------------------- live log call
---------------------------------[0m
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:322 Copying
Beam SDK
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Xlang_IO_Dataflow/src/sdks/python/build/apache_beam-2.55.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl"
to staging location.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:395 Pipeline
has additional dependencies to be installed in SDK worker container, consider
using the SDK container image pre-building workflow to avoid repetitive
installations. Learn more on
https://cloud.google.com/dataflow/docs/guides/using-custom-containers#prebuild
[32mINFO [0m root:environments.py:314 Using provided Python SDK container
image: gcr.io/apache-beam-testing/beam-sdk/beam_python3.11_sdk:latest
[32mINFO [0m root:environments.py:321 Python SDK container image set to
"gcr.io/apache-beam-testing/beam-sdk/beam_python3.11_sdk:latest" for Docker
environment
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function pack_combiners at 0x7f78dc51fec0>
====================
[32mINFO [0m
apache_beam.runners.portability.fn_api_runner.translations:translations.py:712
==================== <function sort_stages at 0x7f78dc5087c0>
====================
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:677 Starting GCS
upload to
gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0125084832-147032-xz6xnthv.1706172512.147201/beam-sdks-java-io-expansion-service-2.55.0-SNAPSHOT-5ZFuXBfMw1dEzpgbL_ewC6VPcbcJty8XYI6HK5Xlyl4.jar...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:687 Completed GCS
upload to
gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0125084832-147032-xz6xnthv.1706172512.147201/beam-sdks-java-io-expansion-service-2.55.0-SNAPSHOT-5ZFuXBfMw1dEzpgbL_ewC6VPcbcJty8XYI6HK5Xlyl4.jar
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:677 Starting GCS
upload to
gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0125084832-147032-xz6xnthv.1706172512.147201/apache_beam-2.55.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:687 Completed GCS
upload to
gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0125084832-147032-xz6xnthv.1706172512.147201/apache_beam-2.55.0.dev0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:677 Starting GCS
upload to
gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0125084832-147032-xz6xnthv.1706172512.147201/pipeline.pb...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:687 Completed GCS
upload to
gs://dataflow-staging-us-central1-77b801c0838aee13391c0d1885860494/beamapp-jenkins-0125084832-147032-xz6xnthv.1706172512.147201/pipeline.pb
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:858 Create job:
<Job
clientRequestId: '20240125084832148001-1145'
createTime: '2024-01-25T08:48:34.091325Z'
currentStateTime: '1970-01-01T00:00:00Z'
id: '2024-01-25_00_48_33-490873882583629114'
location: 'us-central1'
name: 'beamapp-jenkins-0125084832-147032-xz6xnthv'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2024-01-25T08:48:34.091325Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)>
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:860 Created job
with id: [2024-01-25_00_48_33-490873882583629114]
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:861 Submitted job:
2024-01-25_00_48_33-490873882583629114
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:862 To access the
Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2024-01-25_00_48_33-490873882583629114?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2024-01-25_00_48_33-490873882583629114?project=apache-beam-testing
[32mINFO [0m
apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:58
Console log:
[32mINFO [0m
apache_beam.runners.dataflow.test_dataflow_runner:test_dataflow_runner.py:59
https://console.cloud.google.com/dataflow/jobs/us-central1/2024-01-25_00_48_33-490873882583629114?project=apache-beam-testing
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:151 Job
2024-01-25_00_48_33-490873882583629114 is in state JOB_STATE_RUNNING
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:201
2024-01-25T08:48:36.977Z: JOB_MESSAGE_BASIC: Worker configuration:
e2-standard-2 in us-central1-b.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:201
2024-01-25T08:48:39.071Z: JOB_MESSAGE_BASIC: Executing operation
Generate/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Create
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:201
2024-01-25T08:48:39.122Z: JOB_MESSAGE_BASIC: Starting 1 workers in
us-central1-b...
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:201
2024-01-25T08:48:39.353Z: JOB_MESSAGE_BASIC: Finished operation
Generate/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Create
FATAL: command execution failed
java.io.IOException: Backing channel 'apache-beam-jenkins-13' is disconnected.
at
hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:215)
at
hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285)
at com.sun.proxy.$Proxy197.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1215)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1207)
at hudson.Launcher$ProcStarter.join(Launcher.java:524)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:321)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:818)
at hudson.model.Build$BuildExecution.build(Build.java:199)
at hudson.model.Build$BuildExecution.doRun(Build.java:164)
at
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:526)
at hudson.model.Run.execute(Run.java:1900)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
at hudson.model.ResourceController.execute(ResourceController.java:101)
at hudson.model.Executor.run(Executor.java:442)
Caused by: java.io.IOException: Unexpected termination of the channel
at
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:75)
Caused by: java.io.EOFException
at
java.base/java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2911)
at
java.base/java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3406)
at
java.base/java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:932)
at
java.base/java.io.ObjectInputStream.<init>(ObjectInputStream.java:375)
at
hudson.remoting.ObjectInputStreamEx.<init>(ObjectInputStreamEx.java:49)
at hudson.remoting.Command.readFrom(Command.java:142)
at hudson.remoting.Command.readFrom(Command.java:128)
at
hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35)
at
hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:61)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-13 is offline; cannot locate jdk_1.8_latest
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
