See
<https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow_V2/3250/display/redirect?page=changes>
Changes:
[egalpin] [BEAM-14003] Adds compat for Elasticsearch 8.0.0
[egalpin] [BEAM-13136] Removes support for Elasticsearch 2.x
[noreply] Merge pull request #17149 from [BEAM-13883] [Playground] Increase test
------------------------------------------
[...truncated 80.06 KB...]
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_list_twice
[gw1] [32mPASSED[0m
apache_beam/transforms/combinefn_lifecycle_test.py::CombineFnLifecycleTest::test_non_liftable_combine
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_dict_twice
[gw6] [32mPASSED[0m
apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_multiple_pcollections_having_multiple_consumers
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_singleton_without_unique_labels
[gw5] [32mPASSED[0m
apache_beam/transforms/ptransform_test.py::PTransformTest::test_flatten_a_flattened_pcollection
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_singleton_with_different_defaults
[gw4] [32mPASSED[0m
apache_beam/transforms/dofn_lifecycle_test.py::DoFnLifecycleTest::test_dofn_lifecycle
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_default_value_singleton_side_input
[gw3] [32mPASSED[0m
apache_beam/transforms/ptransform_test.py::PTransformTest::test_multiple_empty_outputs
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_empty_singleton_side_input
[gw7] [32mPASSED[0m
apache_beam/transforms/ptransform_test.py::PTransformTest::test_undeclared_outputs
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_iterable_side_input
[gw2] [32mPASSED[0m
apache_beam/transforms/ptransform_test.py::PTransformTest::test_par_do_with_multiple_outputs_and_using_yield
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_flattened_side_input
[gw1] [32mPASSED[0m
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_dict_twice
apache_beam/transforms/util_test.py::ReshuffleTest::test_reshuffle_preserves_timestamps
[gw0] [32mPASSED[0m
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_list_twice
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_reiterable_side_input
[gw6] [32mPASSED[0m
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_singleton_without_unique_labels
[gw4] [32mPASSED[0m
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_default_value_singleton_side_input
[gw5] [32mPASSED[0m
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_as_singleton_with_different_defaults
[gw3] [32mPASSED[0m
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_empty_singleton_side_input
[gw2] [32mPASSED[0m
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_flattened_side_input
[gw1] [32mPASSED[0m
apache_beam/transforms/util_test.py::ReshuffleTest::test_reshuffle_preserves_timestamps
[gw0] [32mPASSED[0m
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_reiterable_side_input
[gw7] [32mPASSED[0m
apache_beam/transforms/sideinputs_test.py::SideInputsTest::test_iterable_side_input
[33m=============================== warnings summary
===============================[0m
<unknown>:54
<unknown>:54
<unknown>:54
<unknown>:54
<unknown>:54
<unknown>:54
<unknown>:54
<unknown>:54
<unknown>:54: DeprecationWarning: invalid escape sequence \c
<unknown>:62
<unknown>:62
<unknown>:62
<unknown>:62
<unknown>:62
<unknown>:62
<unknown>:62
<unknown>:62
<unknown>:62: DeprecationWarning: invalid escape sequence \d
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py:42
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py:42
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py:42
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py:42
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py:42
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py:42
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py:42
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py:42
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py:42:
DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use
"async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/runners/direct/transform_evaluator.py:1029
apache_beam/runners/direct/transform_evaluator.py:1029
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/sdks/python/apache_beam/runners/direct/transform_evaluator.py:1029:
DeprecationWarning: Using or importing the ABCs from 'collections' instead of
from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop
working
isinstance(element.value, collections.Iterable) and
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file:
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/sdks/python/pytest_validatesRunnerStreamingTests-df-py39-xdist.xml
-
[33m[1m============= 32 passed, 1 skipped, 26 warnings in 3057.49 seconds
=============[0m
>>> RUNNING integration tests with pipeline options:
>>> --runner=TestDataflowRunner --project=apache-beam-testing
>>> --region=us-central1
>>> --staging_location=gs://temp-storage-for-end-to-end-tests/staging-it
>>> --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it
>>> --output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
>>> --sdk_location=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/sdks/python/build/apache-beam.tar.gz
>>> --requirements_file=postcommit_requirements.txt --num_workers=1
>>> --sleep_secs=20 --streaming
>>> --dataflow_worker_jar=/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/runners/google-cloud-dataflow-java/worker/build/libs/beam-runners-google-cloud-dataflow-java-fn-api-worker-2.39.0-SNAPSHOT.jar
>>> --experiments=use_runner_v2 --experiments=shuffle_mode=appliance
>>> --enable_streaming_engine
>>> pytest options: --capture=no --timeout=4500 --color=yes
>>> --log-cli-level=INFO
>>> collect markers: -m=it_validatesrunner and not no_sickbay_streaming and
>>> no_xdist
[1m============================= test session starts
==============================[0m
platform linux -- Python 3.9.10, pytest-4.6.11, py-1.11.0, pluggy-0.13.1
rootdir:
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/sdks/python,
inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.4.0, requests-mock-1.9.3
timeout: 4500.0s
timeout method: signal
timeout func_only: False
[1m----------------------------- live log collection
------------------------------[0m
[33mWARNING [0m root:avroio_test.py:51 python-snappy is not installed; some
tests will be skipped.
[33mWARNING [0m root:tfrecordio_test.py:55 Tensorflow is not installed, so
skipping some tests.
[33mWARNING [0m
apache_beam.runners.interactive.interactive_environment:interactive_environment.py:190
Dependencies required for Interactive Beam PCollection visualization are not
available, please use: `pip install apache-beam[interactive]` to install
necessary dependencies to enable all data visualization features.
[33mWARNING [0m
apache_beam.runners.interactive.interactive_environment:interactive_environment.py:199
You cannot use Interactive Beam features when you are not in an interactive
environment such as a Jupyter notebook or ipython terminal.
[33mWARNING [0m root:environments.py:371 Make sure that locally built Python
SDK docker image has Python 3.9 interpreter.
[32mINFO [0m root:environments.py:380 Default Python SDK image for
environment is apache/beam_python3.9_sdk:2.39.0.dev
collected 5257 items / 5256 deselected / 1 selected
apache_beam/runners/dataflow/dataflow_exercise_streaming_metrics_pipeline_test.py::ExerciseStreamingMetricsPipelineTest::test_streaming_pipeline_returns_expected_user_metrics_fnapi_it
[1m-------------------------------- live log call
---------------------------------[0m
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:754
Executing command:
['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/build/gradleenv/-1734967050/bin/python3.9',
'-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r',
'/tmp/tmpmdqde9zv/tmp_requirements.txt', '--exists-action', 'i', '--no-deps',
'--implementation', 'cp', '--abi', 'cp39', '--platform', 'manylinux2014_x86_64']
E0330 19:46:26.995738512 2645429 fork_posix.cc:70] Fork support is
only compatible with the epoll1 and poll polling strategies
[32mINFO [0m apache_beam.runners.portability.stager:stager.py:325 Copying
Beam SDK
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/sdks/python/build/apache-beam.tar.gz"
to staging location.
[33mWARNING [0m root:environments.py:371 Make sure that locally built Python
SDK docker image has Python 3.9 interpreter.
[32mINFO [0m root:environments.py:380 Default Python SDK image for
environment is apache/beam_python3.9_sdk:2.39.0.dev
[32mINFO [0m root:environments.py:295 Using provided Python SDK container
image: gcr.io/cloud-dataflow/v1beta3/python39-fnapi:beam-master-20220208
[32mINFO [0m root:environments.py:302 Python SDK container image set to
"gcr.io/cloud-dataflow/v1beta3/python39-fnapi:beam-master-20220208" for Docker
environment
[32mINFO [0m apache_beam.internal.gcp.auth:auth.py:122 Setting socket
default timeout to 60 seconds.
[32mINFO [0m apache_beam.internal.gcp.auth:auth.py:124 socket default
timeout is 60.0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/requirements.txt...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/requirements.txt
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/pickled_main_session...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/pickled_main_session
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/pbr-5.8.0.tar.gz...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/pbr-5.8.0.tar.gz
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/pbr-5.8.1.tar.gz...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/pbr-5.8.1.tar.gz
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/mock-2.0.0.tar.gz...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/mock-2.0.0.tar.gz
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/six-1.16.0.tar.gz...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/six-1.16.0.tar.gz
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/soupsieve-2.3.1.tar.gz...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/soupsieve-2.3.1.tar.gz
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/PyHamcrest-1.10.1.tar.gz...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/PyHamcrest-1.10.1.tar.gz
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/parameterized-0.7.5.tar.gz...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/parameterized-0.7.5.tar.gz
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/beautifulsoup4-4.10.0.tar.gz...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/beautifulsoup4-4.10.0.tar.gz
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/mock-2.0.0-py2.py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/mock-2.0.0-py2.py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/seaborn-0.11.2-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/seaborn-0.11.2-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/PyHamcrest-1.10.1-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/PyHamcrest-1.10.1-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/beautifulsoup4-4.10.0-py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/beautifulsoup4-4.10.0-py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/parameterized-0.7.5-py2.py3-none-any.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/parameterized-0.7.5-py2.py3-none-any.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/matplotlib-3.3.4-cp36-cp36m-manylinux1_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/matplotlib-3.3.4-cp36-cp36m-manylinux1_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/matplotlib-3.5.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/matplotlib-3.5.1-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/matplotlib-3.5.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/matplotlib-3.5.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/matplotlib-3.5.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/matplotlib-3.5.1-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.whl
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/dataflow_python_sdk.tar...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/dataflow_python_sdk.tar
in 0 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/dataflow-worker.jar...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/dataflow-worker.jar
in 7 seconds.
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:710 Starting GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/pipeline.pb...
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:726 Completed GCS
upload to
gs://temp-storage-for-end-to-end-tests/staging-it/beamapp-jenkins-0330194628-009643-nthvafm2.1648669588.009833/pipeline.pb
in 0 seconds.
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:335
Discarding unparseable args: ['--sleep_secs=20']
[33mWARNING [0m apache_beam.options.pipeline_options:pipeline_options.py:335
Discarding unparseable args: ['--sleep_secs=20']
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:890 Create job:
<Job
clientRequestId: '20220330194628010996-2535'
createTime: '2022-03-30T19:46:41.536768Z'
currentStateTime: '1970-01-01T00:00:00Z'
id:
'2022-03-30_12_46_41-7472780305117069098'
location: 'us-central1'
name: 'beamapp-jenkins-0330194628-009643-nthvafm2'
projectId: 'apache-beam-testing'
stageStates: []
startTime: '2022-03-30T19:46:41.536768Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_STREAMING, 2)>
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:892 Created job
with id: [2022-03-30_12_46_41-7472780305117069098]
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:893 Submitted job:
2022-03-30_12_46_41-7472780305117069098
[32mINFO [0m
apache_beam.runners.dataflow.internal.apiclient:apiclient.py:894 To access the
Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-30_12_46_41-7472780305117069098?project=apache-beam-testing
Worker logs:
https://console.cloud.google.com/dataflow/jobs/us-central1/2022-03-30_12_46_41-7472780305117069098?project=apache-beam-testing
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:191 Job
2022-03-30_12_46_41-7472780305117069098 is in state JOB_STATE_RUNNING
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:42.240Z: JOB_MESSAGE_WARNING: Autoscaling is enabled for
Dataflow Streaming Engine. Workers will scale between 1 and 100 unless
maxNumWorkers is specified.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:42.708Z: JOB_MESSAGE_DETAILED: Autoscaling is enabled for job
2022-03-30_12_46_41-7472780305117069098. The number of workers will be between
1 and 100.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:42.742Z: JOB_MESSAGE_DETAILED: Autoscaling was automatically
enabled for job 2022-03-30_12_46_41-7472780305117069098.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:48.307Z: JOB_MESSAGE_BASIC: Worker configuration:
e2-standard-2 in us-central1-b.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:48.979Z: JOB_MESSAGE_DETAILED: Expanding SplittableParDo
operations into optimizable parts.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.023Z: JOB_MESSAGE_DETAILED: Expanding CollectionToSingleton
operations into optimizable parts.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.070Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey
operations into optimizable parts.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.118Z: JOB_MESSAGE_DETAILED: Expanding
SplittableProcessKeyed operations into optimizable parts.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.160Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations
into streaming Read/Write steps
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.196Z: JOB_MESSAGE_DETAILED: Lifting
ValueCombiningMappingFns into MergeBucketsMappingFns
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.220Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner
information.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.253Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read,
Write, and Flatten operations
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.283Z: JOB_MESSAGE_DETAILED: Fusing consumer
generate_metrics into ReadFromPubSub/Read
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.318Z: JOB_MESSAGE_DETAILED: Fusing consumer
dump_to_pub/ToProtobuf into generate_metrics
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.354Z: JOB_MESSAGE_DETAILED: Fusing consumer
dump_to_pub/Write into dump_to_pub/ToProtobuf
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.394Z: JOB_MESSAGE_BASIC: Running job using Streaming Engine
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.434Z: JOB_MESSAGE_DEBUG: Workflow config is missing a
default resource spec.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.460Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and
teardown to workflow graph.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.492Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop
steps.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.527Z: JOB_MESSAGE_DEBUG: Assigning stage ids.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.592Z: JOB_MESSAGE_DEBUG: Starting worker pool setup.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.624Z: JOB_MESSAGE_DEBUG: Starting worker pool setup.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:46:49.654Z: JOB_MESSAGE_BASIC: Starting 1 workers in
us-central1-b...
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:47:07.081Z: JOB_MESSAGE_BASIC: Your project already contains 100
Dataflow-created metric descriptors, so new user metrics of the form
custom.googleapis.com/* will not be created. However, all user metrics are also
available in the metric dataflow.googleapis.com/job/user_counter. If you rely
on the custom metrics, you can delete old / unused metric descriptors. See
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:47:17.963Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number
of workers to 1 so that the pipeline can catch up with its backlog and keep up
with its input rate.
[32mINFO [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:236
2022-03-30T19:47:51.849Z: JOB_MESSAGE_DETAILED: Workers have started
successfully.
[33mWARNING [0m
apache_beam.runners.dataflow.dataflow_runner:dataflow_runner.py:247 Timing out
on waiting for job 2022-03-30_12_46_41-7472780305117069098 after 60 seconds
[32mPASSED[0m
[33m=============================== warnings summary
===============================[0m
<unknown>:54
<unknown>:54: DeprecationWarning: invalid escape sequence \c
<unknown>:62
<unknown>:62: DeprecationWarning: invalid escape sequence \d
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py:42
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py:42:
DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use
"async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file:
/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow_V2/src/sdks/python/pytest_validatesRunnerStreamingTests-df-py39-noxdist.xml
-
[33m[1m=========== 1 passed, 5256 deselected, 3 warnings in 684.73 seconds
============[0m
> Task :sdks:python:test-suites:dataflow:validatesRunnerStreamingTestsV2
BUILD SUCCESSFUL in 1h 52m 47s
93 actionable tasks: 58 executed, 33 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/l4z3dql3irzkw
FATAL: command execution failed
hudson.remoting.ChannelClosedException: Channel
"hudson.remoting.Channel@7670020:apache-beam-jenkins-11": Remote call on
apache-beam-jenkins-11 failed. The channel is closing down or has closed down
at hudson.remoting.Channel.call(Channel.java:994)
at
hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:286)
at com.sun.proxy.$Proxy132.isAlive(Unknown Source)
at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1213)
at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1205)
at hudson.Launcher$ProcStarter.join(Launcher.java:522)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:317)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at
hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:806)
at hudson.model.Build$BuildExecution.build(Build.java:198)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:514)
at hudson.model.Run.execute(Run.java:1888)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:99)
at hudson.model.Executor.run(Executor.java:432)
Caused by: java.io.IOException
at hudson.remoting.Channel.close(Channel.java:1477)
at hudson.remoting.Channel.close(Channel.java:1454)
at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:894)
at hudson.slaves.SlaveComputer.access$100(SlaveComputer.java:108)
at hudson.slaves.SlaveComputer$2.run(SlaveComputer.java:774)
at
jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
at
jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
ERROR: apache-beam-jenkins-11 is offline; cannot locate jdk_1.8_latest
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
