See
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8231/display/redirect?page=changes>
Changes:
[noreply] Added type annotations to some combiners missing it. (#15414)
[noreply] [BEAM-12634] JmsIO auto scaling feature (#15464)
[noreply] [BEAM-12662] Get Flink version from cluster. (#15223)
------------------------------------------
[...truncated 123.67 KB...]
while thread.is_alive():
time.sleep(5.0)
# TODO: Merge the termination code in poll_for_job_completion and
# is_in_terminal_state.
terminated = self.is_in_terminal_state()
assert duration or terminated, (
'Job did not reach to a terminal state after waiting indefinitely.')
if terminated and self.state != PipelineState.DONE:
# TODO(BEAM-1290): Consider converting this to an error log based on
# theresolution of the issue.
raise DataflowRuntimeException(
'Dataflow pipeline failed. State: %s, Error:\n%s' %
(self.state, getattr(self._runner, 'last_error_msg', None)),
> self)
E
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow
pipeline failed. State: FAILED, Error:
E Workflow failed.
apache_beam/runners/dataflow/dataflow_runner.py:1635: DataflowRuntimeException
______________ ExerciseMetricsPipelineTest.test_metrics_fnapi_it _______________
[gw0] linux -- Python 3.7.3
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self =
<apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest
testMethod=test_metrics_fnapi_it>
@pytest.mark.it_postcommit
@pytest.mark.it_validatescontainer
def test_metrics_fnapi_it(self):
> result = self.run_pipeline(experiment='beam_fn_api')
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:56:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:43: in
run_pipeline
return dataflow_exercise_metrics_pipeline.apply_and_run(p)
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline.py:176: in
apply_and_run
result = pipeline.run()
apache_beam/pipeline.py:542: in run
self._options).run(False)
apache_beam/pipeline.py:566: in run
return self.runner.run_pipeline(self, self._options)
apache_beam/runners/dataflow/test_dataflow_runner.py:65: in run_pipeline
self.result.wait_until_finish(duration=wait_duration)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <DataflowPipelineResult <Job
clientRequestId: '20210910182558506794-5768'
createTime: '2021-09-10T18:25:59.949896Z'
...021-09-10T18:25:59.949896Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> at 0x7efeefa46cf8>
duration = None
def wait_until_finish(self, duration=None):
if not self.is_in_terminal_state():
if not self.has_job:
raise IOError('Failed to get the Dataflow job id.')
thread = threading.Thread(
target=DataflowRunner.poll_for_job_completion,
args=(self._runner, self, duration))
# Mark the thread as a daemon thread so a keyboard interrupt on the main
# thread will terminate everything. This is also the reason we will not
# use thread.join() to wait for the polling thread.
thread.daemon = True
thread.start()
while thread.is_alive():
time.sleep(5.0)
# TODO: Merge the termination code in poll_for_job_completion and
# is_in_terminal_state.
terminated = self.is_in_terminal_state()
assert duration or terminated, (
'Job did not reach to a terminal state after waiting indefinitely.')
if terminated and self.state != PipelineState.DONE:
# TODO(BEAM-1290): Consider converting this to an error log based on
# theresolution of the issue.
raise DataflowRuntimeException(
'Dataflow pipeline failed. State: %s, Error:\n%s' %
(self.state, getattr(self._runner, 'last_error_msg', None)),
> self)
E
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow
pipeline failed. State: FAILED, Error:
E Workflow failed.
apache_beam/runners/dataflow/dataflow_runner.py:1635: DataflowRuntimeException
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54:
DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint:
disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62:
DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), #
pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.7_sdk.xml>
-
==================== 4 failed, 2 warnings in 791.64 seconds ====================
cleanup_container
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210910-181217005449927
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:68e510d62bc7e21adc241c4bf5e7811d4ab35f681d2f542e21c574f901367bde
docker images --format '{{.Repository}}:{{.Tag}}' | grep
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
[gw0] [ 50%] FAILED
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_cloud_build
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_local_docker
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED
Untagged:
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:a6a32375-76cb-44ac-886f-b368ecddaed2
Untagged:
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:d8e01f4f892910be1185a57abd9fb667a75a278262e3748ab1cb5ea982896b51
Deleted: sha256:ed2431b7ce8bc4651b33def5b49bf637314b6c73b2ac904b2dae988bfd807cb1
Deleted: sha256:ad2553f69ef8a3fce5d880da7c248867cfe203d26d5b72c9c362127e52f53e4e
Deleted: sha256:5ad64f455985fff81f86af27951679794a7ff8e08ba481301fedc0c20615a8fc
Deleted: sha256:18e0ba12141ca62cf8f6e63bd28ed3830af79efc0682dae4b9357ac329a2af01
Deleted: sha256:277d788677def706087298cf73f07407816d0c6d9e96ffae7eb8e646f4d5a826
Deleted: sha256:0fde726c6313ed355318f08b3a4737b226f5fccdfe23d93b00150defe24ac566
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:68e510d62bc7e21adc241c4bf5e7811d4ab35f681d2f542e21c574f901367bde
Associated tags:
- 20210910-181217005449927
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210910-181217005449927
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210910-181217005449927].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:68e510d62bc7e21adc241c4bf5e7811d4ab35f681d2f542e21c574f901367bde].
gcloud container images list-tags
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk
--format="get(digest)"
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:d8e01f4f892910be1185a57abd9fb667a75a278262e3748ab1cb5ea982896b51
Associated tags:
- a6a32375-76cb-44ac-886f-b368ecddaed2
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:a6a32375-76cb-44ac-886f-b368ecddaed2].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:d8e01f4f892910be1185a57abd9fb667a75a278262e3748ab1cb5ea982896b51].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:adcd186acc19586d0948dfda7db8548998fcb82001c18703f955c33888fc16e7
Associated tags:
- 3ee4b483-cb80-4852-ac85-4bb896266e89
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:3ee4b483-cb80-4852-ac85-4bb896266e89].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:adcd186acc19586d0948dfda7db8548998fcb82001c18703f955c33888fc16e7].
Removed the container
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:4b2ff9afbc79b123c7caf6552dc80d6f1afe744cb5451c4fb618ed17ac1cf3a9
Associated tags:
- 20210910-181216996579364
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210910-181216996579364
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210910-181216996579364].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:4b2ff9afbc79b123c7caf6552dc80d6f1afe744cb5451c4fb618ed17ac1cf3a9].
gcloud container images list-tags
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk
--format="get(digest)"
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:dc9ead970e1982183d6aa5fca0b58b1439c01517aaa77658819c0b684cfb21a8
Associated tags:
- b76958f5-bd0d-4426-84f5-a4364c6ade57
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:b76958f5-bd0d-4426-84f5-a4364c6ade57].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:dc9ead970e1982183d6aa5fca0b58b1439c01517aaa77658819c0b684cfb21a8].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:60ed9a1612f7a2d9505150ac8daf0f29a1dff74c9e059ddeedcd17a8ee5d271e
Associated tags:
- cd0be990-f07f-4810-8b4a-9a31679862f7
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:cd0be990-f07f-4810-8b4a-9a31679862f7].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:60ed9a1612f7a2d9505150ac8daf0f29a1dff74c9e059ddeedcd17a8ee5d271e].
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:48be161bbb14d1703b3dc2b83e6754b8bde69ca099e59200dc39b93fe0ee53c9
Associated tags:
- 20210910-181223269403059
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210910-181223269403059
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210910-181223269403059].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:48be161bbb14d1703b3dc2b83e6754b8bde69ca099e59200dc39b93fe0ee53c9].
gcloud container images list-tags
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk
--format="get(digest)"
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:3b03d57a6596920b80cd3fb949e9368304167a53aa8aaa1b60e083f0006f3a9d
Associated tags:
- 086e399e-5ec2-459b-9dee-468d8bf03646
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:086e399e-5ec2-459b-9dee-468d8bf03646].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:3b03d57a6596920b80cd3fb949e9368304167a53aa8aaa1b60e083f0006f3a9d].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:c2bcc129720a5ca06cd66620b8c6d0fa9d69e1e159e8b957cca67d8c76118d99
Associated tags:
- 61831813-3965-4121-9ce8-8a3910a197de
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:61831813-3965-4121-9ce8-8a3910a197de].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:c2bcc129720a5ca06cd66620b8c6d0fa9d69e1e159e8b957cca67d8c76118d99].
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 276
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 276
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 276
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 41m 1s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/y3x3wk5jt3i5i
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
