See
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8225/display/redirect?page=changes>
Changes:
[noreply] Merge pull request #15480: [BEAM-12356] Make sure DatasetService is
[noreply] [BEAM-11981] Java Bigtable - Implement IO Request Count metrics
(#15342)
[noreply] [BEAM-12834] Improve Go SDK cross-language documentation and API.
------------------------------------------
[...truncated 1.21 MB...]
while thread.is_alive():
time.sleep(5.0)
# TODO: Merge the termination code in poll_for_job_completion and
# is_in_terminal_state.
terminated = self.is_in_terminal_state()
assert duration or terminated, (
'Job did not reach to a terminal state after waiting indefinitely.')
if terminated and self.state != PipelineState.DONE:
# TODO(BEAM-1290): Consider converting this to an error log based on
# theresolution of the issue.
raise DataflowRuntimeException(
'Dataflow pipeline failed. State: %s, Error:\n%s' %
(self.state, getattr(self._runner, 'last_error_msg', None)),
> self)
E
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow
pipeline failed. State: FAILED, Error:
E Workflow failed.
apache_beam/runners/dataflow/dataflow_runner.py:1635: DataflowRuntimeException
______________ ExerciseMetricsPipelineTest.test_metrics_fnapi_it _______________
[gw0] linux -- Python 3.6.8
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967053/bin/python3.6>
self =
<apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest
testMethod=test_metrics_fnapi_it>
@pytest.mark.it_postcommit
@pytest.mark.it_validatescontainer
def test_metrics_fnapi_it(self):
> result = self.run_pipeline(experiment='beam_fn_api')
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:56:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py:43: in
run_pipeline
return dataflow_exercise_metrics_pipeline.apply_and_run(p)
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline.py:176: in
apply_and_run
result = pipeline.run()
apache_beam/pipeline.py:541: in run
self._options).run(False)
apache_beam/pipeline.py:565: in run
return self.runner.run_pipeline(self, self._options)
apache_beam/runners/dataflow/test_dataflow_runner.py:65: in run_pipeline
self.result.wait_until_finish(duration=wait_duration)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <DataflowPipelineResult <Job
clientRequestId: '20210909064032924743-3474'
createTime: '2021-09-09T06:40:34.263486Z'
...021-09-09T06:40:34.263486Z'
steps: []
tempFiles: []
type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> at 0x7f3a16dc3748>
duration = None
def wait_until_finish(self, duration=None):
if not self.is_in_terminal_state():
if not self.has_job:
raise IOError('Failed to get the Dataflow job id.')
thread = threading.Thread(
target=DataflowRunner.poll_for_job_completion,
args=(self._runner, self, duration))
# Mark the thread as a daemon thread so a keyboard interrupt on the main
# thread will terminate everything. This is also the reason we will not
# use thread.join() to wait for the polling thread.
thread.daemon = True
thread.start()
while thread.is_alive():
time.sleep(5.0)
# TODO: Merge the termination code in poll_for_job_completion and
# is_in_terminal_state.
terminated = self.is_in_terminal_state()
assert duration or terminated, (
'Job did not reach to a terminal state after waiting indefinitely.')
if terminated and self.state != PipelineState.DONE:
# TODO(BEAM-1290): Consider converting this to an error log based on
# theresolution of the issue.
raise DataflowRuntimeException(
'Dataflow pipeline failed. State: %s, Error:\n%s' %
(self.state, getattr(self._runner, 'last_error_msg', None)),
> self)
E
apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow
pipeline failed. State: FAILED, Error:
E Workflow failed.
apache_beam/runners/dataflow/dataflow_runner.py:1635: DataflowRuntimeException
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54:
DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint:
disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62:
DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), #
pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.6_sdk.xml>
-
============== 2 failed, 2 passed, 2 warnings in 1391.42 seconds ===============
cleanup_container
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210909-061615624694614
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:0625caa10afd47b7e56205796982a5610585f461a87ee49840028a8ca5035be4
docker images --format '{{.Repository}}:{{.Tag}}' | grep
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Untagged:
us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:d4940465-1786-4843-8cea-88c07b588b55
Untagged:
us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:70e05e302472e514b35200ee1dcc9ef4d2f7f257f45102eacf1d936efc745fcf
Deleted: sha256:ca9db12e0dba559e6624655d8279cb0c32b99c2275a6baa55c65ac05b5fe7d2b
Deleted: sha256:d691a0953fe7ed42c766ea540f76579b47035f719691bbd8885862156dae45da
Deleted: sha256:e75e97dc10d0692dddc7934b53cefc135ff31da16c0202eedd7ee04fb7dd7ab4
Deleted: sha256:1c7ba57f0d9b85973910335192c33e3897ee22e642a9632d3fb5cf67fd991556
Deleted: sha256:51366c8f425c46da9d79db81323150ac5e54de5e8ee795512da00f7bc4d89eb5
Deleted: sha256:a65c5f19a348ad8282bd06748d5baa6e1bd98924ddc2ee54800e78d6a842ae59
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:0625caa10afd47b7e56205796982a5610585f461a87ee49840028a8ca5035be4
Associated tags:
- 20210909-061615624694614
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210909-061615624694614
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210909-061615624694614].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:0625caa10afd47b7e56205796982a5610585f461a87ee49840028a8ca5035be4].
gcloud container images list-tags
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk
--format="get(digest)"
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:70e05e302472e514b35200ee1dcc9ef4d2f7f257f45102eacf1d936efc745fcf
Associated tags:
- d4940465-1786-4843-8cea-88c07b588b55
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:d4940465-1786-4843-8cea-88c07b588b55].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:70e05e302472e514b35200ee1dcc9ef4d2f7f257f45102eacf1d936efc745fcf].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:f3d056a4eb83562510c8fd2a3aac194f30217676e7dda72d7d21489dfbd9ca37
Associated tags:
- eda9e51b-2a95-455f-8fa2-738a90152720
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk:eda9e51b-2a95-455f-8fa2-738a90152720].
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED
Untagged:
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:68ee563b-8f4e-40bb-a54d-9263d1176511
Untagged:
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:6d3b42f3e25f10844307084ce5b82e2b8ec4a9e0e0e3034732c305e8f609a040
Deleted: sha256:c457bdadf6166572d5c9d21476a7323dc9a4167e023fe8a416c98a0160b88b9b
Deleted: sha256:f033c1e1c2d024e595f885f87a2011665673f526878e8f6435af3b513ac3af87
Deleted: sha256:924f12b8568cfc74258ce0d9cf69df1d7afe60099d23779b78fab994373c4e93
Deleted: sha256:2f117f1d61dca1c898ee046d862029f49b1258c8cdb464383d0934f4baea1fa3
Deleted: sha256:a1dace58de4d2a78fd6dadf924881ecdc0528d5ded2812e63da4cadc311182a0
Deleted: sha256:5fc93132abd98079287103cd3c5255f4156b5c43a62d3cb5a7ce03a77183ab37
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:d04d2063b96efae5d3a63e7ec6e116ccb267874c8d200060eefe17a6a8a0bc48
Associated tags:
- 20210909-061614777953047
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210909-061614777953047
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20210909-061614777953047].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:d04d2063b96efae5d3a63e7ec6e116ccb267874c8d200060eefe17a6a8a0bc48].
gcloud container images list-tags
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk
--format="get(digest)"
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:6d3b42f3e25f10844307084ce5b82e2b8ec4a9e0e0e3034732c305e8f609a040
Associated tags:
- 68ee563b-8f4e-40bb-a54d-9263d1176511
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:68ee563b-8f4e-40bb-a54d-9263d1176511].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:6d3b42f3e25f10844307084ce5b82e2b8ec4a9e0e0e3034732c305e8f609a040].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:dc6c2d564c8771a36669134d47b020211543c186af8c1908af39afe79b0f292a
Associated tags:
- 46527acb-8a75-4dd8-acc1-9f2b7090a82e
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:46527acb-8a75-4dd8-acc1-9f2b7090a82e].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:dc6c2d564c8771a36669134d47b020211543c186af8c1908af39afe79b0f292a].
Removed the container
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer FAILED
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python36_sdk/beam_python_prebuilt_sdk@sha256:f3d056a4eb83562510c8fd2a3aac194f30217676e7dda72d7d21489dfbd9ca37].
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:e40d22509f88a26866b00bd6784869613ba8f7a8bbe9eb98fa063e31a2cec953
Associated tags:
- 20210909-061617872034472
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210909-061617872034472
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20210909-061617872034472].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:e40d22509f88a26866b00bd6784869613ba8f7a8bbe9eb98fa063e31a2cec953].
gcloud container images list-tags
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk
--format="get(digest)"
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:2cf02936e0fce6a39afd0f1c6fab22f94bb9ad8a71fc6b275e4273be63bdce3a
Associated tags:
- 2b8551d0-7f3b-483a-83a6-8d1586360972
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:2b8551d0-7f3b-483a-83a6-8d1586360972].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:2cf02936e0fce6a39afd0f1c6fab22f94bb9ad8a71fc6b275e4273be63bdce3a].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:c539c4375efb7745098674c42f019ecc465bbcbb2df804d48a6c86643accbc4d
Associated tags:
- 53cbabe9-d75d-4737-94d0-7d8b094d3ea9
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:53cbabe9-d75d-4737-94d0-7d8b094d3ea9].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:c539c4375efb7745098674c42f019ecc465bbcbb2df804d48a6c86643accbc4d].
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 276
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 276
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 276
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 44m 57s
44 actionable tasks: 40 executed, 4 up-to-date
Publishing build scan...
https://gradle.com/s/4a3cvp4mvgxmm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
