See
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/8675/display/redirect?page=changes>
Changes:
[mmack] [BEAM-13009] Fix tests to ensure absence of duplicates only per request
[mmack] [adhoc] Use predefined min/max combiners in SQS reader to gather stats
------------------------------------------
[...truncated 1.15 MB...]
artifacts = Stager.create_job_resources(self._options, tmp)
resources = Stager.extract_staging_tuple_iter(artifacts)
# make a copy of the staged artifacts into the temp source folder.
file_names = []
> for path, name in resources:
E ValueError: too many values to unpack (expected 2)
apache_beam/runners/portability/sdk_container_builder.py:100: ValueError
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54:
DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint:
disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62:
DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), #
pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.6_sdk.xml>
-
=============== 2 failed, 2 passed, 2 warnings in 861.39 seconds ===============
cleanup_container
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20211230-181526629955138
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:66427024c0ef5a4709f278545ec90bec99707620df4821a5ae36d0a02d5fb8bd
Deleted: sha256:5b49ab0bcc4f997083938c38119413802cc4247313114c284227f10d9d0ec25f
Deleted: sha256:bda73fc2400d16bbefccf36c975691853e9d15b0f3bdd797d121aac0d221c3c1
Deleted: sha256:eff19cf3ce323a42fec1e631101afaad884e56637083c615580d444e2237efe7
Deleted: sha256:c795c1ab2a8033c2e08740605a8b8dffd76a1a1fbf0d4d381f7fbc034d3c0da7
docker images --format '{{.Repository}}:{{.Tag}}' | grep
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:66427024c0ef5a4709f278545ec90bec99707620df4821a5ae36d0a02d5fb8bd
Associated tags:
- 20211230-181526629955138
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20211230-181526629955138
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20211230-181526629955138].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:66427024c0ef5a4709f278545ec90bec99707620df4821a5ae36d0a02d5fb8bd].
gcloud container images list-tags
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk
--format="get(digest)"
Removed the container
> Task :sdks:python:test-suites:dataflow:py36:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer
[gw0] [100%] PASSED
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it
=================================== FAILURES ===================================
____ WordCountIT.test_wordcount_it_with_prebuilt_sdk_container_cloud_build _____
[gw0] linux -- Python 3.7.10
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.wordcount_it_test.WordCountIT
testMethod=test_wordcount_it_with_prebuilt_sdk_container_cloud_build>
@pytest.mark.it_validatescontainer
def test_wordcount_it_with_prebuilt_sdk_container_cloud_build(self):
self._run_wordcount_it(
wordcount.run,
experiment='beam_fn_api',
> prebuild_sdk_container_engine='cloud_build')
apache_beam/examples/wordcount_it_test.py:67:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/wordcount_it_test.py:109: in _run_wordcount_it
save_main_session=False,
apache_beam/examples/wordcount.py:89: in run
output | 'Write' >> WriteToText(known_args.output)
apache_beam/pipeline.py:596: in __exit__
self.result = self.run()
apache_beam/pipeline.py:549: in run
self._options).run(False)
apache_beam/pipeline.py:573: in run
return self.runner.run_pipeline(self, self._options)
apache_beam/runners/dataflow/test_dataflow_runner.py:53: in run_pipeline
self.result = super().run_pipeline(pipeline, options)
apache_beam/runners/dataflow/dataflow_runner.py:459: in run_pipeline
environments.DockerEnvironment.from_options(options))
apache_beam/transforms/environments.py:341: in from_options
options)
apache_beam/runners/portability/sdk_container_builder.py:139: in
build_container_image
return builder._build()
apache_beam/runners/portability/sdk_container_builder.py:89: in _build
self._prepare_dependencies()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
<apache_beam.runners.portability.sdk_container_builder._SdkContainerImageCloudBuilder
object at 0x7f35daa23290>
def _prepare_dependencies(self):
with tempfile.TemporaryDirectory() as tmp:
artifacts = Stager.create_job_resources(self._options, tmp)
resources = Stager.extract_staging_tuple_iter(artifacts)
# make a copy of the staged artifacts into the temp source folder.
file_names = []
> for path, name in resources:
E ValueError: too many values to unpack (expected 2)
apache_beam/runners/portability/sdk_container_builder.py:100: ValueError
____ WordCountIT.test_wordcount_it_with_prebuilt_sdk_container_local_docker ____
[gw0] linux -- Python 3.7.10
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967052/bin/python3.7>
self = <apache_beam.examples.wordcount_it_test.WordCountIT
testMethod=test_wordcount_it_with_prebuilt_sdk_container_local_docker>
@pytest.mark.it_validatescontainer
def test_wordcount_it_with_prebuilt_sdk_container_local_docker(self):
self._run_wordcount_it(
wordcount.run,
experiment='beam_fn_api',
> prebuild_sdk_container_engine='local_docker')
apache_beam/examples/wordcount_it_test.py:60:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/examples/wordcount_it_test.py:109: in _run_wordcount_it
save_main_session=False,
apache_beam/examples/wordcount.py:89: in run
output | 'Write' >> WriteToText(known_args.output)
apache_beam/pipeline.py:596: in __exit__
self.result = self.run()
apache_beam/pipeline.py:549: in run
self._options).run(False)
apache_beam/pipeline.py:573: in run
return self.runner.run_pipeline(self, self._options)
apache_beam/runners/dataflow/test_dataflow_runner.py:53: in run_pipeline
self.result = super().run_pipeline(pipeline, options)
apache_beam/runners/dataflow/dataflow_runner.py:459: in run_pipeline
environments.DockerEnvironment.from_options(options))
apache_beam/transforms/environments.py:341: in from_options
options)
apache_beam/runners/portability/sdk_container_builder.py:139: in
build_container_image
return builder._build()
apache_beam/runners/portability/sdk_container_builder.py:89: in _build
self._prepare_dependencies()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self =
<apache_beam.runners.portability.sdk_container_builder._SdkContainerImageLocalBuilder
object at 0x7f35da6e0c90>
def _prepare_dependencies(self):
with tempfile.TemporaryDirectory() as tmp:
artifacts = Stager.create_job_resources(self._options, tmp)
resources = Stager.extract_staging_tuple_iter(artifacts)
# make a copy of the staged artifacts into the temp source folder.
file_names = []
> for path, name in resources:
E ValueError: too many values to unpack (expected 2)
apache_beam/runners/portability/sdk_container_builder.py:100: ValueError
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54:
DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint:
disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62:
DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), #
pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.7_sdk.xml>
-
=============== 2 failed, 2 passed, 2 warnings in 905.81 seconds ===============
cleanup_container
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20211230-181525992976995
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:fd00c9065cb6525dcee9a947a743808e0907c684cd2d9e7d4a8446f8123d620b
docker images --format '{{.Repository}}:{{.Tag}}' | grep
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:fd00c9065cb6525dcee9a947a743808e0907c684cd2d9e7d4a8446f8123d620b
Associated tags:
- 20211230-181525992976995
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20211230-181525992976995
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20211230-181525992976995].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:fd00c9065cb6525dcee9a947a743808e0907c684cd2d9e7d4a8446f8123d620b].
gcloud container images list-tags
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk
--format="get(digest)"
Removed the container
> Task :sdks:python:test-suites:dataflow:py37:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
cleanup_container
docker images --format '{{.Repository}}:{{.Tag}}' | grep
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:399ad41831fa13067a6ec50ec888f160531966d7672dfe298b98dd89d9aa96e5
Associated tags:
- 20211230-181523907959512
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20211230-181523907959512
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20211230-181523907959512].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:399ad41831fa13067a6ec50ec888f160531966d7672dfe298b98dd89d9aa96e5].
gcloud container images list-tags
$PREBUILD_SDK_CONTAINER_REGISTRY_PATH/beam_python_prebuilt_sdk
--format="get(digest)"
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 321
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py36:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 321
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 321
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 32m 43s
51 actionable tasks: 41 executed, 4 from cache, 6 up-to-date
Publishing build scan...
https://gradle.com/s/x2putr2ltdyb2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
