See
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/6868/display/redirect>
Changes:
------------------------------------------
[...truncated 911.64 KB...]
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py ->
apache-beam-2.26.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py ->
apache-beam-2.26.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py ->
apache-beam-2.26.0.dev0/apache_beam/utils
Writing apache-beam-2.26.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.26.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:452:
UserWarning: Normalizing '2.26.0.dev' to '2.26.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243:
UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50:
UserWarning: VCF IO will support Python 3 after migration to Nucleus, see:
BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies
required for Interactive Beam PCollection visualization are not available,
please use: `pip install apache-beam[interactive]` to install necessary
dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use
Interactive Beam features when you are not in an interactive environment such
as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python
3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python
3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python
3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT)
... ok
test_wordcount_it_with_prebuilt_sdk_container
(apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it
(apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest)
... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container
(apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py";,>
line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py";,>
line 107, in _run_wordcount_it
save_main_session=False,
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py";,>
line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py";,>
line 568, in __exit__
self.result = self.run()
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py";,>
line 518, in run
allow_proto_holders=True).run(False)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py";,>
line 547, in run
return self.runner.run_pipeline(self, self._options)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py";,>
line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py";,>
line 486, in run_pipeline
environments.DockerEnvironment.from_options(options))
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py";,>
line 259, in from_options
options)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py";,>
line 141, in build_container_image
return builder.build()
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py";,>
line 90, in build
self.invoke_docker_build_and_push(container_image_name)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py";,>
line 154, in invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 423, in run
with Popen(*popenargs, **kwargs) as process:
TypeError: __init__() got an unexpected keyword argument 'capture_output'
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML:
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 555.408s
FAILED (errors=1)
cleanup_container
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20200924-120027
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:5c175eb5d17c251cf1e607fcbe2cc731b21fab9563ae588b314b8325fd9ad32c
Deleted: sha256:d6b0b3198e2e5bc1dd3628813eeaa272264e425cd276c8ac3a719e48d9d03e91
Deleted: sha256:65a8110049c883be53c11659b6322f28ff5a0eff7c3523be1a60cae8d6976e83
Deleted: sha256:f0f87fcb9da59542b059ec59a0bc436a26cc127d8377c580157fd071df9ed4b7
Deleted: sha256:21f4f741e7bc63d822b6b2ff3ec70b425f1786bc924b312c908fa06a7df86462
Deleted: sha256:431f91fa0b557f1dc0da2b900cce84e5b7a5cf877474894fde563dffef8b374b
Deleted: sha256:f12362cd02cd6c13216cccfd5299e953892de7fed73da08ee62d15c5c65076f4
Deleted: sha256:01c6ed6ce76a96c4d5ca8cbb8ea6d0f5a88b5370f161718aa339cd5febb873e2
Deleted: sha256:81b5ee2855d184cf80f47b753b9722935877aa595f1adc17c32fc5ed252c217a
Deleted: sha256:8d7c0847eaa36d52c5c38f3b160ce073344e5a1f3e41df3f07969434b0c90217
Deleted: sha256:a80ef378ec6053f58b5b0c4442b8efa4737950c8e4e4a18101d728cec7101326
Deleted: sha256:ab1a28fe275c3dc520a34be4a321b4b0a14eaeb0abcee81f1cbe867fde440656
Deleted: sha256:5dfa5e02c346b53b482c686b09358d7902e1cac16119c5e02001d0e357a7e534
Deleted: sha256:5e7746eddb352cea33575918e26b60d33afd008faaef36e9e83db42c4e4adbf6
Deleted: sha256:4f079515f3edeb0c09e1bd281ae426873b50d849cc67e09f40765167b3da1e56
Deleted: sha256:9652d3440910d7618e3d0ea7dbffbb8fc9122d7d778efdf10759278fff942e3b
Deleted: sha256:b4e2121880cda00f4e8028c53f2df32994cabceb893316d07b558f6a4cef72f0
Deleted: sha256:e948c90c19935628b82a2f262b2801bb17271121e70f0628f7bd2cb88f2cb311
Deleted: sha256:8940657ab46e80bf4f3a5f9efb995d371438f64990e7fce668456fa355683476
Deleted: sha256:4b7cdb57e430c92ce55e31dd98c2fa0a72bf22c598ae7bf675831932a4b0081f
Deleted: sha256:d814f56f0364b3fb973f321f521b44053f723241846e097e230916984ebf1f6e
Deleted: sha256:568d1da4c7f2092b6b3580cd1016c45e7ed144fd0c8f031bf7854f259875bbeb
Deleted: sha256:5fb08a3cc7e7dd0c1707286a97593d64e549b5f4805380967783511f81de926f
Deleted: sha256:bf54b17451a31a4b8167f200040695be8215e34e178352149c1323409e90ed5c
Deleted: sha256:b1e63a47f62deb6843546fc3e12ded29da87a162760ef59ddf79f2cc20658aa6
Deleted: sha256:41e14972c818d8fe97c9dea626c5d79d47497605f44892c32b0b94275e365778
Deleted: sha256:82775a1a44464a2546b4010927a43926f98fb3a1b0a8046e009c7abd7838b770
Deleted: sha256:a3ddb653d75043736859c44ca0beba5030a37d7f384bd3d1023a2630cf1f6289
Deleted: sha256:cdf45b12d0b3348942727cc6b196e7c93e8c66a77319ada37b9c81184bcc554d
Deleted: sha256:db5d943005f787ce10753f93d5dfbc9a6fa628c5790cdfa94e64c7839b473210
Deleted: sha256:84047311d9b06604df3d9b6f9e163afb54f596f807e8a4f503153b41ec7a38c1
Deleted: sha256:39c6d3e590efbae3fb0a776a06984cc60ebaa6330eedd8ec12ddb56e37946bcb
Deleted: sha256:085f2d8614990b8b9ce907ae9c10beb78887b2365382c120823a2d6bbdd89cd6
Deleted: sha256:9d627dfa590b397e0ba97de86615093845980ac578ec27194455d426ce80989a
Deleted: sha256:d19c66e274726b85f24921d31422695d4f8544d5600a107cff5f13e6cfa4447a
Deleted: sha256:b9253d72ca78ea774949a06a3c028bf896bf7383b7a7b4054ff98565931b9b17
Deleted: sha256:f714f0414df1e22fd7b1f504ca52b0bf1c56ccd302e91605269124ae604b2182
Deleted: sha256:ecf5d11dea9153c67e7bd69c214920630d6cd9d0f4093bdc2aee056a5d67ebfc
Deleted: sha256:0184b7537712f0b74e1c7f5ca554cbf2a94be512e926c66347834e56c610a086
Deleted: sha256:e0fb15226b080fca908ea5679b0bc609012ca6a3136dea1948a9338870282771
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:5c175eb5d17c251cf1e607fcbe2cc731b21fab9563ae588b314b8325fd9ad32c
Associated tags:
- 20200924-120027
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20200924-120027
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20200924-120027].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:5c175eb5d17c251cf1e607fcbe2cc731b21fab9563ae588b314b8325fd9ad32c].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
