See
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/6337/display/redirect?page=changes>
Changes:
[pabloem] [BEAM-9967] Adding support for BQ labels on Query/Export jobs. (Roll
[github] [BEAM-9430] Fixes the bounds of initial watermark set to estimators
[kcweaver] [BEAM-9993] Add option defaults for Flink Python tests.
[github] [BEAM-9941] Added a test of a GBK followed by a Flatten with an unknown
[github] Clarify pubsub IO comment about timestamps (#11672)
[iemejia] [BEAM-9833] Add yamllint config
[iemejia] [BEAM-9833] Fix .asf.yaml issues, sort labels and disable rebase
button
[github] Clarifies an error message in Katas to explain what is actually wrong.
[github] Merge pull request #11210 from [BEAM-8949] SpannerIO integration tests
[github] [BEAM-9876] Migrate the Beam website from Jekyll to Hugo to enable
[github] Update the range for pyarrow to qualify pyarrow 0.17.x (#11699)
------------------------------------------
[...truncated 733.04 KB...]
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1"
setup.py:262: UserWarning: You are using Apache Beam with Python 2. New
releases of Apache Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/setuptools/dist.py>:476:
UserWarning: Normalizing '2.22.0.dev' to '2.22.0.dev0'
normalized_version,
running nosetests
running egg_info
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:241:
UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/__init__.py>:82:
UserWarning: You are using Apache Beam with Python 2. New releases of Apache
Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
WARNING:apache_beam.runners.interactive.interactive_environment:Interactive
Beam requires Python 3.5.3+.
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies
required for Interactive Beam PCollection visualization are not available,
please use: `pip install apache-beam[interactive]` to install necessary
dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use
Interactive Beam features when you are not in an interactive environment such
as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python
2.7 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python
2.7 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT)
... ERROR
test_metrics_fnapi_it
(apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest)
... ERROR
======================================================================
ERROR: test_wordcount_fnapi_it
(apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py",>
line 812, in run
test(orig)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",>
line 45, in __call__
return self.run(*arg, **kwarg)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",>
line 133, in run
self.runTest(result)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",>
line 151, in runTest
test(result)
File "/usr/lib/python2.7/unittest/case.py", line 393, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",>
line 54, in test_wordcount_fnapi_it
self._run_wordcount_it(wordcount.run, experiment='beam_fn_api')
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",>
line 91, in _run_wordcount_it
save_main_session=False)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",>
line 121, in run
result = p.run()
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",>
line 512, in run
self._options).run(False)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",>
line 525, in run
return self.runner.run_pipeline(self, self._options)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",>
line 75, in run_pipeline
self.result.cancel()
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",>
line 1665, in cancel
return self.state
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",>
line 1603, in state
self._update_job()
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",>
line 1556, in _update_job
self._job = self._runner.dataflow_client.get_job(self.job_id())
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/utils/retry.py",>
line 236, in wrapper
return fun(*args, **kwargs)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",>
line 807, in get_job
response = self._client.projects_locations_jobs.Get(request)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py",>
line 699, in Get
config, request, global_params=global_params)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/apitools/base/py/base_api.py",>
line 729, in _RunMethod
http, http_request, **opts)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/apitools/base/py/http_wrapper.py",>
line 346, in MakeRequest
check_response_func=check_response_func)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/apitools/base/py/http_wrapper.py",>
line 396, in _MakeRequestNoRetry
redirections=redirections, connection_type=connection_type)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/oauth2client/transport.py",>
line 169, in new_request
redirections, connection_type)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/oauth2client/transport.py",>
line 169, in new_request
redirections, connection_type)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/httplib2/__init__.py",>
line 2175, in request
cachekey,
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/httplib2/__init__.py",>
line 1835, in _request
conn, request_uri, method, body, headers
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/httplib2/__init__.py",>
line 1776, in _conn_request
response = conn.getresponse()
File "/usr/lib/python2.7/httplib.py", line 1165, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 463, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 419, in _read_status
line = self.fp.readline(_MAXLINE + 1)
File "/usr/lib/python2.7/socket.py", line 480, in readline
data = self._sock.recv(self._rbufsize)
File "/usr/lib/python2.7/ssl.py", line 756, in recv
return self.read(buflen)
File "/usr/lib/python2.7/ssl.py", line 643, in read
v = self._sslobj.read(len)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py",>
line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_wordcount_fnapi_it
(apache_beam.examples.wordcount_it_test.WordCountIT)'
======================================================================
ERROR: test_metrics_fnapi_it
(apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py",>
line 812, in run
test(orig)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",>
line 45, in __call__
return self.run(*arg, **kwarg)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",>
line 133, in run
self.runTest(result)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",>
line 151, in runTest
test(result)
File "/usr/lib/python2.7/unittest/case.py", line 393, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py",>
line 57, in test_metrics_fnapi_it
result = self.run_pipeline(experiment='beam_fn_api')
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py",>
line 45, in run_pipeline
return dataflow_exercise_metrics_pipeline.apply_and_run(p)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline.py",>
line 167, in apply_and_run
| 'm_out' >> beam.FlatMap(
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/ptransform.py",>
line 562, in __ror__
result = p.apply(self, pvalueish, label)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",>
line 654, in apply
pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",>
line 153, in apply
return super(DataflowRunner, self).apply(transform, input, options)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py",>
line 198, in apply
return m(transform, input, options)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py",>
line 228, in apply_PTransform
return transform.expand(input)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/core.py",>
line 2715, in expand
| Map(coder.decode).with_output_types(self.get_output_type()))
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pvalue.py",>
line 140, in __or__
return self.pipeline.apply(ptransform, self)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",>
line 654, in apply
pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",>
line 153, in apply
return super(DataflowRunner, self).apply(transform, input, options)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py",>
line 198, in apply
return m(transform, input, options)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py",>
line 228, in apply_PTransform
return transform.expand(input)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/core.py",>
line 2706, in expand
return pcoll | Reshuffle()
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pvalue.py",>
line 140, in __or__
return self.pipeline.apply(ptransform, self)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",>
line 654, in apply
pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",>
line 153, in apply
return super(DataflowRunner, self).apply(transform, input, options)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py",>
line 198, in apply
return m(transform, input, options)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py",>
line 228, in apply_PTransform
return transform.expand(input)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/util.py",>
line 715, in expand
| 'RemoveRandomKeys' >> Map(lambda t: t[1]).with_input_types(
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pvalue.py",>
line 140, in __or__
return self.pipeline.apply(ptransform, self)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",>
line 668, in apply
self._infer_result_type(transform, inputs, result)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",>
line 724, in _infer_result_type
type_options = self._options.view_as(TypeOptions)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/options/pipeline_options.py",>
line 355, in view_as
view = cls(self._flags)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/options/pipeline_options.py",>
line 202, in __init__
cls._add_argparse_args(parser) # type: ignore
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/options/pipeline_options.py",>
line 466, in _add_argparse_args
help='Enable type checking at pipeline execution '
File "/usr/lib/python2.7/argparse.py", line 1304, in add_argument
self._get_formatter()._format_args(action, None)
File "/usr/lib/python2.7/argparse.py", line 574, in _format_args
get_metavar = self._metavar_formatter(action, default_metavar)
File "/usr/lib/python2.7/argparse.py", line 566, in _metavar_formatter
def format(tuple_size):
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py",>
line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_metrics_fnapi_it
(apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest)'
----------------------------------------------------------------------
XML: nosetests-beam_python2.7_sdk.xml
----------------------------------------------------------------------
XML:
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 2 tests in 904.721s
FAILED (errors=2)
cleanup_container
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk:20200515-000205
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk@sha256:37413ae4ef9a5802d11f5f44e8035f7dc897d6df169e0167efa6f02ec0fa2dde
Deleted: sha256:94dd426057342a0ce81799570a0b0395008d7268232bb4b2c4c72d97289e6741
Deleted: sha256:bc7ea336440e60cc8c3d650e434459ba6546d99946dae0fcdda58a6270f55cc3
Deleted: sha256:95b4f5874ad2ee2f1222ac5db714f6992c4afc8b257cb18195d2558406e5285a
Deleted: sha256:0ffc5d8091508828b7ba50fb2877ba09376a343fd3e9cb50099972e2b2579b7b
Deleted: sha256:22ba5e3217345586aa8ac79bb85f9322c4118a2241857ab479acd34bbf5ebbd3
Deleted: sha256:98d24d2b1cb2bdb56c00bf838893cb8b0d2fac479ae67b006630bb923eaf79aa
Deleted: sha256:b56f22939f492ab5ac8bb7709ece77055e487df45f4de109a9d430344dfc5c28
Deleted: sha256:7008f2d3e783f1c998a54bba13e83c3b0feb6496898d807a3da6f7f057ebc5a0
Deleted: sha256:c3b97702d9280cedf7a50a1d46a55d91fe8489c9fe5e810a07ca806f65cba910
Deleted: sha256:3c48159e3d61926944441330baab5683e8ceba1562878c77dc2a9325a46e7c7c
Deleted: sha256:250aa078776d811d478d4ea3b10b259dc81655ac722e35bc369219f8803bb70c
Deleted: sha256:f70fb797b2a0c5d3c5e3740492fc1b102c5ced0ab6b913969ef46cf29e07f4ee
Deleted: sha256:29dc74d325a8efe56167070e32ee7fbe3ea110d44084efd6199bb3bb73f074b7
Deleted: sha256:1cf7945150f51e3b63cf768fd0a83b4586fde5d681e5692135dbaf1e59a8d8e5
Deleted: sha256:277a2a466e04a7490f96be3664e0d43f3b7a5408df2c23e5f32826c7e2e7358d
Deleted: sha256:549d84d4ef5d62292f5399ebcb54c9896fe8198e0c017038b6b91cac51cae7dd
Deleted: sha256:afcb51975aec2aec5f5f6ec045adcde9c5d8a9904a55e327ec750afea61cf7fe
Deleted: sha256:0ce42709e0cd62329570aa1f27fdc164d64c71200b0e8a4fbf241c6c03d9faae
Deleted: sha256:15e45f3d00c3923593a27258e40e749a513a8addbe2e23a8f5cc8d5c97f32167
Deleted: sha256:bba4ead16415580e93effb53055634f46c65902ea622b7a57fa630b56deada8f
Deleted: sha256:36353ef7ea9d52c8c0b0c4ff68777318afa1eecf5ae1fc9ba2ce4e3fd6a42446
Deleted: sha256:e047dcb846faf28f1605f4effe10f984680789d6a136cc011706043a4bd3d397
Deleted: sha256:b5f8e6ff3b7e71a85a0534743f38638b79179414d92972594f973bce3898192f
Deleted: sha256:ccfd69969bb9b15cb0c455e78b2277d3a32a6a850b909fc2936043c881ba88ad
Deleted: sha256:6411166d1f76c97ddda5e4ebc17c241af0b9e376397be2155cf963e46f4834a5
Deleted: sha256:f4ebadd04fffe1becdd19d6e1b59f9805f589fe6bc8f0e5ed9e76c6d689abab6
Deleted: sha256:e09f78da55859274d2282943daee86f46a1f9050a0cd480e3f7267df6eda80d1
Deleted: sha256:379f45f0c189c547ecb9d2079ce6921ed8adc0fa4bd1010124f996cc60de50da
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk@sha256:37413ae4ef9a5802d11f5f44e8035f7dc897d6df169e0167efa6f02ec0fa2dde
Associated tags:
- 20200515-000205
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk:20200515-000205
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk:20200515-000205].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk@sha256:37413ae4ef9a5802d11f5f44e8035f7dc897d6df169e0167efa6f02ec0fa2dde].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]