See
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/6142/display/redirect?page=changes>
Changes:
[kcweaver] [BEAM-9756] Nexmark: only use --region in Dataflow.
[github] [BEAM-8889] add gRPC suport in GCS connector (behind an
------------------------------------------
[...truncated 645.13 KB...]
copying apache_beam/typehints/typehints_test.py ->
apache-beam-2.22.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py ->
apache-beam-2.22.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py ->
apache-beam-2.22.0.dev0/apache_beam/utils
Writing apache-beam-2.22.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.22.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1"
setup.py:260: UserWarning: You are using Apache Beam with Python 2. New
releases of Apache Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/setuptools/dist.py>:476:
UserWarning: Normalizing '2.22.0.dev' to '2.22.0.dev0'
normalized_version,
running nosetests
running egg_info
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:241:
UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/__init__.py>:82:
UserWarning: You are using Apache Beam with Python 2. New releases of Apache
Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
WARNING:apache_beam.runners.interactive.interactive_environment:Interactive
Beam requires Python 3.5.3+.
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies
required for Interactive Beam PCollection visualization are not available,
please use: `pip install apache-beam[interactive]` to install necessary
dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use
Interactive Beam features when you are not in an interactive environment such
as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python
2.7 interpreter.
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/trigger_test.py>:575:
YAMLLoadWarning: calling yaml.load_all() without Loader=... is deprecated, as
the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full
details.
for spec in yaml.load_all(open(transcript_filename)):
WARNING:root:Make sure that locally built Python SDK docker image has Python
2.7 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT)
... ERROR
test_metrics_fnapi_it
(apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest)
... ERROR
======================================================================
ERROR: test_wordcount_fnapi_it
(apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py";,>
line 812, in run
test(orig)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py";,>
line 45, in __call__
return self.run(*arg, **kwarg)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py";,>
line 133, in run
self.runTest(result)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py";,>
line 151, in runTest
test(result)
File "/usr/lib/python2.7/unittest/case.py", line 393, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py";,>
line 54, in test_wordcount_fnapi_it
self._run_wordcount_it(wordcount.run, experiment='beam_fn_api')
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py";,>
line 91, in _run_wordcount_it
save_main_session=False)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py";,>
line 121, in run
result = p.run()
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py";,>
line 501, in run
self._options).run(False)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py";,>
line 514, in run
return self.runner.run_pipeline(self, self._options)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py";,>
line 75, in run_pipeline
self.result.cancel()
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py";,>
line 1656, in cancel
return self.state
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py";,>
line 1594, in state
self._update_job()
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py";,>
line 1547, in _update_job
self._job = self._runner.dataflow_client.get_job(self.job_id())
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/utils/retry.py";,>
line 236, in wrapper
return fun(*args, **kwargs)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py";,>
line 806, in get_job
response = self._client.projects_locations_jobs.Get(request)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py";,>
line 699, in Get
config, request, global_params=global_params)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/apitools/base/py/base_api.py";,>
line 729, in _RunMethod
http, http_request, **opts)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/apitools/base/py/http_wrapper.py";,>
line 346, in MakeRequest
check_response_func=check_response_func)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/apitools/base/py/http_wrapper.py";,>
line 396, in _MakeRequestNoRetry
redirections=redirections, connection_type=connection_type)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/oauth2client/transport.py";,>
line 169, in new_request
redirections, connection_type)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/oauth2client/transport.py";,>
line 169, in new_request
redirections, connection_type)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/httplib2/__init__.py";,>
line 2133, in request
cachekey,
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/httplib2/__init__.py";,>
line 1796, in _request
conn, request_uri, method, body, headers
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/httplib2/__init__.py";,>
line 1737, in _conn_request
response = conn.getresponse()
File "/usr/lib/python2.7/httplib.py", line 1152, in getresponse
response.begin()
File "/usr/lib/python2.7/httplib.py", line 463, in begin
version, status, reason = self._read_status()
File "/usr/lib/python2.7/httplib.py", line 419, in _read_status
line = self.fp.readline(_MAXLINE + 1)
File "/usr/lib/python2.7/socket.py", line 480, in readline
data = self._sock.recv(self._rbufsize)
File "/usr/lib/python2.7/ssl.py", line 756, in recv
return self.read(buflen)
File "/usr/lib/python2.7/ssl.py", line 643, in read
v = self._sslobj.read(len)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py";,>
line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_wordcount_fnapi_it
(apache_beam.examples.wordcount_it_test.WordCountIT)'
======================================================================
ERROR: test_metrics_fnapi_it
(apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py";,>
line 812, in run
test(orig)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py";,>
line 45, in __call__
return self.run(*arg, **kwarg)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py";,>
line 133, in run
self.runTest(result)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py";,>
line 151, in runTest
test(result)
File "/usr/lib/python2.7/unittest/case.py", line 393, in __call__
return self.run(*args, **kwds)
File "/usr/lib/python2.7/unittest/case.py", line 329, in run
testMethod()
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py";,>
line 57, in test_metrics_fnapi_it
result = self.run_pipeline(experiment='beam_fn_api')
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py";,>
line 38, in run_pipeline
test_pipeline = TestPipeline(is_integration_test=True)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/testing/test_pipeline.py";,>
line 103, in __init__
self._parse_test_option_args(argv) + additional_pipeline_args)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/testing/test_pipeline.py";,>
line 135, in _parse_test_option_args
parser = argparse.ArgumentParser()
File "/usr/lib/python2.7/argparse.py", line 1598, in __init__
self._optionals = add_group(_('optional arguments'))
File "/usr/lib/python2.7/gettext.py", line 584, in gettext
return dgettext(_current_domain, message)
File "/usr/lib/python2.7/gettext.py", line 548, in dgettext
codeset=_localecodesets.get(domain))
File "/usr/lib/python2.7/gettext.py", line 483, in translation
mofiles = find(domain, localedir, languages, all=1)
File "/usr/lib/python2.7/gettext.py", line 431, in find
val = os.environ.get(envar)
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/lib/python2.7/UserDict.py";,>
line 92, in get
if key not in self:
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/lib/python2.7/UserDict.py";,>
line 103, in __contains__
def __contains__(self, key):
File
"<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py";,>
line 276, in signalhandler
raise TimedOutException()
TimedOutException: 'test_metrics_fnapi_it
(apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest)'
----------------------------------------------------------------------
XML: nosetests-beam_python2.7_sdk.xml
----------------------------------------------------------------------
XML:
<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 2 tests in 904.883s
FAILED (errors=2)
cleanup_container
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk:20200415-183447
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk@sha256:debcad98387e4d473cf8fee9ff5491eaea41756b531cf6a05b004a80f94ebd37
Deleted: sha256:dbff00049470dd70790443a700977ae6563117314f938f69f74b97cb323e0fd4
Deleted: sha256:395d85701d697668fdcdca2eda511ed37fe58b2c88ccd98ad93d13f3fb4c6e60
Deleted: sha256:6a1721073c922ed368ea1a6eb5ea8cd56a95e53d1265fcae43854d22f86559d4
Deleted: sha256:4beeae84b21b037815b9b8d756a1f7244e21438983b6a4537ff7fb1b6cc11ac4
Deleted: sha256:60e13940bf688ce7f6cc788838cdb0beee9869bcf7995e1e3ae5edb641092358
Deleted: sha256:e835b72de317bab4f23539469fafc57d44d1676062ce60a5f7a1a536a1621200
Deleted: sha256:87434bfeaebc0afd247ca007d91b34086ac1e7170a611b6b1c113efe4f33996e
Deleted: sha256:50e9e26ad72ea5caf104d6863ebbf1a3347c53c0d5d017deada2f6d4d75ebbc0
Deleted: sha256:d8c525c2be314d1a06366b29e1123dc1511efecab0ac73b6bfac5b658f296644
Deleted: sha256:5395aa19ab3d69d22badeee1cd02fa36387e75a12e92ef8cebf7343e770bb5ff
Deleted: sha256:09bcb3dcea0aa493861481b84637270e9d92230813afc5cbc789a5ac971e7160
Deleted: sha256:8c96b6f2d42645fbc71fa223b2c075a7f6b420b378092e9accd3b13884cf93fe
Deleted: sha256:7d40475b280256196cbbe717a5c07ac9bbd75b156c2cfbbbaf8f4326e78858d8
Deleted: sha256:80728cba7ab81e1c29be9763394cb22faac145862345cdad36f75ca7edc4f630
Deleted: sha256:f402c8d4fec6734c9ac5243b2d7dcb5c0866c37328bd722fc087f89fd6800331
Deleted: sha256:50d87736ebb8148623b8cb54e2a8521f1c62b0178ca325c171525069f6b6f0bf
Deleted: sha256:961163330c2b34434a327b88233416ba63b62a5c73bacfc1bf69a0ce821e8c2e
Deleted: sha256:4d3dfc5f3e959594f4873435fae7092c5b115e9dbc30692cc8a5155a14f1f808
Deleted: sha256:f44c0e15b3f6b8e45862bac353e834d7d91bedae06577549a96fba1ab0497c8d
Deleted: sha256:a0d5fab528c48b37c75f33ce7cc3ff842b67bc5adf1742c2c6a87407f30e3ecc
Deleted: sha256:7f1413d650c11f7b9f4218297afa840d4e4df85339b04545bd68ad40fae5531b
Deleted: sha256:38d4d6ede50b6a7851720fcd76d881ae25c792d27c21c66d961f9bb2e9b88dd7
Deleted: sha256:0785797a7103b43c4695aef3e8789167fbf1a329ba2f4742c8bd2fa6cb0f7ac9
Deleted: sha256:31cef71159041b1202bef1f8b9b21a8c1ec6906aa71ade56ea00de955756c058
Deleted: sha256:cb4b51b127a1da16cd373f64bbfdb172b090823b076efdeaedfea5dd4bd679db
Deleted: sha256:e8b918cc0ca8767c31c5286425f0b03c002aa5aff532c629d447415e7f761945
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk@sha256:debcad98387e4d473cf8fee9ff5491eaea41756b531cf6a05b004a80f94ebd37
Associated tags:
- 20200415-183447
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk:20200415-183447
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk:20200415-183447].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk@sha256:debcad98387e4d473cf8fee9ff5491eaea41756b531cf6a05b004a80f94ebd37].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
