See <https://builds.apache.org/job/beam_PostCommit_Py_ValCont/6118/display/redirect?page=changes>
Changes: [chamikara] Updates Dataflow stateful DoFn setup to support external transforms ------------------------------------------ [...truncated 644.85 KB...] 'You are using Apache Beam with Python 2. ' <https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/setuptools/dist.py>:476: UserWarning: Normalizing '2.22.0.dev' to '2.22.0.dev0' normalized_version, running nosetests running egg_info <https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:241: UserWarning: Installing grpcio-tools is recommended for development. warnings.warn('Installing grpcio-tools is recommended for development.') INFO:gen_protos:Skipping proto regeneration: all files up to date writing requirements to apache_beam.egg-info/requires.txt writing apache_beam.egg-info/PKG-INFO writing top-level names to apache_beam.egg-info/top_level.txt writing dependency_links to apache_beam.egg-info/dependency_links.txt writing entry points to apache_beam.egg-info/entry_points.txt reading manifest file 'apache_beam.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching 'README.md' warning: no files found matching 'NOTICE' warning: no files found matching 'LICENSE' writing manifest file 'apache_beam.egg-info/SOURCES.txt' <https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/__init__.py>:82: UserWarning: You are using Apache Beam with Python 2. New releases of Apache Beam will soon support Python 3 only. 'You are using Apache Beam with Python 2. ' WARNING:root:python-snappy is not installed; some tests will be skipped. WARNING:root:Tensorflow is not installed, so skipping some tests. WARNING:apache_beam.runners.interactive.interactive_environment:Interactive Beam requires Python 3.5.3+. WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features. WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal. WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter. <https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/trigger_test.py>:575: YAMLLoadWarning: calling yaml.load_all() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. for spec in yaml.load_all(open(transcript_filename)): test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ERROR ====================================================================== ERROR: test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ---------------------------------------------------------------------- Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py";,> line 812, in run test(orig) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py";,> line 45, in __call__ return self.run(*arg, **kwarg) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py";,> line 133, in run self.runTest(result) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py";,> line 151, in runTest test(result) File "/usr/lib/python2.7/unittest/case.py", line 393, in __call__ return self.run(*args, **kwds) File "/usr/lib/python2.7/unittest/case.py", line 329, in run testMethod() File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py";,> line 54, in test_wordcount_fnapi_it self._run_wordcount_it(wordcount.run, experiment='beam_fn_api') File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py";,> line 91, in _run_wordcount_it save_main_session=False) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py";,> line 121, in run result = p.run() File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py";,> line 501, in run self._options).run(False) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py";,> line 514, in run return self.runner.run_pipeline(self, self._options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py";,> line 75, in run_pipeline self.result.cancel() File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py";,> line 1656, in cancel return self.state File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py";,> line 1594, in state self._update_job() File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py";,> line 1547, in _update_job self._job = self._runner.dataflow_client.get_job(self.job_id()) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/utils/retry.py";,> line 236, in wrapper return fun(*args, **kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py";,> line 806, in get_job response = self._client.projects_locations_jobs.Get(request) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py";,> line 699, in Get config, request, global_params=global_params) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/apitools/base/py/base_api.py";,> line 729, in _RunMethod http, http_request, **opts) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/apitools/base/py/http_wrapper.py";,> line 346, in MakeRequest check_response_func=check_response_func) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/apitools/base/py/http_wrapper.py";,> line 396, in _MakeRequestNoRetry redirections=redirections, connection_type=connection_type) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/oauth2client/transport.py";,> line 169, in new_request redirections, connection_type) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/oauth2client/transport.py";,> line 169, in new_request redirections, connection_type) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/httplib2/__init__.py";,> line 2133, in request cachekey, File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/httplib2/__init__.py";,> line 1796, in _request conn, request_uri, method, body, headers File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/httplib2/__init__.py";,> line 1737, in _conn_request response = conn.getresponse() File "/usr/lib/python2.7/httplib.py", line 1152, in getresponse response.begin() File "/usr/lib/python2.7/httplib.py", line 463, in begin version, status, reason = self._read_status() File "/usr/lib/python2.7/httplib.py", line 419, in _read_status line = self.fp.readline(_MAXLINE + 1) File "/usr/lib/python2.7/socket.py", line 480, in readline data = self._sock.recv(self._rbufsize) File "/usr/lib/python2.7/ssl.py", line 756, in recv return self.read(buflen) File "/usr/lib/python2.7/ssl.py", line 643, in read v = self._sslobj.read(len) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py";,> line 276, in signalhandler raise TimedOutException() TimedOutException: 'test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT)' ====================================================================== ERROR: test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py";,> line 812, in run test(orig) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py";,> line 45, in __call__ return self.run(*arg, **kwarg) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py";,> line 133, in run self.runTest(result) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py";,> line 151, in runTest test(result) File "/usr/lib/python2.7/unittest/case.py", line 393, in __call__ return self.run(*args, **kwds) File "/usr/lib/python2.7/unittest/case.py", line 329, in run testMethod() File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py";,> line 57, in test_metrics_fnapi_it result = self.run_pipeline(experiment='beam_fn_api') File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py";,> line 45, in run_pipeline return dataflow_exercise_metrics_pipeline.apply_and_run(p) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline.py";,> line 167, in apply_and_run | 'm_out' >> beam.FlatMap( File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/ptransform.py";,> line 562, in __ror__ result = p.apply(self, pvalueish, label) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py";,> line 634, in apply pvalueish_result = self.runner.apply(transform, pvalueish, self._options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py";,> line 141, in apply return super(DataflowRunner, self).apply(transform, input, options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 198, in apply return m(transform, input, options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 228, in apply_PTransform return transform.expand(input) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/core.py";,> line 2715, in expand | Map(coder.decode).with_output_types(self.get_output_type())) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pvalue.py";,> line 140, in __or__ return self.pipeline.apply(ptransform, self) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py";,> line 634, in apply pvalueish_result = self.runner.apply(transform, pvalueish, self._options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py";,> line 141, in apply return super(DataflowRunner, self).apply(transform, input, options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 198, in apply return m(transform, input, options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 228, in apply_PTransform return transform.expand(input) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/core.py";,> line 2706, in expand return pcoll | Reshuffle() File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pvalue.py";,> line 140, in __or__ return self.pipeline.apply(ptransform, self) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py";,> line 634, in apply pvalueish_result = self.runner.apply(transform, pvalueish, self._options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py";,> line 141, in apply return super(DataflowRunner, self).apply(transform, input, options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 198, in apply return m(transform, input, options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 228, in apply_PTransform return transform.expand(input) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/util.py";,> line 716, in expand | 'RemoveRandomKeys' >> Map(lambda t: t[1]).with_input_types( File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pvalue.py";,> line 140, in __or__ return self.pipeline.apply(ptransform, self) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py";,> line 634, in apply pvalueish_result = self.runner.apply(transform, pvalueish, self._options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py";,> line 141, in apply return super(DataflowRunner, self).apply(transform, input, options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 198, in apply return m(transform, input, options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py";,> line 228, in apply_PTransform return transform.expand(input) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/util.py";,> line 687, in expand | FlatMap(restore_timestamps).with_output_types(Any)) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pvalue.py";,> line 140, in __or__ return self.pipeline.apply(ptransform, self) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py";,> line 634, in apply pvalueish_result = self.runner.apply(transform, pvalueish, self._options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py";,> line 140, in apply self._maybe_add_unified_worker_missing_options(options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py";,> line 580, in _maybe_add_unified_worker_missing_options if apiclient._use_unified_worker(options): File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py";,> line 1017, in _use_unified_worker debug_options = pipeline_options.view_as(DebugOptions) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/options/pipeline_options.py";,> line 355, in view_as view = cls(self._flags) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/options/pipeline_options.py";,> line 197, in __init__ parser = _BeamArgumentParser() File "/usr/lib/python2.7/argparse.py", line 1598, in __init__ self._optionals = add_group(_('optional arguments')) File "/usr/lib/python2.7/gettext.py", line 584, in gettext return dgettext(_current_domain, message) File "/usr/lib/python2.7/gettext.py", line 548, in dgettext codeset=_localecodesets.get(domain)) File "/usr/lib/python2.7/gettext.py", line 483, in translation mofiles = find(domain, localedir, languages, all=1) File "/usr/lib/python2.7/gettext.py", line 445, in find result = [] File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py";,> line 276, in signalhandler raise TimedOutException() TimedOutException: 'test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest)' ---------------------------------------------------------------------- XML: nosetests-beam_python2.7_sdk.xml ---------------------------------------------------------------------- XML: <https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml> ---------------------------------------------------------------------- Ran 2 tests in 904.677s FAILED (errors=2) cleanup_container Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk:20200413-173008 Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk@sha256:6ff6d1f60d94dbbe115e87ac3e921a83a3f2fc6248d605d8f6b95d9958a2d223 Deleted: sha256:db2268e6ce9fd3b2ea39bf23bd5d3b8d6262cbe86cc5a58ab86fc481cb1dc6d3 Deleted: sha256:5d03709e97d1a5c9256c3564364bf674c98b68d8edd7a2da670c850da00c6ab7 Deleted: sha256:c57816206bf3e29dd227fb5f0c6cdf588742c8880ffdb25fba2a282af03e9f05 Deleted: sha256:e535278b1efd2ca2a608266ab8ac4d92274c979910683fc9fb5ed2ca00bedf6e Deleted: sha256:c91a36a6a854930fc4291566b1bc1a879aca8b9e8afea329bda6a7a0048c2aae Deleted: sha256:655338e2693fff537162764aec50ce7858e0c4670453323ca6c949db7fc4dc19 Deleted: sha256:e4e2d6fa52b0313e6d1ce1586e6d6e5b39aa77fc8a8ffb9ba47a38176ebb72c3 Deleted: sha256:c687ba5d6f640c1baacf6413bbafe93d2598bfd29d0176bf603ed5b8b010c904 Deleted: sha256:1a20dab0369bf5927bc32bce2e2df7f506f9d17d89cb4630d1f67eb1eb27668c Deleted: sha256:23496a4715153599e0f30468855b18a6d6c8f8e2cb8cd4a4b4582fb91c80eff6 Deleted: sha256:6f0e9583f546fc3f447503cb4a26ea0b8e19bd14bb9de58e0d9dc2f7d5c21637 Deleted: sha256:28afdb97a43cae4cd5f126659aeddf1ad712007488cd76d0919735d8ad449d23 Deleted: sha256:acf0f80f262afdfe07f15053345a40355b2e37bc40bc3244d5ddadd8851cee5a Deleted: sha256:92229c4883988c15877086ca77f6e8e68747c9444cb4ce5b8f87600644d403e8 Deleted: sha256:f608df427ead00cf7a9bef69fd15af4538452ad46ebb5fb724e263ad00cdf0ca Deleted: sha256:58160ff51e576248ed2e1aea291539973f74517287a458fedbec90114b664099 Deleted: sha256:14d7f58039e4d9ad33b0bc31658b1a92bed00df185c85100be3080d9fd7311bb Deleted: sha256:eb811a4dba852aa6d2c19e6e44702305065711705b6481f2211e25f2bdc4eda0 Deleted: sha256:2bad72b9e9ec001c7ab0e2c9276255a211f059bd5a7b6d95ea823016bcd8894b Deleted: sha256:c53134cded96db48005c77a3777868dc69dae649287377f80add4c6f47d74dca Deleted: sha256:b864ca49ee7ac77867639d8574b5954d33f81da0e439346a4d3a5e9cde3ed853 Deleted: sha256:33eac4ded2b88234057e1268a3ffec3b2b52c032ef53eee4c722d64a44f2c823 Deleted: sha256:15eac338ebd7e050e5a9bfaa03e68df24fe16b2b54e767d90a7cc5616284c844 Deleted: sha256:1727fdc1fcb1a25f411e3a3854780bda76fbcdc500cf7a910260998585bb189f Deleted: sha256:cbe2a22802a5e05ff117d71c7e9b42b6d9d5e7091a05a3be26c385b5e7874592 Deleted: sha256:0384477a91c274cbf101ae9231a1edae64bff552b89f8b89f236cbbb574dc2c3 Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk@sha256:6ff6d1f60d94dbbe115e87ac3e921a83a3f2fc6248d605d8f6b95d9958a2d223 Associated tags: - 20200413-173008 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk:20200413-173008 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk:20200413-173008]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk@sha256:6ff6d1f60d94dbbe115e87ac3e921a83a3f2fc6248d605d8f6b95d9958a2d223]. Removed the container Build step 'Execute shell' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
