See <https://builds.apache.org/job/beam_PostCommit_Py_ValCont/6193/display/redirect?page=changes>
Changes: [github] [BEAM-9496] Add to_dataframe and to_pcollection APIs. (#11264) ------------------------------------------ [...truncated 650.34 KB...] <https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:241: UserWarning: Installing grpcio-tools is recommended for development. warnings.warn('Installing grpcio-tools is recommended for development.') INFO:gen_protos:Skipping proto regeneration: all files up to date writing requirements to apache_beam.egg-info/requires.txt writing apache_beam.egg-info/PKG-INFO writing top-level names to apache_beam.egg-info/top_level.txt writing dependency_links to apache_beam.egg-info/dependency_links.txt writing entry points to apache_beam.egg-info/entry_points.txt reading manifest file 'apache_beam.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching 'README.md' warning: no files found matching 'NOTICE' warning: no files found matching 'LICENSE' writing manifest file 'apache_beam.egg-info/SOURCES.txt' <https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/__init__.py>:82: UserWarning: You are using Apache Beam with Python 2. New releases of Apache Beam will soon support Python 3 only. 'You are using Apache Beam with Python 2. ' WARNING:root:python-snappy is not installed; some tests will be skipped. WARNING:root:Tensorflow is not installed, so skipping some tests. WARNING:apache_beam.runners.interactive.interactive_environment:Interactive Beam requires Python 3.5.3+. WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features. WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal. WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter. <https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/trigger_test.py>:575: YAMLLoadWarning: calling yaml.load_all() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. for spec in yaml.load_all(open(transcript_filename)): WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter. test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ERROR ====================================================================== ERROR: test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ---------------------------------------------------------------------- Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py",> line 812, in run test(orig) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",> line 45, in __call__ return self.run(*arg, **kwarg) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",> line 133, in run self.runTest(result) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",> line 151, in runTest test(result) File "/usr/lib/python2.7/unittest/case.py", line 393, in __call__ return self.run(*args, **kwds) File "/usr/lib/python2.7/unittest/case.py", line 329, in run testMethod() File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 54, in test_wordcount_fnapi_it self._run_wordcount_it(wordcount.run, experiment='beam_fn_api') File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 91, in _run_wordcount_it save_main_session=False) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 121, in run result = p.run() File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 501, in run self._options).run(False) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 514, in run return self.runner.run_pipeline(self, self._options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 75, in run_pipeline self.result.cancel() File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1658, in cancel return self.state File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1596, in state self._update_job() File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 1549, in _update_job self._job = self._runner.dataflow_client.get_job(self.job_id()) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/utils/retry.py",> line 236, in wrapper return fun(*args, **kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",> line 806, in get_job response = self._client.projects_locations_jobs.Get(request) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/internal/clients/dataflow/dataflow_v1b3_client.py",> line 699, in Get config, request, global_params=global_params) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/apitools/base/py/base_api.py",> line 729, in _RunMethod http, http_request, **opts) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/apitools/base/py/http_wrapper.py",> line 346, in MakeRequest check_response_func=check_response_func) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/apitools/base/py/http_wrapper.py",> line 396, in _MakeRequestNoRetry redirections=redirections, connection_type=connection_type) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/oauth2client/transport.py",> line 169, in new_request redirections, connection_type) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/oauth2client/transport.py",> line 169, in new_request redirections, connection_type) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/httplib2/__init__.py",> line 2133, in request cachekey, File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/httplib2/__init__.py",> line 1796, in _request conn, request_uri, method, body, headers File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/httplib2/__init__.py",> line 1737, in _conn_request response = conn.getresponse() File "/usr/lib/python2.7/httplib.py", line 1152, in getresponse response.begin() File "/usr/lib/python2.7/httplib.py", line 463, in begin version, status, reason = self._read_status() File "/usr/lib/python2.7/httplib.py", line 419, in _read_status line = self.fp.readline(_MAXLINE + 1) File "/usr/lib/python2.7/socket.py", line 480, in readline data = self._sock.recv(self._rbufsize) File "/usr/lib/python2.7/ssl.py", line 756, in recv return self.read(buflen) File "/usr/lib/python2.7/ssl.py", line 643, in read v = self._sslobj.read(len) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py",> line 276, in signalhandler raise TimedOutException() TimedOutException: 'test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT)' ====================================================================== ERROR: test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py",> line 812, in run test(orig) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",> line 45, in __call__ return self.run(*arg, **kwarg) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",> line 133, in run self.runTest(result) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/case.py",> line 151, in runTest test(result) File "/usr/lib/python2.7/unittest/case.py", line 393, in __call__ return self.run(*args, **kwds) File "/usr/lib/python2.7/unittest/case.py", line 329, in run testMethod() File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py",> line 57, in test_metrics_fnapi_it result = self.run_pipeline(experiment='beam_fn_api') File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py",> line 45, in run_pipeline return dataflow_exercise_metrics_pipeline.apply_and_run(p) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline.py",> line 167, in apply_and_run | 'm_out' >> beam.FlatMap( File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/ptransform.py",> line 562, in __ror__ result = p.apply(self, pvalueish, label) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 634, in apply pvalueish_result = self.runner.apply(transform, pvalueish, self._options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 146, in apply return super(DataflowRunner, self).apply(transform, input, options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply return m(transform, input, options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform return transform.expand(input) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/core.py",> line 2716, in expand | Map(coder.decode).with_output_types(self.get_output_type())) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pvalue.py",> line 140, in __or__ return self.pipeline.apply(ptransform, self) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 634, in apply pvalueish_result = self.runner.apply(transform, pvalueish, self._options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 146, in apply return super(DataflowRunner, self).apply(transform, input, options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply return m(transform, input, options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform return transform.expand(input) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/core.py",> line 2707, in expand return pcoll | Reshuffle() File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pvalue.py",> line 140, in __or__ return self.pipeline.apply(ptransform, self) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 634, in apply pvalueish_result = self.runner.apply(transform, pvalueish, self._options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 146, in apply return super(DataflowRunner, self).apply(transform, input, options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply return m(transform, input, options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform return transform.expand(input) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/util.py",> line 716, in expand | 'RemoveRandomKeys' >> Map(lambda t: t[1]).with_input_types( File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pvalue.py",> line 140, in __or__ return self.pipeline.apply(ptransform, self) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 634, in apply pvalueish_result = self.runner.apply(transform, pvalueish, self._options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 146, in apply return super(DataflowRunner, self).apply(transform, input, options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py",> line 198, in apply return m(transform, input, options) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/runner.py",> line 228, in apply_PTransform return transform.expand(input) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/util.py",> line 674, in expand ungrouped = pcoll | Map(reify_timestamps).with_output_types(Any) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/core.py",> line 1534, in Map pardo = FlatMap(wrapper, *args, **kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/core.py",> line 1480, in FlatMap pardo = ParDo(CallableWrapperDoFn(fn), *args, **kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/core.py",> line 1246, in __init__ self._signature = DoFnSignature(self.fn) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/common.py",> line 286, in __init__ self._validate() File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/common.py",> line 311, in _validate self._validate_stateful_dofn() File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/common.py",> line 338, in _validate_stateful_dofn userstate.validate_stateful_dofn(self.do_fn) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/userstate.py",> line 272, in validate_stateful_dofn all_state_specs, all_timer_specs = get_dofn_specs(dofn) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/userstate.py",> line 243, in get_dofn_specs method = MethodWrapper(dofn, method_name) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/common.py",> line 183, in __init__ method_name) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/core.py",> line 339, in get_function_arguments return get_function_args_defaults(f) File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/core.py",> line 369, in get_function_args_defaults p.default for p in signature.parameters.values() File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/funcsigs/__init__.py",> line 601, in parameters return OrderedDict(self._parameters.items()) File "/usr/lib/python2.7/collections.py", line 127, in items return [(key, self[key]) for key in self] File "/usr/lib/python2.7/collections.py", line 96, in __iter__ yield curr[2] # yield the curr[KEY] File "<https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python/local/lib/python2.7/site-packages/nose/plugins/multiprocess.py",> line 276, in signalhandler raise TimedOutException() TimedOutException: 'test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest)' ---------------------------------------------------------------------- XML: nosetests-beam_python2.7_sdk.xml ---------------------------------------------------------------------- XML: <https://builds.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml> ---------------------------------------------------------------------- Ran 2 tests in 904.251s FAILED (errors=2) cleanup_container Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk:20200420-213952 Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk@sha256:eb004e95160b02a9f0d4c156cd89b6bd154cb22f64a84e0a50051851a86fe899 Deleted: sha256:f2cf01b1e906d447390e4e5a4ac89d61eed036501c225d5ff338e1f1649a8fff Deleted: sha256:6894a60f255f481bdba470846ad36926f2298e6fa52f505871b2430407ee3e37 Deleted: sha256:c845abba0acfcbf638d94fe35f1ce921932bc0bf40a1e4c46dc072342471db5d Deleted: sha256:bf92a53395b5bd2bf61aff3cc4a15aef5d653e9f9859fb352a193406991a866d Deleted: sha256:5334ba3d37a65985ab77cd835e9c27369c3b5fa898f1cbf7535e9c0095217329 Deleted: sha256:c30bae8016b83e4c82ae4215ead18cc881fc6de9ace8beb1f183ed03fd483181 Deleted: sha256:a344a3c4722550cb4c7cab094f04b56256e44c201c98ff74a8a9a5dac07c4409 Deleted: sha256:2ce17dbd4c34ec6cd2089a907ba4e90743a8ca2b5830441d536fe852840cb226 Deleted: sha256:adc42487f6c77c703703d3537d47a3ab450d83a705f4cdc2b40297c1617e9e94 Deleted: sha256:8be1ac5bb88d636c0ece473029e29182bfacf4840f882c57cdbb61ff15665f0a Deleted: sha256:37cca2946f13db2e849374509bbbdc3cdf494665e7f12e95145d2de93f92067e Deleted: sha256:b7b76cfd00049d7fd273a5563903e37e12571ba546f88c3717b71723a7cb4d91 Deleted: sha256:62b6d642209f54e74f248466b87abc883e255372e0ebe3f02411c7c5d7877a10 Deleted: sha256:08a316d5276b778136d167bb07915d042510c0f4f70b0e1b56ad77223f1be9bb Deleted: sha256:b19e0ee3b9c445dc4280ecf5536bc75f96367fa2bb6c9b419dde0ac75115cbf2 Deleted: sha256:8b8f48ee18af59333aa2cebd76ebe303adaaffbeef7177fe4fcb61410871fbcb Deleted: sha256:55d178ee1fdd867e92dcd343d4001abfe2d51b4fe740b515ad351c067a223f87 Deleted: sha256:73d62c1436cc7bb30a80867c6e62fdbda96bb8e6fdfb12a994c759df51218a1b Deleted: sha256:c91083c36e23385f553d9ab77b714a784e86af939d3b5d9252d12d9587e8b353 Deleted: sha256:92a2208ed124273a09a620546db96acf020b326571a41d122a44a0bb57886969 Deleted: sha256:f74fe48022dcdef84c207488f6f1e0dbbba247fde1d611dd1b6a70d050a377d3 Deleted: sha256:02b09eb4ee4ad82e93b03c34ca057fae2150a86f83bedcc2f310548e805e7051 Deleted: sha256:21a25561f755aafa69bc2db8377b52546bf4a4958047a106e9e47237cf8d369e Deleted: sha256:77f9337711366a5b3e65f45fe3eed7c692aca4f2dcf252fca0de06d00bd58201 Deleted: sha256:b93c742f2b767e7adf6630631e16f04bdfef4fe6172650f229f772f86cbf1c42 Deleted: sha256:2a7d7e5e23a14b2a8b34fc19159134e49888faa0c8ec37a13c1eeab793cdeba4 Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk@sha256:eb004e95160b02a9f0d4c156cd89b6bd154cb22f64a84e0a50051851a86fe899 Associated tags: - 20200420-213952 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk:20200420-213952 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk:20200420-213952]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python2.7_sdk@sha256:eb004e95160b02a9f0d4c156cd89b6bd154cb22f64a84e0a50051851a86fe899]. Removed the container Build step 'Execute shell' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
