See <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1324/display/redirect?page=changes>
Changes: [ankurgoenka] [BEAM-5697] Support legacy pipeline option deserialization ------------------------------------------ [...truncated 82.81 KB...] Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.3.0.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.3.0.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.3.0.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.3.0.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.3.0.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.3.0.tar.gz Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1)) Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.3.0.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.4.3.zip Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.3.0.tar.gz Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported print('Found: %s.' % self.build_console_url(pipeline.options)) <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported print('Found: %s.' % self.build_console_url(pipeline.options)) <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported print('Found: %s.' % self.build_console_url(pipeline.options)) <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported print('Found: %s.' % self.build_console_url(pipeline.options)) <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported print('Found: %s.' % self.build_console_url(pipeline.options)) <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported print('Found: %s.' % self.build_console_url(pipeline.options)) <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported print('Found: %s.' % self.build_console_url(pipeline.options)) <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported print('Found: %s.' % self.build_console_url(pipeline.options)) WARNING:root:Waiting indefinitely for streaming job. WARNING:root:Waiting indefinitely for streaming job. WARNING:root:Waiting indefinitely for streaming job. WARNING:root:Waiting indefinitely for streaming job. WARNING:root:Waiting indefinitely for streaming job. WARNING:root:Waiting indefinitely for streaming job. WARNING:root:Waiting indefinitely for streaming job. WARNING:root:Waiting indefinitely for streaming job. test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.4.3.zip Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.3.0.tar.gz Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr WARNING:root:Waiting indefinitely for streaming job. test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.4.3.zip Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.4.3.zip Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.3.0.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.3.0.tar.gz Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest) ... ok Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.4.3.zip Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.3.0.tar.gz Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz WARNING:root:Waiting indefinitely for streaming job. Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.4.3.zip WARNING:root:Waiting indefinitely for streaming job. Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.3.0.tar.gz Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok WARNING:root:Waiting indefinitely for streaming job. test_par_do_with_multiple_outputs_and_using_return (apache_beam.transforms.ptransform_test.PTransformTest) ... ok WARNING:root:Waiting indefinitely for streaming job. test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok ====================================================================== ERROR: test_flatten_multiple_pcollections_having_multiple_consumers (apache_beam.transforms.ptransform_test.PTransformTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/transforms/ptransform_test.py";,> line 571, in test_flatten_multiple_pcollections_having_multiple_consumers pipeline.run() File "<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/testing/test_pipeline.py";,> line 104, in run result = super(TestPipeline, self).run(test_runner_api) File "<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/pipeline.py";,> line 403, in run self.to_runner_api(), self.runner, self._options).run(False) File "<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/pipeline.py";,> line 416, in run return self.runner.run_pipeline(self) File "<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py";,> line 50, in run_pipeline self.result = super(TestDataflowRunner, self).run_pipeline(pipeline) File "<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py";,> line 389, in run_pipeline self.dataflow_client.create_job(self.job), self) File "<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/utils/retry.py";,> line 184, in wrapper return fun(*args, **kwargs) File "<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py";,> line 490, in create_job self.create_job_description(job) File "<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py";,> line 519, in create_job_description resources = self._stage_resources(job.options) File "<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py";,> line 452, in _stage_resources staging_location=google_cloud_options.staging_location) File "<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/portability/stager.py";,> line 161, in stage_job_resources requirements_cache_path) File "<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/portability/stager.py";,> line 411, in _populate_requirements_cache processes.check_call(cmd_args) File "<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/utils/processes.py";,> line 46, in check_call return subprocess.check_call(*args, **kwargs) File "/usr/lib/python2.7/subprocess.py", line 541, in check_call raise CalledProcessError(retcode, cmd) CalledProcessError: Command '['<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/build/gradleenv/bin/python',> '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', ':all:']' returned non-zero exit status 1 ---------------------------------------------------------------------- XML: <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/nosetests.xml> ---------------------------------------------------------------------- Ran 14 tests in 851.102s FAILED (errors=1) > Task :beam-sdks-python:validatesRunnerStreamingTests FAILED :beam-sdks-python:validatesRunnerStreamingTests (Thread[Task worker for ':',5,main]) completed. Took 14 mins 12.709 secs. FAILURE: Build completed with 2 failures. 1: Task failed with an exception. ----------- * Where: Build file '<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/build.gradle'> line: 317 * What went wrong: Execution failed for task ':beam-sdks-python:validatesRunnerBatchTests'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 2: Task failed with an exception. ----------- * Where: Build file '<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/build.gradle'> line: 326 * What went wrong: Execution failed for task ':beam-sdks-python:validatesRunnerStreamingTests'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights. ============================================================================== * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 1h 12m 1s 4 actionable tasks: 4 executed Publishing build scan... https://gradle.com/s/4a7a3ne5kusz4 Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure
