See <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/662/display/redirect?page=changes>
Changes: [rober] Avoid redundant computation when invoking ProcessElements in ParDo [github] Update fn_test.go [rober] De-dup pre & post invoke code in pardo.go [github] Properly revert log line changes in fn_test.go [rober] Add comparitive benchmark between cached and single use invoker. [ebeach] [BEAM-4862] Fixes bug in Spanner's MutationGroupEncoder by converting [herohde] Add utilities for manipulating proto pipelines [herohde] [BEAM-4636] Refactor Go dataflow runner for more flexible uses [herohde] [BEAM-4813] Switch to translation from model protos to Dataflow v1beta3 [herohde] Add more Go integration tests and fix CoGBK bugs [herohde] [BEAM-4636] Ensure unique filename for compiled worker binaries [herohde] Handle explicit flatten [herohde] Fix Dataflow argument bug [herohde] CR: tweak code and fix comments [aaltay] [BEAM-4751] fix missing pylint3 check for io subpackage (#5916) ------------------------------------------ [...truncated 215.68 KB...] File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz Collecting mock (from -r postcommit_requirements.txt (line 2)) Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz Collecting mock (from -r postcommit_requirements.txt (line 2)) Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.0.0.zip File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.0.0.zip File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.0.0.zip File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.0.0.zip File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.0.0.zip File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.0.0.zip File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.0.0.zip File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.0.0.zip Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.2.0.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.2.0.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.2.0.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.2.0.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.2.0.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.2.0.tar.gz File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.2.0.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.2.0.tar.gz Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported print('Found: %s.' % self.build_console_url(pipeline.options)) <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported print('Found: %s.' % self.build_console_url(pipeline.options)) <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported print('Found: %s.' % self.build_console_url(pipeline.options)) <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported print('Found: %s.' % self.build_console_url(pipeline.options)) <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported print('Found: %s.' % self.build_console_url(pipeline.options)) <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported print('Found: %s.' % self.build_console_url(pipeline.options)) <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported print('Found: %s.' % self.build_console_url(pipeline.options)) <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:54: DeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported print('Found: %s.' % self.build_console_url(pipeline.options)) test_flatten_multiple_pcollections_having_multiple_consumers (apache_beam.transforms.ptransform_test.PTransformTest) ... ok Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.0.0.zip Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.2.0.tar.gz Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.0.0.zip Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.2.0.tar.gz Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr test_par_do_with_multiple_outputs_and_using_return (apache_beam.transforms.ptransform_test.PTransformTest) ... ok Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.0.0.zip Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.2.0.tar.gz Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest) ... ok Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.0.0.zip Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.2.0.tar.gz Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.0.0.zip Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.2.0.tar.gz Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok Collecting pyhamcrest (from -r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/PyHamcrest-1.9.0.tar.gz Collecting mock (from -r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/mock-2.0.0.tar.gz Collecting setuptools (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/setuptools-40.0.0.zip Collecting six (from pyhamcrest->-r postcommit_requirements.txt (line 1)) File was already downloaded /tmp/dataflow-requirements-cache/six-1.11.0.tar.gz Collecting funcsigs>=1 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/funcsigs-1.0.2.tar.gz Collecting pbr>=0.11 (from mock->-r postcommit_requirements.txt (line 2)) File was already downloaded /tmp/dataflow-requirements-cache/pbr-4.2.0.tar.gz Successfully downloaded pyhamcrest mock setuptools six funcsigs pbr test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok ---------------------------------------------------------------------- XML: <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/nosetests.xml> ---------------------------------------------------------------------- Ran 14 tests in 1068.609s OK Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-07-30_11_18_35-2874758296689184047?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-07-30_11_25_18-17420576355803193626?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-07-30_11_18_35-17511752578304635560?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-07-30_11_18_34-10045992208741738649?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-07-30_11_26_18-5600933279780173805?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-07-30_11_18_35-13622949327172453093?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-07-30_11_25_38-10352217940383528766?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-07-30_11_18_35-7937352806674474834?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-07-30_11_25_58-12691729623340350368?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-07-30_11_18_35-14727019040213159470?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-07-30_11_25_28-11551601176028696269?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-07-30_11_18_34-6751832118223324777?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-07-30_11_18_34-5291373507925183457?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-07-30_11_26_28-4904954088025835772?project=apache-beam-testing. :beam-sdks-python:validatesRunnerStreamingTests (Thread[Task worker for ':',5,main]) completed. Took 17 mins 50.306 secs. FAILURE: Build failed with an exception. * Where: Build file '<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/build.gradle'> line: 237 * What went wrong: Execution failed for task ':beam-sdks-python:validatesRunnerBatchTests'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0. See https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 36m 1s 4 actionable tasks: 4 executed Publishing build scan... https://gradle.com/s/g35ldtxyupc3k Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure
