See <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/4130/display/redirect>
------------------------------------------ [...truncated 618.77 KB...] type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> root: INFO: Created job with id: [2019-07-26_23_09_17-16044876168138452283] root: INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_09_17-16044876168138452283?project=apache-beam-testing root: INFO: Job 2019-07-26_23_09_17-16044876168138452283 is in state JOB_STATE_RUNNING root: INFO: 2019-07-27T06:09:17.198Z: JOB_MESSAGE_DETAILED: Autoscaling is enabled for job 2019-07-26_23_09_17-16044876168138452283. The number of workers will be between 1 and 1000. root: INFO: 2019-07-27T06:09:17.264Z: JOB_MESSAGE_DETAILED: Autoscaling was automatically enabled for job 2019-07-26_23_09_17-16044876168138452283. root: INFO: 2019-07-27T06:09:20.385Z: JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service Account. root: INFO: 2019-07-27T06:09:21.037Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-1 in us-central1-a. root: INFO: 2019-07-27T06:09:21.601Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts. root: INFO: 2019-07-27T06:09:21.648Z: JOB_MESSAGE_DEBUG: Combiner lifting skipped for step assert_that/Group/GroupByKey: GroupByKey not followed by a combiner. root: INFO: 2019-07-27T06:09:21.689Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts. root: INFO: 2019-07-27T06:09:21.732Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns root: INFO: 2019-07-27T06:09:21.816Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information. root: INFO: 2019-07-27T06:09:21.881Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations root: INFO: 2019-07-27T06:09:21.921Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/Map(_merge_tagged_vals_under_key) into assert_that/Group/GroupByKey/GroupByWindow root: INFO: 2019-07-27T06:09:21.965Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Match into assert_that/Unkey root: INFO: 2019-07-27T06:09:22.012Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Unkey into assert_that/Group/Map(_merge_tagged_vals_under_key) root: INFO: 2019-07-27T06:09:22.050Z: JOB_MESSAGE_DETAILED: Unzipping flatten s11 for input s10.out root: INFO: 2019-07-27T06:09:22.094Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of assert_that/Group/GroupByKey/Reify, through flatten assert_that/Group/Flatten, into producer assert_that/Group/pair_with_1 root: INFO: 2019-07-27T06:09:22.141Z: JOB_MESSAGE_DETAILED: Unzipping flatten s11-u13 for input s12-reify-value0-c11 root: INFO: 2019-07-27T06:09:22.179Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of assert_that/Group/GroupByKey/Write, through flatten assert_that/Group/Flatten/Unzipped-1, into producer assert_that/Group/GroupByKey/Reify root: INFO: 2019-07-27T06:09:22.223Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/GroupByKey/GroupByWindow into assert_that/Group/GroupByKey/Read root: INFO: 2019-07-27T06:09:22.259Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/GroupByKey/Reify into assert_that/Group/pair_with_0 root: INFO: 2019-07-27T06:09:22.297Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/GroupByKey/Write into assert_that/Group/GroupByKey/Reify root: INFO: 2019-07-27T06:09:22.331Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/pair_with_0 into assert_that/Create/Read root: INFO: 2019-07-27T06:09:22.370Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/ToVoidKey into assert_that/WindowInto(WindowIntoFn) root: INFO: 2019-07-27T06:09:22.412Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/WindowInto(WindowIntoFn) into Map(<lambda at sideinputs_test.py:238>)/Map(<lambda at sideinputs_test.py:238>) root: INFO: 2019-07-27T06:09:22.451Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/pair_with_1 into assert_that/ToVoidKey root: INFO: 2019-07-27T06:09:22.495Z: JOB_MESSAGE_DETAILED: Fusing consumer Map(<lambda at sideinputs_test.py:238>)/Map(<lambda at sideinputs_test.py:238>) into main input/Read root: INFO: 2019-07-27T06:09:22.531Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec. root: INFO: 2019-07-27T06:09:22.575Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph. root: INFO: 2019-07-27T06:09:22.610Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps. root: INFO: 2019-07-27T06:09:22.649Z: JOB_MESSAGE_DEBUG: Assigning stage ids. root: INFO: 2019-07-27T06:09:22.840Z: JOB_MESSAGE_DEBUG: Executing wait step start21 root: INFO: 2019-07-27T06:09:22.933Z: JOB_MESSAGE_BASIC: Executing operation assert_that/Group/GroupByKey/Create root: INFO: 2019-07-27T06:09:22.975Z: JOB_MESSAGE_BASIC: Executing operation side list/Read root: INFO: 2019-07-27T06:09:22.989Z: JOB_MESSAGE_DEBUG: Starting worker pool setup. root: INFO: 2019-07-27T06:09:23.024Z: JOB_MESSAGE_BASIC: Finished operation side list/Read root: INFO: 2019-07-27T06:09:23.036Z: JOB_MESSAGE_BASIC: Starting 1 workers in us-central1-a... root: INFO: 2019-07-27T06:09:23.096Z: JOB_MESSAGE_BASIC: Finished operation assert_that/Group/GroupByKey/Create root: INFO: 2019-07-27T06:09:23.123Z: JOB_MESSAGE_DEBUG: Value "side list/Read.out" materialized. root: INFO: 2019-07-27T06:09:23.192Z: JOB_MESSAGE_DEBUG: Value "assert_that/Group/GroupByKey/Session" materialized. root: INFO: 2019-07-27T06:09:23.233Z: JOB_MESSAGE_BASIC: Executing operation Map(<lambda at sideinputs_test.py:238>)/_UnpickledSideInput(Read.out.0) root: INFO: 2019-07-27T06:09:23.272Z: JOB_MESSAGE_BASIC: Executing operation Map(<lambda at sideinputs_test.py:238>)/_UnpickledSideInput(Read.out.1) root: INFO: 2019-07-27T06:09:23.275Z: JOB_MESSAGE_BASIC: Finished operation Map(<lambda at sideinputs_test.py:238>)/_UnpickledSideInput(Read.out.0) root: INFO: 2019-07-27T06:09:23.301Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-1 in us-central1-a. root: INFO: 2019-07-27T06:09:23.309Z: JOB_MESSAGE_BASIC: Executing operation assert_that/Create/Read+assert_that/Group/pair_with_0+assert_that/Group/GroupByKey/Reify+assert_that/Group/GroupByKey/Write root: INFO: 2019-07-27T06:09:23.311Z: JOB_MESSAGE_BASIC: Finished operation Map(<lambda at sideinputs_test.py:238>)/_UnpickledSideInput(Read.out.1) root: INFO: 2019-07-27T06:09:23.362Z: JOB_MESSAGE_DEBUG: Value "Map(<lambda at sideinputs_test.py:238>)/_UnpickledSideInput(Read.out.0).output" materialized. root: INFO: 2019-07-27T06:09:23.398Z: JOB_MESSAGE_DEBUG: Value "Map(<lambda at sideinputs_test.py:238>)/_UnpickledSideInput(Read.out.1).output" materialized. root: INFO: 2019-07-27T06:09:23.500Z: JOB_MESSAGE_BASIC: Executing operation main input/Read+Map(<lambda at sideinputs_test.py:238>)/Map(<lambda at sideinputs_test.py:238>)+assert_that/WindowInto(WindowIntoFn)+assert_that/ToVoidKey+assert_that/Group/pair_with_1+assert_that/Group/GroupByKey/Reify+assert_that/Group/GroupByKey/Write root: INFO: 2019-07-27T06:10:38.273Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s). root: INFO: 2019-07-27T06:10:46.332Z: JOB_MESSAGE_DETAILED: Workers have started successfully. root: INFO: 2019-07-27T06:10:46.372Z: JOB_MESSAGE_DETAILED: Workers have started successfully. --------------------- >> end captured logging << --------------------- ---------------------------------------------------------------------- XML: nosetests-validatesRunnerBatchTests-df.xml ---------------------------------------------------------------------- XML: <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/nosetests.xml> ---------------------------------------------------------------------- Ran 17 tests in 1251.914s FAILED (failures=5) Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_01_31-9460421448468004564?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_08_42-15782623495599363378?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_01_33-11654486132952016261?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_09_28-8622648415513399483?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_01_33-1230157420844595820?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_09_33-11605761103783478212?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_01_32-3731572127840426172?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_09_47-18241229147229057455?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_14_33-16518728134990777961?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_01_33-189406782887530204?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_10_08-183899830771076682?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_01_32-6310056600417931175?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_09_17-16044876168138452283?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_01_33-2770852922842503634?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_10_13-5530466878730180351?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_01_33-10274237867557998022?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_09_12-11024709144101565150?project=apache-beam-testing. > Task :sdks:python:validatesRunnerBatchTests FAILED > Task :sdks:python:validatesRunnerStreamingTests >>> RUNNING integration tests with pipeline options: >>> --runner=TestDataflowRunner --project=apache-beam-testing >>> --staging_location=gs://temp-storage-for-end-to-end-tests/staging-it >>> --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it >>> --output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output >>> --sdk_location=build/apache-beam.tar.gz >>> --requirements_file=postcommit_requirements.txt --num_workers=1 >>> --sleep_secs=20 --streaming >>> --dataflow_worker_jar=<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/build/libs/beam-runners-google-cloud-dataflow-java-fn-api-worker-2.15.0-SNAPSHOT.jar> >>> >>> --kms_key_name=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test >>> >>> --dataflow_kms_key=projects/apache-beam-testing/locations/global/keyRings/beam-it/cryptoKeys/test >>> test options: --nocapture --processes=8 --process-timeout=4500 >>> --attr=ValidatesRunner,!sickbay-streaming <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/setuptools/dist.py>:472: UserWarning: Normalizing '2.15.0.dev' to '2.15.0.dev0' normalized_version, running nosetests running egg_info writing requirements to apache_beam.egg-info/requires.txt writing apache_beam.egg-info/PKG-INFO writing top-level names to apache_beam.egg-info/top_level.txt writing dependency_links to apache_beam.egg-info/dependency_links.txt writing entry points to apache_beam.egg-info/entry_points.txt reading manifest file 'apache_beam.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching 'README.md' warning: no files found matching 'NOTICE' warning: no files found matching 'LICENSE' writing manifest file 'apache_beam.egg-info/SOURCES.txt' test_flatten_multiple_pcollections_having_multiple_consumers (apache_beam.transforms.ptransform_test.PTransformTest) ... ok test_undeclared_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok test_par_do_with_multiple_outputs_and_using_return (apache_beam.transforms.ptransform_test.PTransformTest) ... ok test_dofn_lifecycle (apache_beam.transforms.dofn_lifecycle_test.DoFnLifecycleTest) ... ok test_as_list_and_as_dict_side_inputs (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) ... ok test_par_do_with_multiple_outputs_and_using_yield (apache_beam.transforms.ptransform_test.PTransformTest) ... ok test_as_dict_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_as_singleton_with_different_defaults (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_default_value_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_as_list_twice (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_as_singleton_without_unique_labels (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_flattened_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_iterable_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok test_empty_singleton_side_input (apache_beam.transforms.sideinputs_test.SideInputsTest) ... ok ---------------------------------------------------------------------- XML: nosetests-validatesRunnerStreamingTests-df.xml ---------------------------------------------------------------------- XML: <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/nosetests.xml> ---------------------------------------------------------------------- Ran 15 tests in 1132.198s OK Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_22_21-9617016664646473389?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_30_51-6983748372591504547?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_22_20-11160476618862440475?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_30_25-4803406998904287797?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_22_22-10465935877199027064?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_29_46-14781207011373419197?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_22_22-12171073071228704830?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_30_52-8833264010651986453?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_22_21-7953890273046132053?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_30_21-7000979796318370651?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_22_21-2293448599971577548?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_31_21-10620748756739928596?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_22_21-961290327847635534?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_30_02-15090547286709827096?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-07-26_23_22_21-5742371911382302852?project=apache-beam-testing. FAILURE: Build completed with 7 failures. 1: Task failed with an exception. ----------- * Where: Build file '<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/py36/build.gradle'> line: 69 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesRunnerBatchTests'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 2: Task failed with an exception. ----------- * Where: Build file '<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/py35/build.gradle'> line: 69 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py35:validatesRunnerBatchTests'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 3: Task failed with an exception. ----------- * Where: Build file '<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/py36/build.gradle'> line: 90 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py36:validatesRunnerStreamingTests'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 4: Task failed with an exception. ----------- * Where: Build file '<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/py37/build.gradle'> line: 100 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesRunnerBatchTests'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 5: Task failed with an exception. ----------- * Where: Build file '<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/py35/build.gradle'> line: 90 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py35:validatesRunnerStreamingTests'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 6: Task failed with an exception. ----------- * Where: Build file '<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/test-suites/dataflow/py37/build.gradle'> line: 121 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesRunnerStreamingTests'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== 7: Task failed with an exception. ----------- * Where: Build file '<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/build.gradle'> line: 202 * What went wrong: Execution failed for task ':sdks:python:validatesRunnerBatchTests'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. ============================================================================== * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 40m 50s 76 actionable tasks: 59 executed, 17 from cache Publishing build scan... https://gradle.com/s/alc3pwmuaz6nc Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
