See <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/436/display/redirect?page=changes>
Changes: [amaliujia] support unbounded limit. ------------------------------------------ [...truncated 201.56 KB...] "component_encodings": [ { "@type": "FastPrimitivesCoder$eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/", "component_encodings": [ { "@type": "FastPrimitivesCoder$eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/", "component_encodings": [] }, { "@type": "FastPrimitivesCoder$eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/", "component_encodings": [] } ], "is_pair_like": true }, { "@type": "kind:global_window" } ], "is_wrapper": true }, "output_name": "out", "user_name": "assert:even/Unkey.out" } ], "parallel_input": { "@type": "OutputReference", "output_name": "out", "step_name": "s30" }, "serialized_fn": "<string of 960 bytes>", "user_name": "assert:even/Unkey" } }, { "kind": "ParallelDo", "name": "s32", "properties": { "display_data": [ { "key": "fn", "label": "Transform Function", "namespace": "apache_beam.transforms.core.CallableWrapperDoFn", "type": "STRING", "value": "_equal" }, { "key": "fn", "label": "Transform Function", "namespace": "apache_beam.transforms.core.ParDo", "shortValue": "CallableWrapperDoFn", "type": "STRING", "value": "apache_beam.transforms.core.CallableWrapperDoFn" } ], "non_parallel_inputs": {}, "output_info": [ { "encoding": { "@type": "kind:windowed_value", "component_encodings": [ { "@type": "FastPrimitivesCoder$eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/", "component_encodings": [ { "@type": "FastPrimitivesCoder$eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/", "component_encodings": [] }, { "@type": "FastPrimitivesCoder$eNprYEpOLEhMzkiNT0pNzNVLzk9JLSqGUlxuicUlAUWZuZklmWWpxc4gQa5CBs3GQsbaQqZQ/vi0xJycpMTk7Hiw+kJmPEYFZCZn56RCjWABGsFaW8iWVJykBwDlGS3/", "component_encodings": [] } ], "is_pair_like": true }, { "@type": "kind:global_window" } ], "is_wrapper": true }, "output_name": "out", "user_name": "assert:even/Match.out" } ], "parallel_input": { "@type": "OutputReference", "output_name": "out", "step_name": "s31" }, "serialized_fn": "<string of 1140 bytes>", "user_name": "assert:even/Match" } } ], "type": "JOB_TYPE_BATCH" } root: INFO: Create job: <Job createTime: u'2018-06-21T22:34:10.026666Z' currentStateTime: u'1970-01-01T00:00:00Z' id: u'2018-06-21_15_34_09-13638563469120977061' location: u'us-central1' name: u'beamapp-jenkins-0621223403-760126' projectId: u'apache-beam-testing' stageStates: [] steps: [] tempFiles: [] type: TypeValueValuesEnum(JOB_TYPE_BATCH, 1)> root: INFO: Created job with id: [2018-06-21_15_34_09-13638563469120977061] root: INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-21_15_34_09-13638563469120977061?project=apache-beam-testing root: INFO: Job 2018-06-21_15_34_09-13638563469120977061 is in state JOB_STATE_RUNNING root: INFO: 2018-06-21T22:34:09.141Z: JOB_MESSAGE_WARNING: Job 2018-06-21_15_34_09-13638563469120977061 might autoscale up to 1000 workers. root: INFO: 2018-06-21T22:34:09.169Z: JOB_MESSAGE_DETAILED: Autoscaling is enabled for job 2018-06-21_15_34_09-13638563469120977061. The number of workers will be between 1 and 1000. root: INFO: 2018-06-21T22:34:09.197Z: JOB_MESSAGE_DETAILED: Autoscaling was automatically enabled for job 2018-06-21_15_34_09-13638563469120977061. root: INFO: 2018-06-21T22:34:12.715Z: JOB_MESSAGE_DETAILED: Checking required Cloud APIs are enabled. root: INFO: 2018-06-21T22:34:12.882Z: JOB_MESSAGE_DETAILED: Checking permissions granted to controller Service Account. root: INFO: 2018-06-21T22:34:13.734Z: JOB_MESSAGE_BASIC: Worker configuration: n1-standard-1 in us-central1-b. root: INFO: 2018-06-21T22:34:14.248Z: JOB_MESSAGE_DETAILED: Expanding CoGroupByKey operations into optimizable parts. root: INFO: 2018-06-21T22:34:14.340Z: JOB_MESSAGE_DEBUG: Combiner lifting skipped for step assert:even/Group/GroupByKey: GroupByKey not followed by a combiner. root: INFO: 2018-06-21T22:34:14.389Z: JOB_MESSAGE_DEBUG: Combiner lifting skipped for step assert:odd/Group/GroupByKey: GroupByKey not followed by a combiner. root: INFO: 2018-06-21T22:34:14.446Z: JOB_MESSAGE_DEBUG: Combiner lifting skipped for step assert_that/Group/GroupByKey: GroupByKey not followed by a combiner. root: INFO: 2018-06-21T22:34:14.498Z: JOB_MESSAGE_DETAILED: Expanding GroupByKey operations into optimizable parts. root: INFO: 2018-06-21T22:34:14.531Z: JOB_MESSAGE_DETAILED: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns root: INFO: 2018-06-21T22:34:14.723Z: JOB_MESSAGE_DEBUG: Annotating graph with Autotuner information. root: INFO: 2018-06-21T22:34:14.824Z: JOB_MESSAGE_DETAILED: Fusing adjacent ParDo, Read, Write, and Flatten operations root: INFO: 2018-06-21T22:34:14.879Z: JOB_MESSAGE_DETAILED: Unzipping flatten s8 for input s6.out root: INFO: 2018-06-21T22:34:14.947Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of assert_that/Group/GroupByKey/Reify, through flatten assert_that/Group/Flatten, into producer assert_that/Group/pair_with_0 root: INFO: 2018-06-21T22:34:14.968Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Unkey into assert_that/Group/Map(_merge_tagged_vals_under_key) root: INFO: 2018-06-21T22:34:14.989Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Match into assert_that/Unkey root: INFO: 2018-06-21T22:34:15.013Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:odd/Unkey into assert:odd/Group/Map(_merge_tagged_vals_under_key) root: INFO: 2018-06-21T22:34:15.033Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:odd/Match into assert:odd/Unkey root: INFO: 2018-06-21T22:34:15.067Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:odd/Group/Map(_merge_tagged_vals_under_key) into assert:odd/Group/GroupByKey/GroupByWindow root: INFO: 2018-06-21T22:34:15.108Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/GroupByKey/GroupByWindow into assert_that/Group/GroupByKey/Read root: INFO: 2018-06-21T22:34:15.145Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/Map(_merge_tagged_vals_under_key) into assert_that/Group/GroupByKey/GroupByWindow root: INFO: 2018-06-21T22:34:15.168Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:odd/Group/GroupByKey/GroupByWindow into assert:odd/Group/GroupByKey/Read root: INFO: 2018-06-21T22:34:15.196Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:even/Match into assert:even/Unkey root: INFO: 2018-06-21T22:34:15.222Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:odd/Group/GroupByKey/Write into assert:odd/Group/GroupByKey/Reify root: INFO: 2018-06-21T22:34:15.250Z: JOB_MESSAGE_DETAILED: Unzipping flatten s18 for input s16.out root: INFO: 2018-06-21T22:34:15.281Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of assert:odd/Group/GroupByKey/Reify, through flatten assert:odd/Group/Flatten, into producer assert:odd/Group/pair_with_0 root: INFO: 2018-06-21T22:34:15.329Z: JOB_MESSAGE_DETAILED: Unzipping flatten s8-u31 for input s9-reify-value18-c29 root: INFO: 2018-06-21T22:34:15.362Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of assert_that/Group/GroupByKey/Write, through flatten s8-u31, into producer assert_that/Group/GroupByKey/Reify root: INFO: 2018-06-21T22:34:15.395Z: JOB_MESSAGE_DETAILED: Unzipping flatten s28 for input s26.out root: INFO: 2018-06-21T22:34:15.427Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of assert:even/Group/GroupByKey/Reify, through flatten assert:even/Group/Flatten, into producer assert:even/Group/pair_with_0 root: INFO: 2018-06-21T22:34:15.462Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:even/Group/GroupByKey/GroupByWindow into assert:even/Group/GroupByKey/Read root: INFO: 2018-06-21T22:34:15.493Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:even/Unkey into assert:even/Group/Map(_merge_tagged_vals_under_key) root: INFO: 2018-06-21T22:34:15.527Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:even/Group/Map(_merge_tagged_vals_under_key) into assert:even/Group/GroupByKey/GroupByWindow root: INFO: 2018-06-21T22:34:15.568Z: JOB_MESSAGE_DETAILED: Unzipping flatten s28-u42 for input s29-reify-value9-c40 root: INFO: 2018-06-21T22:34:15.611Z: JOB_MESSAGE_DETAILED: Fusing unzipped copy of assert:even/Group/GroupByKey/Write, through flatten s28-u42, into producer assert:even/Group/GroupByKey/Reify root: INFO: 2018-06-21T22:34:15.658Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/GroupByKey/Reify into assert_that/Group/pair_with_1 root: INFO: 2018-06-21T22:34:15.701Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:even/Group/GroupByKey/Write into assert:even/Group/GroupByKey/Reify root: INFO: 2018-06-21T22:34:15.734Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/GroupByKey/Write into assert_that/Group/GroupByKey/Reify root: INFO: 2018-06-21T22:34:15.772Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:even/Group/GroupByKey/Reify into assert:even/Group/pair_with_1 root: INFO: 2018-06-21T22:34:15.819Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:odd/Group/GroupByKey/Reify into assert:odd/Group/pair_with_1 root: INFO: 2018-06-21T22:34:15.864Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:even/ToVoidKey into assert:even/WindowInto(WindowIntoFn) root: INFO: 2018-06-21T22:34:15.911Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:even/Group/pair_with_1 into assert:even/ToVoidKey root: INFO: 2018-06-21T22:34:15.986Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/pair_with_1 into assert_that/ToVoidKey root: INFO: 2018-06-21T22:34:16.019Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/ToVoidKey into assert_that/WindowInto(WindowIntoFn) root: INFO: 2018-06-21T22:34:16.054Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:even/WindowInto(WindowIntoFn) into ClassifyNumbers/FlatMap(<lambda at ptransform_test.py:251>) root: INFO: 2018-06-21T22:34:16.097Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:odd/ToVoidKey into assert:odd/WindowInto(WindowIntoFn) root: INFO: 2018-06-21T22:34:16.123Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/WindowInto(WindowIntoFn) into ClassifyNumbers/FlatMap(<lambda at ptransform_test.py:251>) root: INFO: 2018-06-21T22:34:16.149Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:odd/Group/pair_with_1 into assert:odd/ToVoidKey root: INFO: 2018-06-21T22:34:16.212Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:odd/WindowInto(WindowIntoFn) into ClassifyNumbers/FlatMap(<lambda at ptransform_test.py:251>) root: INFO: 2018-06-21T22:34:16.242Z: JOB_MESSAGE_DETAILED: Fusing consumer ClassifyNumbers/FlatMap(<lambda at ptransform_test.py:251>) into Some Numbers/Read root: INFO: 2018-06-21T22:34:16.266Z: JOB_MESSAGE_DETAILED: Fusing consumer assert_that/Group/pair_with_0 into assert_that/Create/Read root: INFO: 2018-06-21T22:34:16.308Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:even/Group/pair_with_0 into assert:even/Create/Read root: INFO: 2018-06-21T22:34:16.331Z: JOB_MESSAGE_DETAILED: Fusing consumer assert:odd/Group/pair_with_0 into assert:odd/Create/Read root: INFO: 2018-06-21T22:34:16.368Z: JOB_MESSAGE_DEBUG: Workflow config is missing a default resource spec. root: INFO: 2018-06-21T22:34:16.420Z: JOB_MESSAGE_DEBUG: Adding StepResource setup and teardown to workflow graph. root: INFO: 2018-06-21T22:34:16.443Z: JOB_MESSAGE_DEBUG: Adding workflow start and stop steps. root: INFO: 2018-06-21T22:34:16.484Z: JOB_MESSAGE_DEBUG: Assigning stage ids. root: INFO: 2018-06-21T22:34:16.780Z: JOB_MESSAGE_DEBUG: Executing wait step start54 root: INFO: 2018-06-21T22:34:16.856Z: JOB_MESSAGE_BASIC: Executing operation assert:odd/Group/GroupByKey/Create root: INFO: 2018-06-21T22:34:16.899Z: JOB_MESSAGE_BASIC: Executing operation assert:even/Group/GroupByKey/Create root: INFO: 2018-06-21T22:34:16.912Z: JOB_MESSAGE_DEBUG: Starting worker pool setup. root: INFO: 2018-06-21T22:34:16.936Z: JOB_MESSAGE_BASIC: Executing operation assert_that/Group/GroupByKey/Create root: INFO: 2018-06-21T22:34:16.936Z: JOB_MESSAGE_BASIC: Starting 1 workers in us-central1-b... root: INFO: 2018-06-21T22:34:17.009Z: JOB_MESSAGE_DEBUG: Value "assert:odd/Group/GroupByKey/Session" materialized. root: INFO: 2018-06-21T22:34:17.041Z: JOB_MESSAGE_DEBUG: Value "assert_that/Group/GroupByKey/Session" materialized. root: INFO: 2018-06-21T22:34:17.070Z: JOB_MESSAGE_DEBUG: Value "assert:even/Group/GroupByKey/Session" materialized. root: INFO: 2018-06-21T22:34:17.104Z: JOB_MESSAGE_BASIC: Executing operation assert:odd/Create/Read+assert:odd/Group/pair_with_0+assert:odd/Group/GroupByKey/Reify+assert:odd/Group/GroupByKey/Write root: INFO: 2018-06-21T22:34:17.135Z: JOB_MESSAGE_BASIC: Executing operation assert_that/Create/Read+assert_that/Group/pair_with_0+assert_that/Group/GroupByKey/Reify+assert_that/Group/GroupByKey/Write root: INFO: 2018-06-21T22:34:17.180Z: JOB_MESSAGE_BASIC: Executing operation Some Numbers/Read+ClassifyNumbers/FlatMap(<lambda at ptransform_test.py:251>)+assert:even/WindowInto(WindowIntoFn)+assert:even/ToVoidKey+assert:even/Group/pair_with_1+assert:even/Group/GroupByKey/Reify+assert:even/Group/GroupByKey/Write+assert_that/WindowInto(WindowIntoFn)+assert_that/ToVoidKey+assert_that/Group/pair_with_1+assert_that/Group/GroupByKey/Reify+assert_that/Group/GroupByKey/Write+assert:odd/WindowInto(WindowIntoFn)+assert:odd/ToVoidKey+assert:odd/Group/pair_with_1+assert:odd/Group/GroupByKey/Reify+assert:odd/Group/GroupByKey/Write root: INFO: 2018-06-21T22:34:17.225Z: JOB_MESSAGE_BASIC: Executing operation assert:even/Create/Read+assert:even/Group/pair_with_0+assert:even/Group/GroupByKey/Reify+assert:even/Group/GroupByKey/Write root: INFO: 2018-06-21T22:34:26.039Z: JOB_MESSAGE_DETAILED: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s). root: INFO: 2018-06-21T22:35:10.989Z: JOB_MESSAGE_ERROR: Startup of the worker pool in zone us-central1-b failed to bring up any of the desired 1 workers. QUOTA_EXCEEDED: Quota 'DISKS_TOTAL_GB' exceeded. Limit: 45000.0 in region us-central1. root: INFO: 2018-06-21T22:35:11.035Z: JOB_MESSAGE_ERROR: Workflow failed. root: INFO: 2018-06-21T22:35:11.360Z: JOB_MESSAGE_DETAILED: Cleaning up. root: INFO: 2018-06-21T22:35:11.522Z: JOB_MESSAGE_DEBUG: Starting worker pool teardown. root: INFO: 2018-06-21T22:35:11.559Z: JOB_MESSAGE_BASIC: Stopping worker pool... root: INFO: 2018-06-21T22:35:20.018Z: JOB_MESSAGE_BASIC: Worker pool stopped. root: INFO: 2018-06-21T22:35:20.087Z: JOB_MESSAGE_DEBUG: Tearing down pending resources... root: INFO: Job 2018-06-21_15_34_09-13638563469120977061 is in state JOB_STATE_FAILED --------------------- >> end captured logging << --------------------- ---------------------------------------------------------------------- XML: <https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/nosetests.xml> ---------------------------------------------------------------------- Ran 16 tests in 911.271s FAILED (errors=1) Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-21_15_34_14-17319796383851608055?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-21_15_41_27-11475092675492468109?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-21_15_34_09-10510281063585880707?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-21_15_41_56-9392635919542724050?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-21_15_34_08-6238676362110880355?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-21_15_41_41-12086918879327860532?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-21_15_34_09-13638563469120977061?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-21_15_35_41-14658258186579908261?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-21_15_34_08-3317272200765023543?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-21_15_40_37-16795324053260545513?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-21_15_34_08-16203959564021268350?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-21_15_40_56-15767023983752200440?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-21_15_34_09-5174267500240232157?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-21_15_41_21-6687343149163834602?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-21_15_34_09-16837957011330199990?project=apache-beam-testing. Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-06-21_15_40_51-6306043244517598598?project=apache-beam-testing. > Task :beam-sdks-python:validatesRunnerTests FAILED :beam-sdks-python:validatesRunnerTests (Thread[Task worker for ':',5,main]) completed. Took 15 mins 12.684 secs. FAILURE: Build failed with an exception. * Where: Build file '<https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/ws/src/sdks/python/build.gradle'> line: 211 * What went wrong: Execution failed for task ':beam-sdks-python:validatesRunnerTests'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0. See https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 15m 35s 3 actionable tasks: 3 executed Publishing build scan... https://gradle.com/s/t6fw3fn3m4bi6 Build cache (/home/jenkins/.gradle/caches/build-cache-1) remove files older than Thu Jun 14 22:33:45 UTC 2018. Build cache (/home/jenkins/.gradle/caches/build-cache-1) removing 63 cache entries (12 MB reclaimed). Build cache (/home/jenkins/.gradle/caches/build-cache-1) cleaned up in 0.025 secs. Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure Not sending mail to unregistered user amaliu...@163.com