See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/158/display/redirect?page=changes>
Changes: [ajamato] [BEAM-11994] Refactor BigQueryTornadoes to make more options [zyichi] [BEAM-12221] Add experimental pipeline option to control the execution [Kenneth Knowles] Add View.toList pipeline to translator tests [Kenneth Knowles] Temporarily disable FnApiDoFnRunnerTest.testUsingMetrics [Kenneth Knowles] Resolve beam_fn_api experiment for side inputs, using portable expansion [noreply] [BEAM-12232] Expose public factory methods in SparkJobServerDriver [Kyle Weaver] [BEAM-12009] Enable Calc splitting rule by default. [noreply] [BEAM-12016] Initial implementation for add_suffix, add_prefix (#14623) ------------------------------------------ [...truncated 592.57 KB...] INFO:root:Running (((((ref_AppliedPTransform_assert_that-Create-Impulse_29)+(ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2930-_30))+(ref_AppliedPTransform_assert_that-Create-Map-decode-_32))+(ref_AppliedPTransform_assert_that-Group-pair_with_0_36))+(assert_that/Group/Flatten/Transcode/0))+(assert_that/Group/Flatten/Write/0) INFO:root:Running (assert_that/Group/Flatten/Read)+(assert_that/Group/GroupByKey/Write) INFO:root:Running (((assert_that/Group/GroupByKey/Read)+(ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_40))+(ref_AppliedPTransform_assert_that-Unkey_41))+(ref_AppliedPTransform_assert_that-Match_42) INFO:root:Successfully completed job in 14.285139322280884 seconds. INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE ok test_windowing_before_sql (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar> INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar'> '43361'] DEBUG:root:Waiting for grpc channel to be ready at localhost:43361. INFO:apache_beam.utils.subprocess_server:b'Starting expansion service at localhost:43361' DEBUG:root:Waiting for grpc channel to be ready at localhost:43361. INFO:apache_beam.utils.subprocess_server:b'Apr 28, 2021 12:40:51 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms' INFO:apache_beam.utils.subprocess_server:b'INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]' INFO:apache_beam.utils.subprocess_server:b'\tbeam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5680a178' INFO:apache_beam.utils.subprocess_server:b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5fdef03a' DEBUG:root:Waiting for grpc channel to be ready at localhost:43361. DEBUG:root:Waiting for grpc channel to be ready at localhost:43361. DEBUG:root:Waiting for grpc channel to be ready at localhost:43361. DEBUG:root:Waiting for grpc channel to be ready at localhost:43361. INFO:apache_beam.utils.subprocess_server:b'Apr 28, 2021 12:40:52 AM org.apache.beam.sdk.expansion.service.ExpansionService expand' INFO:apache_beam.utils.subprocess_server:b"INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'" INFO:apache_beam.utils.subprocess_server:b'Apr 28, 2021 12:40:53 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig' INFO:apache_beam.utils.subprocess_server:b"WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach." INFO:apache_beam.utils.subprocess_server:b'Apr 28, 2021 12:40:55 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel' INFO:apache_beam.utils.subprocess_server:b'INFO: SQL:' INFO:apache_beam.utils.subprocess_server:b'SELECT COUNT(*) AS `count`' INFO:apache_beam.utils.subprocess_server:b'FROM `beam`.`PCOLLECTION` AS `PCOLLECTION`' INFO:apache_beam.utils.subprocess_server:b'Apr 28, 2021 12:40:55 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel' INFO:apache_beam.utils.subprocess_server:b'INFO: SQLPlan>' INFO:apache_beam.utils.subprocess_server:b'LogicalAggregate(group=[{}], count=[COUNT()])' INFO:apache_beam.utils.subprocess_server:b' LogicalProject($f0=[0])' INFO:apache_beam.utils.subprocess_server:b' BeamIOSourceRel(table=[[beam, PCOLLECTION]])' INFO:apache_beam.utils.subprocess_server:b'' INFO:apache_beam.utils.subprocess_server:b'Apr 28, 2021 12:40:55 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel' INFO:apache_beam.utils.subprocess_server:b'INFO: BEAMPlan>' INFO:apache_beam.utils.subprocess_server:b'BeamAggregationRel(group=[{}], count=[COUNT()])' INFO:apache_beam.utils.subprocess_server:b' BeamIOSourceRel(table=[[beam, PCOLLECTION]])' INFO:apache_beam.utils.subprocess_server:b'' DEBUG:root:Sending SIGINT to job_server DEBUG:root:Unhandled type_constraint: Union[] DEBUG:root:Unhandled type_constraint: Union[] DEBUG:root:Unhandled type_constraint: Union[] DEBUG:root:Unhandled type_constraint: Union[] WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter. INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev INFO:root:No image given, using default Python SDK image WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter. INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev INFO:root:Python SDK container image set to "apache/beam_python3.6_sdk:2.30.0.dev" for Docker environment INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f08e22c4a60> ==================== DEBUG:apache_beam.runners.portability.fn_api_runner.translations:27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create-Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-2930-_4\n Create/FlatMap(<lambda at core.py:2930>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-Map-decode-_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map-lambda-at-sql_test-py-171-_14\n Map(<lambda at sql_test.py:171>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto-WindowIntoFn-_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-sele\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-Grou\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-Combine-Pa\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToRow-ParM\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-mergeRecord-ParMultiDo-Anonymous-\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2930-_20\n assert_that/Create/FlatMap(<lambda at core.py:2930>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>'] INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f08e22c51e0> ==================== DEBUG:apache_beam.runners.portability.fn_api_runner.translations:27 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['ref_AppliedPTransform_Create-Impulse_3\n Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-2930-_4\n Create/FlatMap(<lambda at core.py:2930>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-AddRandomKeys_7\n Create/MaybeReshuffle/Reshuffle/AddRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_9\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/Map(reify_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-GroupByKey_10\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_11\n Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/FlatMap(restore_timestamps):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-RemoveRandomKeys_12\n Create/MaybeReshuffle/Reshuffle/RemoveRandomKeys:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Create-Map-decode-_13\n Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_Map-lambda-at-sql_test-py-171-_14\n Map(<lambda at sql_test.py:171>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_WindowInto-WindowIntoFn-_15\n WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-sele\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/selectKeys/AddKeys/Map/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-Grou\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-Combine-Pa\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/Combine/ParDo(Anonymous)/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToRow-ParM\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToRow/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-mergeRecord-ParMultiDo-Anonymous-\n SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/mergeRecord/ParMultiDo(Anonymous):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_19\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2930-_20\n assert_that/Create/FlatMap(<lambda at core.py:2930>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_22\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_23\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_24\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_26\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_27\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_28\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_29\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_30\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_31\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_32\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>'] INFO:apache_beam.runners.portability.portable_runner:Job state changed to STOPPED INFO:apache_beam.runners.portability.portable_runner:Job state changed to STARTING INFO:apache_beam.runners.portability.portable_runner:Job state changed to RUNNING INFO:root:==================== <function annotate_downstream_side_inputs at 0x7fb38a605158> ==================== INFO:root:==================== <function fix_side_input_pcoll_coders at 0x7fb38a605268> ==================== INFO:root:==================== <function pack_combiners at 0x7fb38a6056a8> ==================== INFO:root:==================== <function lift_combiners at 0x7fb38a6057b8> ==================== INFO:root:==================== <function expand_sdf at 0x7fb38a605950> ==================== INFO:root:==================== <function expand_gbk at 0x7fb38a6059d8> ==================== INFO:root:==================== <function sink_flattens at 0x7fb38a605ae8> ==================== INFO:root:==================== <function greedily_fuse at 0x7fb38a605b70> ==================== INFO:root:==================== <function read_to_impulse at 0x7fb38a605bf8> ==================== INFO:root:==================== <function impulse_to_input at 0x7fb38a605c80> ==================== INFO:root:==================== <function sort_stages at 0x7fb38a605ea0> ==================== INFO:root:==================== <function setup_timer_mapping at 0x7fb38a605e18> ==================== INFO:root:==================== <function populate_data_channel_coders at 0x7fb38a605f28> ==================== INFO:root:starting control server on port 37981 INFO:root:starting data server on port 37025 INFO:root:starting state server on port 32993 INFO:root:starting logging server on port 38711 INFO:root:Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.DockerSdkWorkerHandler object at 0x7fb389df22e8> for environment ref_Environment_default_environment_2 (beam:env:docker:v1, b'\n$apache/beam_python3.6_sdk:2.30.0.dev') INFO:root:Attempting to pull image apache/beam_python3.6_sdk:2.30.0.dev INFO:root:Unable to pull image apache/beam_python3.6_sdk:2.30.0.dev, defaulting to local image if it exists INFO:root:Waiting for docker to start up. Current status is running INFO:root:Docker container is running. container_id = b'd7a644c99596faa36f40a1f0073c5de76307cd62a54cc010815903aac65cc902', worker_id = worker_166 INFO:root:Running (((((ref_AppliedPTransform_assert_that-Create-Impulse_19)+(ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2930-_20))+(ref_AppliedPTransform_assert_that-Create-Map-decode-_22))+(ref_AppliedPTransform_assert_that-Group-pair_with_0_26))+(assert_that/Group/Flatten/Transcode/0))+(assert_that/Group/Flatten/Write/0) INFO:root:Running ((((ref_AppliedPTransform_Create-Impulse_3)+(ref_AppliedPTransform_Create-FlatMap-lambda-at-core-py-2930-_4))+(ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-AddRandomKeys_7))+(ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-Map-reify_timestamps-_9))+(Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Write) INFO:root:Running ((((((Create/MaybeReshuffle/Reshuffle/ReshufflePerKey/GroupByKey/Read)+(ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-ReshufflePerKey-FlatMap-restore_timestamps-_11))+(ref_AppliedPTransform_Create-MaybeReshuffle-Reshuffle-RemoveRandomKeys_12))+(ref_AppliedPTransform_Create-Map-decode-_13))+(ref_AppliedPTransform_Map-lambda-at-sql_test-py-171-_14))+(ref_AppliedPTransform_WindowInto-WindowIntoFn-_15))+(ref_PCollection_PCollection_1/Write) INFO:root:Created Worker handler <apache_beam.runners.portability.fn_api_runner.worker_handlers.DockerSdkWorkerHandler object at 0x7fb30cf57390> for environment external_8beam:env:docker:v1 (beam:env:docker:v1, b'\n apache/beam_java8_sdk:2.30.0.dev') INFO:root:Attempting to pull image apache/beam_java8_sdk:2.30.0.dev INFO:root:Unable to pull image apache/beam_java8_sdk:2.30.0.dev, defaulting to local image if it exists INFO:root:Waiting for docker to start up. Current status is running INFO:root:Docker container is running. container_id = b'780453683c410f567f6e0484067f558cf1fd6135d7102a7a2a0644037fe20920', worker_id = worker_167 INFO:root:Running ((ref_PCollection_PCollection_1/Read)+(external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToKvs-sele))+(SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey/Write) INFO:root:Running ((((SqlTransform(beam:external:java:sql:v1)/BeamAggregationRel_40/Group.CombineFieldsByFields/ToKvs/GroupByKey/Read)+(external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-Combine-Pa))+(external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-Group-CombineFieldsByFields-ToRow-ParM))+(external_8SqlTransform-beam-external-java-sql-v1--BeamAggregationRel_40-mergeRecord-ParMultiDo-Anonymous-))+(ref_PCollection_PCollection_11/Write) INFO:root:Running (((((ref_PCollection_PCollection_11/Read)+(ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_23))+(ref_AppliedPTransform_assert_that-ToVoidKey_24))+(ref_AppliedPTransform_assert_that-Group-pair_with_1_27))+(assert_that/Group/Flatten/Transcode/1))+(assert_that/Group/Flatten/Write/1) INFO:root:Running (assert_that/Group/Flatten/Read)+(assert_that/Group/GroupByKey/Write) INFO:root:Running (((assert_that/Group/GroupByKey/Read)+(ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_30))+(ref_AppliedPTransform_assert_that-Unkey_31))+(ref_AppliedPTransform_assert_that-Match_32) INFO:root:Successfully completed job in 9.688555479049683 seconds. INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE ok test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ... INFO:apache_beam.utils.subprocess_server:Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar> INFO:apache_beam.utils.subprocess_server:Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar'> '39091'] DEBUG:root:Waiting for grpc channel to be ready at localhost:39091. INFO:apache_beam.utils.subprocess_server:b'Starting expansion service at localhost:39091' DEBUG:root:Waiting for grpc channel to be ready at localhost:39091. INFO:apache_beam.utils.subprocess_server:b'Apr 28, 2021 12:41:10 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms' INFO:apache_beam.utils.subprocess_server:b'INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]' INFO:apache_beam.utils.subprocess_server:b'\tbeam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5680a178' INFO:apache_beam.utils.subprocess_server:b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5fdef03a' DEBUG:root:Waiting for grpc channel to be ready at localhost:39091. DEBUG:root:Waiting for grpc channel to be ready at localhost:39091. DEBUG:root:Waiting for grpc channel to be ready at localhost:39091. DEBUG:root:Waiting for grpc channel to be ready at localhost:39091. INFO:apache_beam.utils.subprocess_server:b'Apr 28, 2021 12:41:11 AM org.apache.beam.sdk.expansion.service.ExpansionService expand' INFO:apache_beam.utils.subprocess_server:b"INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'" INFO:apache_beam.utils.subprocess_server:b'Apr 28, 2021 12:41:12 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig' INFO:apache_beam.utils.subprocess_server:b"WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach." INFO:apache_beam.utils.subprocess_server:b'Apr 28, 2021 12:41:15 AM org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLQueryPlanner convertToBeamRelInternal' INFO:apache_beam.utils.subprocess_server:b'INFO: BEAMPlan>' INFO:apache_beam.utils.subprocess_server:b"BeamZetaSqlCalcRel(expr#0=[{inputs}], expr#1=[1:BIGINT], expr#2=['foo':VARCHAR], expr#3=[3.1400000000000001243E0:DOUBLE], int=[$t1], str=[$t2], flt=[$t3])" INFO:apache_beam.utils.subprocess_server:b' BeamValuesRel(tuples=[[{ 0 }]])' INFO:apache_beam.utils.subprocess_server:b'' DEBUG:root:Sending SIGINT to job_server DEBUG:root:Unhandled type_constraint: Union[] DEBUG:root:Unhandled type_constraint: Union[] DEBUG:root:Unhandled type_constraint: Union[] DEBUG:root:Unhandled type_constraint: Union[] WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter. INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev INFO:root:No image given, using default Python SDK image WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter. INFO:root:Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev INFO:root:Python SDK container image set to "apache/beam_python3.6_sdk:2.30.0.dev" for Docker environment INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function lift_combiners at 0x7f08e22c4a60> ==================== DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2930-_6\n assert_that/Create/FlatMap(<lambda at core.py:2930>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>'] INFO:apache_beam.runners.portability.fn_api_runner.translations:==================== <function sort_stages at 0x7f08e22c51e0> ==================== DEBUG:apache_beam.runners.portability.fn_api_runner.translations:16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] DEBUG:apache_beam.runners.portability.fn_api_runner.translations:Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2930-_6\n assert_that/Create/FlatMap(<lambda at core.py:2930>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>'] ERROR ====================================================================== ERROR: test_zetasql_generate_data (apache_beam.transforms.sql_test.SqlTransformTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/transforms/sql_test.py";,> line 161, in test_zetasql_generate_data assert_that(out, equal_to([(1, "foo", 3.14)])) File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/pipeline.py";,> line 582, in __exit__ self.result = self.run() File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/testing/test_pipeline.py";,> line 114, in run False if self.not_use_test_runner_api else test_runner_api)) File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/pipeline.py";,> line 561, in run return self.runner.run_pipeline(self, self._options) File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py";,> line 439, in run_pipeline job_service_handle.submit(proto_pipeline) File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py";,> line 107, in submit prepare_response = self.prepare(proto_pipeline) File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py";,> line 201, in prepare pipeline_options=self.get_pipeline_options()), File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py";,> line 144, in get_pipeline_options options_response = send_options_request() File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py";,> line 141, in send_options_request raise e File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/runners/portability/portable_runner.py";,> line 134, in send_options_request timeout=self.timeout) File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/build/gradleenv/1922375555/lib/python3.6/site-packages/grpc/_channel.py";,> line 946, in __call__ return _end_unary_response_blocking(state, call, False, None) File "<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/build/gradleenv/1922375555/lib/python3.6/site-packages/grpc/_channel.py";,> line 849, in _end_unary_response_blocking raise _InactiveRpcError(state) grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses" debug_error_string = "{"created":"@1619570485.144340017","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":5419,"referenced_errors":[{"created":"@1619570483.141879539","description":"failed to connect to all addresses","file":"src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc","file_line":397,"grpc_status":14}]}" > -------------------- >> begin captured logging << -------------------- apache_beam.utils.subprocess_server: INFO: Using pre-built snapshot at <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar> apache_beam.utils.subprocess_server: INFO: Starting service with ['java' '-jar' '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.30.0-SNAPSHOT.jar'> '39091'] root: DEBUG: Waiting for grpc channel to be ready at localhost:39091. apache_beam.utils.subprocess_server: INFO: b'Starting expansion service at localhost:39091' root: DEBUG: Waiting for grpc channel to be ready at localhost:39091. apache_beam.utils.subprocess_server: INFO: b'Apr 28, 2021 12:41:10 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms' apache_beam.utils.subprocess_server: INFO: b'INFO: Registering external transforms: [beam:external:java:sql:v1, beam:external:java:generate_sequence:v1]' apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:sql:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5680a178' apache_beam.utils.subprocess_server: INFO: b'\tbeam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$3/1130478920@5fdef03a' root: DEBUG: Waiting for grpc channel to be ready at localhost:39091. root: DEBUG: Waiting for grpc channel to be ready at localhost:39091. root: DEBUG: Waiting for grpc channel to be ready at localhost:39091. root: DEBUG: Waiting for grpc channel to be ready at localhost:39091. apache_beam.utils.subprocess_server: INFO: b'Apr 28, 2021 12:41:11 AM org.apache.beam.sdk.expansion.service.ExpansionService expand' apache_beam.utils.subprocess_server: INFO: b"INFO: Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN 'beam:external:java:sql:v1'" apache_beam.utils.subprocess_server: INFO: b'Apr 28, 2021 12:41:12 AM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig' apache_beam.utils.subprocess_server: INFO: b"WARNING: Configuration class 'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration' has no schema registered. Attempting to construct with setter approach." apache_beam.utils.subprocess_server: INFO: b'Apr 28, 2021 12:41:15 AM org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLQueryPlanner convertToBeamRelInternal' apache_beam.utils.subprocess_server: INFO: b'INFO: BEAMPlan>' apache_beam.utils.subprocess_server: INFO: b"BeamZetaSqlCalcRel(expr#0=[{inputs}], expr#1=[1:BIGINT], expr#2=['foo':VARCHAR], expr#3=[3.1400000000000001243E0:DOUBLE], int=[$t1], str=[$t2], flt=[$t3])" apache_beam.utils.subprocess_server: INFO: b' BeamValuesRel(tuples=[[{ 0 }]])' apache_beam.utils.subprocess_server: INFO: b'' root: DEBUG: Sending SIGINT to job_server root: DEBUG: Unhandled type_constraint: Union[] root: DEBUG: Unhandled type_constraint: Union[] root: DEBUG: Unhandled type_constraint: Union[] root: DEBUG: Unhandled type_constraint: Union[] root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter. root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev root: INFO: No image given, using default Python SDK image root: WARNING: Make sure that locally built Python SDK docker image has Python 3.6 interpreter. root: INFO: Default Python SDK image for environment is apache/beam_python3.6_sdk:2.30.0.dev root: INFO: Python SDK container image set to "apache/beam_python3.6_sdk:2.30.0.dev" for Docker environment apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function lift_combiners at 0x7f08e22c4a60> ==================== apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2930-_6\n assert_that/Create/FlatMap(<lambda at core.py:2930>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>'] apache_beam.runners.portability.fn_api_runner.translations: INFO: ==================== <function sort_stages at 0x7f08e22c51e0> ==================== apache_beam.runners.portability.fn_api_runner.translations: DEBUG: 16 [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1] apache_beam.runners.portability.fn_api_runner.translations: DEBUG: Stages: ['external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--Impulse\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Outp\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(OutputSingleSource)/ParMultiDo(OutputSingleSource):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamValuesRel_13-Create-Values-Read-CreateSource--ParDo-Boun\n SqlTransform(beam:external:java:sql:v1)/BeamValuesRel_13/Create.Values/Read(CreateSource)/ParDo(BoundedSourceAsSDFWrapper)/ParMultiDo(BoundedSourceAsSDFWrapper):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'external_9SqlTransform-beam-external-java-sql-v1--BeamZetaSqlCalcRel_17-ParDo-Calc--ParMultiDo-Calc-\n SqlTransform(beam:external:java:sql:v1)/BeamZetaSqlCalcRel_17/ParDo(Calc)/ParMultiDo(Calc):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Impulse_5\n assert_that/Create/Impulse:beam:transform:impulse:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-FlatMap-lambda-at-core-py-2930-_6\n assert_that/Create/FlatMap(<lambda at core.py:2930>):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Create-Map-decode-_8\n assert_that/Create/Map(decode):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-WindowInto-WindowIntoFn-_9\n assert_that/WindowInto(WindowIntoFn):beam:transform:window_into:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-ToVoidKey_10\n assert_that/ToVoidKey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_0_12\n assert_that/Group/pair_with_0:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-pair_with_1_13\n assert_that/Group/pair_with_1:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Flatten_14\n assert_that/Group/Flatten:beam:transform:flatten:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-GroupByKey_15\n assert_that/Group/GroupByKey:beam:transform:group_by_key:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Group-Map-_merge_tagged_vals_under_key-_16\n assert_that/Group/Map(_merge_tagged_vals_under_key):beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Unkey_17\n assert_that/Unkey:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>', 'ref_AppliedPTransform_assert_that-Match_18\n assert_that/Match:beam:transform:pardo:v1\n must follow: \n downstream_side_inputs: <unknown>'] --------------------- >> end captured logging << --------------------- ---------------------------------------------------------------------- XML: nosetests-xlangSqlValidateRunner.xml ---------------------------------------------------------------------- XML: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/nosetests.xml> ---------------------------------------------------------------------- Ran 9 tests in 187.578s FAILED (errors=1) > Task :runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql > FAILED > Task :runners:spark:3:job-server:validatesCrossLanguageRunnerCleanup > Task :runners:spark:3:job-server:sparkJobServerCleanup FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 40m 56s 180 actionable tasks: 141 executed, 35 from cache, 4 up-to-date Gradle was unable to watch the file system for changes. The inotify watches limit is too low. Publishing build scan... https://gradle.com/s/a3rocxrzkgfk4 Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
