See
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/3467/display/redirect?page=changes>
Changes:
[noreply] Update nbconvert requirement in /sdks/python
[noreply] add clarifier to error message (#24449)
[noreply] Batch rename requests in fileio.WriteToFiles (#24341)
[noreply] Bump golang.org/x/text from 0.4.0 to 0.5.0 in /sdks (#24520)
[noreply] Support for JsonSchema in Kafka Read Schema Transform (#24272)
[noreply] Run go fmt over full go directory with go 1.19 (#24525)
[noreply] Cloudbuild+manualsetup+playground (#24144)
[noreply] Bump golang.org/x/sys from 0.2.0 to 0.3.0 in /sdks (#24519)
[noreply] Bump cloud.google.com/go/bigtable from 1.18.0 to 1.18.1 in /sdks
[noreply] Update from interface{} -> any for core packages (#24505)
[noreply] Implement FileWriteSchemaTransformConfiguration (#24479)
------------------------------------------
[...truncated 2.45 MB...]
INFO root:portable_runner.py:574 Unable to pull image
apache/beam_java8_sdk:2.45.0.dev, defaulting to local image if it exists
INFO root:portable_runner.py:574 Waiting for docker to start up. Current
status is running
INFO root:portable_runner.py:574 Docker container is running. container_id
= b'967e0db62ab4ee1eb9cf25042e067e0c8731a6b1497e9563063d618fc4ad108e',
worker_id = worker_84
INFO root:portable_runner.py:574 Completed job in 17.306925058364868
seconds with state DONE.
INFO apache_beam.runners.portability.portable_runner:portable_runner.py:581
Job state changed to DONE
PASSED [ 77%]
apache_beam/transforms/sql_test.py::SqlTransformTest::test_windowing_before_sql
-------------------------------- live log call ---------------------------------
INFO apache_beam.utils.subprocess_server:subprocess_server.py:239 Using
pre-built snapshot at
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.45.0-SNAPSHOT.jar>
INFO root:external.py:847 Starting a JAR-based expansion service from JAR
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.45.0-SNAPSHOT.jar>
INFO apache_beam.utils.subprocess_server:subprocess_server.py:116 Starting
service with ['java' '-jar'
'<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.45.0-SNAPSHOT.jar'>
'46363'
'--filesToStage=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.45.0-SNAPSHOT.jar']>
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Starting
expansion service at localhost:46363
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Dec 06,
2022 12:30:39 AM org.apache.beam.sdk.expansion.service.ExpansionService
loadRegisteredTransforms
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO:
Registering external transforms: [beam:external:java:sql:v1,
beam:transform:org.apache.beam:pubsub_read:v1,
beam:transform:org.apache.beam:pubsub_write:v1,
beam:transform:org.apache.beam:pubsublite_write:v1,
beam:transform:org.apache.beam:pubsublite_read:v1,
beam:transform:org.apache.beam:spanner_insert:v1,
beam:transform:org.apache.beam:spanner_update:v1,
beam:transform:org.apache.beam:spanner_replace:v1,
beam:transform:org.apache.beam:spanner_insert_or_update:v1,
beam:transform:org.apache.beam:spanner_delete:v1,
beam:transform:org.apache.beam:spanner_read:v1,
beam:transform:org.apache.beam:kafka_read_with_metadata:v1,
beam:transform:org.apache.beam:kafka_read_without_metadata:v1,
beam:transform:org.apache.beam:kafka_write:v1,
beam:external:java:generate_sequence:v1]
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
Registered transforms:
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:external:java:sql:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@38425407
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:pubsub_read:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@43bc63a3
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:pubsub_write:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@702657cc
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:pubsublite_write:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@6a6cb05c
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:pubsublite_read:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@40a4337a
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:spanner_insert:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@6025e1b6
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:spanner_update:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@22ff4249
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:spanner_replace:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@2d1ef81a
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:spanner_insert_or_update:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@4c402120
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:spanner_delete:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@327514f
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:spanner_read:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@5b12b668
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:kafka_read_with_metadata:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@1165b38
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:kafka_read_without_metadata:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@4c12331b
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:kafka_write:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@7586beff
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:external:java:generate_sequence:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@3b69e7d1
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
Registered SchemaTransformProviders:
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:schematransform:org.apache.beam:kafka_read:v1
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:schematransform:org.apache.beam:spanner_write:v1
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:bigquery_storage_read:v1
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Dec 06,
2022 12:30:40 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO:
Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN
'beam:external:java:sql:v1'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Dec 06,
2022 12:30:40 AM
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader
payloadToConfig
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 WARNING:
Configuration class
'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration'
has no schema registered. Attempting to construct with setter approach.
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Dec 06,
2022 12:30:41 AM
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader
payloadToConfig
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 WARNING:
Configuration class
'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration'
has no schema registered. Attempting to construct with setter approach.
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Dec 06,
2022 12:30:42 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner
convertToBeamRel
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO: SQL:
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 SELECT
COUNT(*) AS `count`
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 FROM
`beam`.`PCOLLECTION` AS `PCOLLECTION`
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Dec 06,
2022 12:30:43 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner
convertToBeamRel
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO:
SQLPlan>
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
LogicalAggregate(group=[{}], count=[COUNT()])
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Dec 06,
2022 12:30:43 AM org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner
convertToBeamRel
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO:
BEAMPlan>
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
BeamAggregationRel(group=[{}], count=[COUNT()])
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
BeamIOSourceRel(table=[[beam, PCOLLECTION]])
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
INFO root:environments.py:376 Default Python SDK image for environment is
apache/beam_python3.10_sdk:2.45.0.dev
INFO root:environments.py:299 No image given, using default Python SDK image
INFO root:environments.py:376 Default Python SDK image for environment is
apache/beam_python3.10_sdk:2.45.0.dev
INFO root:environments.py:302 Python SDK container image set to
"apache/beam_python3.10_sdk:2.45.0.dev" for Docker environment
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function pack_combiners at 0x7f06ea926c20>
====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function lift_combiners at 0x7f06ea926cb0>
====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function sort_stages at 0x7f06ea927400>
====================
INFO apache_beam.runners.portability.portable_runner:portable_runner.py:581
Job state changed to STOPPED
INFO apache_beam.runners.portability.portable_runner:portable_runner.py:581
Job state changed to STARTING
INFO apache_beam.runners.portability.portable_runner:portable_runner.py:581
Job state changed to RUNNING
INFO root:portable_runner.py:574 Default Python SDK image for environment
is apache/beam_python3.10_sdk:2.45.0.dev
INFO root:portable_runner.py:574 ==================== <function
annotate_downstream_side_inputs at 0x7fa953d0f760> ====================
INFO root:portable_runner.py:574 ==================== <function
fix_side_input_pcoll_coders at 0x7fa953d0f880> ====================
INFO root:portable_runner.py:574 ==================== <function
pack_combiners at 0x7fa953d0fd90> ====================
INFO root:portable_runner.py:574 ==================== <function
lift_combiners at 0x7fa953d0fe20> ====================
INFO root:portable_runner.py:574 ==================== <function expand_sdf
at 0x7fa953d14040> ====================
INFO root:portable_runner.py:574 ==================== <function expand_gbk
at 0x7fa953d140d0> ====================
INFO root:portable_runner.py:574 ==================== <function
sink_flattens at 0x7fa953d141f0> ====================
INFO root:portable_runner.py:574 ==================== <function
greedily_fuse at 0x7fa953d14280> ====================
INFO root:portable_runner.py:574 ==================== <function
read_to_impulse at 0x7fa953d14310> ====================
INFO root:portable_runner.py:574 ==================== <function
impulse_to_input at 0x7fa953d143a0> ====================
INFO root:portable_runner.py:574 ==================== <function sort_stages
at 0x7fa953d145e0> ====================
INFO root:portable_runner.py:574 ==================== <function
add_impulse_to_dangling_transforms at 0x7fa953d14700> ====================
INFO root:portable_runner.py:574 ==================== <function
setup_timer_mapping at 0x7fa953d14550> ====================
INFO root:portable_runner.py:574 ==================== <function
populate_data_channel_coders at 0x7fa953d14670> ====================
INFO root:portable_runner.py:574 starting control server on port 37715
INFO root:portable_runner.py:574 starting data server on port 44677
INFO root:portable_runner.py:574 starting state server on port 35285
INFO root:portable_runner.py:574 starting logging server on port 42915
INFO root:portable_runner.py:574 Created Worker handler
<apache_beam.runners.portability.fn_api_runner.worker_handlers.DockerSdkWorkerHandler
object at 0x7fa9502c0bb0> for environment
ref_Environment_default_environment_2 (beam:env:docker:v1,
b'\n%apache/beam_python3.10_sdk:2.45.0.dev')
INFO root:portable_runner.py:574 Attempting to pull image
apache/beam_python3.10_sdk:2.45.0.dev
INFO root:portable_runner.py:574 Unable to pull image
apache/beam_python3.10_sdk:2.45.0.dev, defaulting to local image if it exists
INFO root:portable_runner.py:574 Waiting for docker to start up. Current
status is running
INFO root:portable_runner.py:574 Docker container is running. container_id
= b'39de4f5a603a1b32a1791df990c6bd2c444eab30afbf4c4f0408e8013ce4dd49',
worker_id = worker_85
INFO root:portable_runner.py:574 Created Worker handler
<apache_beam.runners.portability.fn_api_runner.worker_handlers.DockerSdkWorkerHandler
object at 0x7fa9502ecc10> for environment external_8beam:env:docker:v1
(beam:env:docker:v1, b'\n apache/beam_java8_sdk:2.45.0.dev')
INFO root:portable_runner.py:574 Attempting to pull image
apache/beam_java8_sdk:2.45.0.dev
INFO root:portable_runner.py:574 Unable to pull image
apache/beam_java8_sdk:2.45.0.dev, defaulting to local image if it exists
INFO root:portable_runner.py:574 Waiting for docker to start up. Current
status is running
INFO root:portable_runner.py:574 Docker container is running. container_id
= b'fd2d7c023db50dd4b57e15e56ce78653f5314bb36562bf671a9f712676471397',
worker_id = worker_86
INFO root:portable_runner.py:574 Completed job in 15.659727573394775
seconds with state DONE.
INFO apache_beam.runners.portability.portable_runner:portable_runner.py:581
Job state changed to DONE
PASSED [ 88%]
apache_beam/transforms/sql_test.py::SqlTransformTest::test_zetasql_generate_data
-------------------------------- live log call ---------------------------------
INFO apache_beam.utils.subprocess_server:subprocess_server.py:239 Using
pre-built snapshot at
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.45.0-SNAPSHOT.jar>
INFO root:external.py:847 Starting a JAR-based expansion service from JAR
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.45.0-SNAPSHOT.jar>
INFO apache_beam.utils.subprocess_server:subprocess_server.py:116 Starting
service with ['java' '-jar'
'<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.45.0-SNAPSHOT.jar'>
'42883'
'--filesToStage=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/java/extensions/sql/expansion-service/build/libs/beam-sdks-java-extensions-sql-expansion-service-2.45.0-SNAPSHOT.jar']>
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Starting
expansion service at localhost:42883
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Dec 06,
2022 12:31:08 AM org.apache.beam.sdk.expansion.service.ExpansionService
loadRegisteredTransforms
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO:
Registering external transforms: [beam:external:java:sql:v1,
beam:transform:org.apache.beam:pubsub_read:v1,
beam:transform:org.apache.beam:pubsub_write:v1,
beam:transform:org.apache.beam:pubsublite_write:v1,
beam:transform:org.apache.beam:pubsublite_read:v1,
beam:transform:org.apache.beam:spanner_insert:v1,
beam:transform:org.apache.beam:spanner_update:v1,
beam:transform:org.apache.beam:spanner_replace:v1,
beam:transform:org.apache.beam:spanner_insert_or_update:v1,
beam:transform:org.apache.beam:spanner_delete:v1,
beam:transform:org.apache.beam:spanner_read:v1,
beam:transform:org.apache.beam:kafka_read_with_metadata:v1,
beam:transform:org.apache.beam:kafka_read_without_metadata:v1,
beam:transform:org.apache.beam:kafka_write:v1,
beam:external:java:generate_sequence:v1]
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
Registered transforms:
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:external:java:sql:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@38425407
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:pubsub_read:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@43bc63a3
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:pubsub_write:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@702657cc
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:pubsublite_write:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@6a6cb05c
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:pubsublite_read:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@40a4337a
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:spanner_insert:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@6025e1b6
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:spanner_update:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@22ff4249
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:spanner_replace:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@2d1ef81a
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:spanner_insert_or_update:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@4c402120
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:spanner_delete:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@327514f
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:spanner_read:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@5b12b668
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:kafka_read_with_metadata:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@1165b38
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:kafka_read_without_metadata:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@4c12331b
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:kafka_write:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@7586beff
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:external:java:generate_sequence:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@3b69e7d1
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
Registered SchemaTransformProviders:
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:schematransform:org.apache.beam:kafka_read:v1
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:schematransform:org.apache.beam:spanner_write:v1
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
beam:transform:org.apache.beam:bigquery_storage_read:v1
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Dec 06,
2022 12:31:09 AM org.apache.beam.sdk.expansion.service.ExpansionService expand
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO:
Expanding 'SqlTransform(beam:external:java:sql:v1)' with URN
'beam:external:java:sql:v1'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Dec 06,
2022 12:31:09 AM
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader
payloadToConfig
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 WARNING:
Configuration class
'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration'
has no schema registered. Attempting to construct with setter approach.
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Dec 06,
2022 12:31:09 AM
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader
payloadToConfig
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 WARNING:
Configuration class
'org.apache.beam.sdk.extensions.sql.expansion.ExternalSqlTransformRegistrar$Configuration'
has no schema registered. Attempting to construct with setter approach.
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Dec 06,
2022 12:31:11 AM com.google.zetasql.io.grpc.NameResolverRegistry
getDefaultRegistry
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 WARNING:
No NameResolverProviders found via ServiceLoader, including for DNS. This is
probably due to a broken build. If using ProGuard, check your configuration
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 Dec 06,
2022 12:31:13 AM org.apache.beam.sdk.extensions.sql.zetasql.ZetaSQLQueryPlanner
convertToBeamRelInternal
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126 INFO:
BEAMPlan>
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
BeamZetaSqlCalcRel(expr#0=[{inputs}], expr#1=[1:BIGINT],
expr#2=['foo':VARCHAR], expr#3=[3.1400000000000001243E0:DOUBLE], int=[$t1],
str=[$t2], flt=[$t3])
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
BeamValuesRel(tuples=[[{ 0 }]])
INFO apache_beam.utils.subprocess_server:subprocess_server.py:126
INFO root:environments.py:376 Default Python SDK image for environment is
apache/beam_python3.10_sdk:2.45.0.dev
INFO root:environments.py:299 No image given, using default Python SDK image
INFO root:environments.py:376 Default Python SDK image for environment is
apache/beam_python3.10_sdk:2.45.0.dev
INFO root:environments.py:302 Python SDK container image set to
"apache/beam_python3.10_sdk:2.45.0.dev" for Docker environment
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function pack_combiners at 0x7f06ea926c20>
====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function lift_combiners at 0x7f06ea926cb0>
====================
INFO
apache_beam.runners.portability.fn_api_runner.translations:translations.py:710
==================== <function sort_stages at 0x7f06ea927400>
====================
INFO apache_beam.runners.portability.portable_runner:portable_runner.py:581
Job state changed to STOPPED
INFO apache_beam.runners.portability.portable_runner:portable_runner.py:581
Job state changed to STARTING
INFO apache_beam.runners.portability.portable_runner:portable_runner.py:581
Job state changed to RUNNING
INFO root:portable_runner.py:574 Default Python SDK image for environment
is apache/beam_python3.10_sdk:2.45.0.dev
INFO root:portable_runner.py:574 ==================== <function
annotate_downstream_side_inputs at 0x7fa953d0f760> ====================
INFO root:portable_runner.py:574 ==================== <function
fix_side_input_pcoll_coders at 0x7fa953d0f880> ====================
INFO root:portable_runner.py:574 ==================== <function
pack_combiners at 0x7fa953d0fd90> ====================
INFO root:portable_runner.py:574 ==================== <function
lift_combiners at 0x7fa953d0fe20> ====================
INFO root:portable_runner.py:574 ==================== <function expand_sdf
at 0x7fa953d14040> ====================
INFO root:portable_runner.py:574 ==================== <function expand_gbk
at 0x7fa953d140d0> ====================
INFO root:portable_runner.py:574 ==================== <function
sink_flattens at 0x7fa953d141f0> ====================
INFO root:portable_runner.py:574 ==================== <function
greedily_fuse at 0x7fa953d14280> ====================
INFO root:portable_runner.py:574 ==================== <function
read_to_impulse at 0x7fa953d14310> ====================
INFO root:portable_runner.py:574 ==================== <function
impulse_to_input at 0x7fa953d143a0> ====================
INFO root:portable_runner.py:574 ==================== <function sort_stages
at 0x7fa953d145e0> ====================
INFO root:portable_runner.py:574 ==================== <function
add_impulse_to_dangling_transforms at 0x7fa953d14700> ====================
INFO root:portable_runner.py:574 ==================== <function
setup_timer_mapping at 0x7fa953d14550> ====================
INFO root:portable_runner.py:574 ==================== <function
populate_data_channel_coders at 0x7fa953d14670> ====================
INFO root:portable_runner.py:574 starting control server on port 34051
INFO root:portable_runner.py:574 starting data server on port 37325
INFO root:portable_runner.py:574 starting state server on port 43109
INFO root:portable_runner.py:574 starting logging server on port 44861
INFO root:portable_runner.py:574 Created Worker handler
<apache_beam.runners.portability.fn_api_runner.worker_handlers.DockerSdkWorkerHandler
object at 0x7fa9501fea40> for environment external_9beam:env:docker:v1
(beam:env:docker:v1, b'\n apache/beam_java8_sdk:2.45.0.dev')
INFO root:portable_runner.py:574 Attempting to pull image
apache/beam_java8_sdk:2.45.0.dev
INFO root:portable_runner.py:574 Unable to pull image
apache/beam_java8_sdk:2.45.0.dev, defaulting to local image if it exists
INFO root:portable_runner.py:574 Waiting for docker to start up. Current
status is running
INFO root:portable_runner.py:574 Docker container is running. container_id
= b'22c89533bfc34e0d3b8ffa8526a1993fcc6481a462e66d910b771e899d0bc039',
worker_id = worker_87
INFO root:portable_runner.py:574 Created Worker handler
<apache_beam.runners.portability.fn_api_runner.worker_handlers.DockerSdkWorkerHandler
object at 0x7fa95043bd00> for environment
ref_Environment_default_environment_2 (beam:env:docker:v1,
b'\n%apache/beam_python3.10_sdk:2.45.0.dev')
INFO root:portable_runner.py:574 Attempting to pull image
apache/beam_python3.10_sdk:2.45.0.dev
INFO root:portable_runner.py:574 Unable to pull image
apache/beam_python3.10_sdk:2.45.0.dev, defaulting to local image if it exists
INFO root:portable_runner.py:574 Waiting for docker to start up. Current
status is running
INFO root:portable_runner.py:574 Docker container is running. container_id
= b'936305b420c69fc4915508d4070ba7ad5ff45fcfbd8ee568391dc7e2456a5c16',
worker_id = worker_88
INFO root:portable_runner.py:574 Completed job in 20.753364324569702
seconds with state DONE.
INFO apache_beam.runners.portability.portable_runner:portable_runner.py:581
Job state changed to DONE
PASSED [100%]
=============================== warnings summary ===============================
../../build/gradleenv/1922375555/lib/python3.10/site-packages/hdfs/config.py:15
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.10/site-packages/hdfs/config.py>:15:
DeprecationWarning: the imp module is deprecated in favour of importlib and
slated for removal in Python 3.12; see the module's documentation for
alternative uses
from imp import load_source
../../build/gradleenv/1922375555/lib/python3.10/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.10/site-packages/google/api_core/operations_v1/abstract_operations_client.py>:17:
DeprecationWarning: The distutils package is deprecated and slated for removal
in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
from distutils import util
../../build/gradleenv/1922375555/lib/python3.10/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/build/gradleenv/1922375555/lib/python3.10/site-packages/tenacity/_asyncio.py>:42:
DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use
"async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/typehints/pandas_type_compatibility_test.py:67
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:67:
FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas
in a future version. Use pandas.Index with the appropriate dtype instead.
}).set_index(pd.Int64Index(range(123, 223), name='an_index')),
apache_beam/typehints/pandas_type_compatibility_test.py:90
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:90:
FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas
in a future version. Use pandas.Index with the appropriate dtype instead.
pd.Int64Index(range(123, 223), name='an_index'),
apache_beam/typehints/pandas_type_compatibility_test.py:91
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:91:
FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas
in a future version. Use pandas.Index with the appropriate dtype instead.
pd.Int64Index(range(475, 575), name='another_index'),
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/python/pytest_xlangSqlValidateRunner.xml>
-
==== 9 passed, 5 skipped, 6735 deselected, 6 warnings in 354.09s (0:05:54) =====
> Task :sdks:python:test-suites:direct:xlang:fnApiJobServerCleanup
Killing process at 4048819
> Task :sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerCleanup
Stopping expansion service pid: 4065638.
Stopping expansion service pid: 4065639.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':sdks:python:test-suites:direct:xlang:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at:
> file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Direct/ws/src/sdks/python/test-suites/direct/xlang/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 39m 13s
232 actionable tasks: 32 executed, 200 up-to-date
Publishing build scan...
https://gradle.com/s/3fugbdonokjuk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
