See
<https://ci-beam.apache.org/job/beam_PostCommit_Python37/5336/display/redirect?page=changes>
Changes:
[noreply] [BEAM-13769]Add no_xdist marker for cloudpickle test (#17538)
------------------------------------------
[...truncated 55.46 MB...]
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Jun 10,
2022 1:47:59 PM org.apache.flink.runtime.executiongraph.Execution
transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO:
MapPartition (MapPartition at [3]Write to Spanner/SpannerIO.Write/Write
mutations to Cloud Spanner/{Gather Sort And Create Batches, Merge, Write
batches to Spanner}) (4/16) (f89ec9409767bf5949d8ded1d765f80a) switched from
RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Jun 10,
2022 1:47:59 PM org.apache.flink.runtime.executiongraph.Execution
transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO:
DataSink (DiscardingOutput) (4/16) (3c5908e5eee5fc566bc1e7c76bf9f912) switched
from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Jun 10,
2022 1:47:59 PM
org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager
processResourceRequirements'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO:
Received resource requirements from job d5e683738dcfc22585a4738a43980059:
[ResourceRequirement{resourceProfile=ResourceProfile{UNKNOWN},
numberOfRequiredSlots=1}]'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Jun 10,
2022 1:47:59 PM org.apache.flink.runtime.taskmanager.Task transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO:
CHAIN MapPartition (MapPartition at [3]Write to Spanner/SpannerIO.Write/Write
mutations to Cloud Spanner/{Filter Unbatchable Mutations, Merge, Write batches
to Spanner}) -> FlatMap (FlatMap at ExtractOutput[0]) (1/16)#0
(2af57aeccea7f6a561aee3c62313413e) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Jun 10,
2022 1:47:59 PM org.apache.flink.runtime.taskmanager.Task doRun'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO:
Freeing task resources for CHAIN MapPartition (MapPartition at [3]Write to
Spanner/SpannerIO.Write/Write mutations to Cloud Spanner/{Filter Unbatchable
Mutations, Merge, Write batches to Spanner}) -> FlatMap (FlatMap at
ExtractOutput[0]) (1/16)#0 (2af57aeccea7f6a561aee3c62313413e).'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Jun 10,
2022 1:47:59 PM org.apache.flink.runtime.taskexecutor.TaskExecutor
unregisterTaskAndNotifyFinalState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO:
Un-registering task and sending final execution state FINISHED to JobManager
for task CHAIN MapPartition (MapPartition at [3]Write to
Spanner/SpannerIO.Write/Write mutations to Cloud Spanner/{Filter Unbatchable
Mutations, Merge, Write batches to Spanner}) -> FlatMap (FlatMap at
ExtractOutput[0]) (1/16)#0 2af57aeccea7f6a561aee3c62313413e.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Jun 10,
2022 1:47:59 PM org.apache.flink.runtime.executiongraph.Execution
transitionState'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO:
CHAIN MapPartition (MapPartition at [3]Write to Spanner/SpannerIO.Write/Write
mutations to Cloud Spanner/{Filter Unbatchable Mutations, Merge, Write batches
to Spanner}) -> FlatMap (FlatMap at ExtractOutput[0]) (1/16)
(2af57aeccea7f6a561aee3c62313413e) switched from RUNNING to FINISHED.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Jun 10,
2022 1:47:59 PM
org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient
unref'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO:
Closing environment urn: "beam:env:external:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'payload: "\\n\\021\\n\\017localhost:44127"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:bytes:v1"'
INFO apache_beam.runners.worker.sdk_worker:sdk_worker.py:261 No more
requests from control plane
INFO apache_beam.runners.worker.sdk_worker:sdk_worker.py:262 SDK Harness
waiting for in-flight requests to complete
INFO apache_beam.runners.worker.data_plane:data_plane.py:805 Closing all
cached grpc data channels.
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:string_utf8:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:kv:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:bool:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:varint:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:double:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:iterable:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:timer:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:interval_window:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:length_prefix:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:global_window:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:windowed_value:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:param_windowed_value:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:state_backed_iterable:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:custom_window:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:row:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:sharded_key:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:nullable:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:protocol:progress_reporting:v0"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:protocol:harness_monitoring_infos:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:protocol:worker_status:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:combinefn:packed_python:v1"'
INFO apache_beam.runners.worker.sdk_worker:sdk_worker.py:877 Closing all
cached gRPC state handlers.
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:version:sdk_base:apache/beam_python3.7_sdk:2.40.0.dev"'
INFO apache_beam.runners.worker.sdk_worker:sdk_worker.py:274 Done consuming
work.
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:transform:to_string:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b''
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Jun 10,
2022 1:47:59 PM
org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer$InboundObserver
onCompleted'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'WARNING: Hanged up for unknown endpoint.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Jun 10,
2022 1:47:59 PM
org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer$InboundObserver
onCompleted'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'WARNING: Hanged up for unknown endpoint.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Jun 10,
2022 1:47:59 PM
org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory$WrappedSdkHarnessClient
unref'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO:
Closing environment urn: "beam:env:docker:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'payload: "\\n apache/beam_java8_sdk:2.40.0.dev"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:bytes:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:bool:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:varint:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:string_utf8:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:iterable:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:timer:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:kv:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:length_prefix:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:global_window:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:interval_window:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:custom_window:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:windowed_value:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:double:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:row:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:param_windowed_value:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:state_backed_iterable:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:sharded_key:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:coder:nullable:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:protocol:multi_core_bundle_processing:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:protocol:progress_reporting:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:protocol:harness_monitoring_infos:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:protocol:control_request_elements_embedding:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:protocol:state_caching:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:version:sdk_base:apache/beam_java8_sdk:2.40.0.dev"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'capabilities: "beam:transform:to_string:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'dependencies {'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'
type_urn: "beam:artifact:type:file:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'
type_payload:
"\\n\\316\\001/tmp/beam-tempqp5h0jm6/artifactswv_8_dhf/3e904c3316a56988d1d404ce58f363a85098858631ec0b1741d13d90c3edecae/1-external_6beam:env:docker-beam-sdks-java-io-google-cloud-platform-expansion-service-2.40.0-SNAPSHO"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'
role_urn: "beam:artifact:role:staging_to:v1"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'
role_payload:
"\\nybeam-sdks-java-io-google-cloud-platform-expansion-service-2.40.0-SNAPSHOT-X2l6a0iUNpijrH-SxXPBu6Gvrmjfub0W-3zIgAK7oR8.jar"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'}'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b''
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Jun 10,
2022 1:47:59 PM org.apache.beam.runners.fnexecution.logging.GrpcLoggingService
close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO: 1
Beam Fn Logging clients still connected during shutdown.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Jun 10,
2022 1:47:59 PM
org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer$InboundObserver
onCompleted'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'WARNING: Hanged up for unknown endpoint.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Jun 10,
2022 1:47:59 PM
org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment
close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO:
Closing Docker container
f1b8dc85fa6f57b8e7df02bd1c8185f45f60139d0ff92c37519c91b16f59b85f. Logs:'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'2022/06/10 13:47:53 Provision info:'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'pipeline_options:{fields:{key:"beam:option:allow_non_deterministic_key_coders:v1"
value:{bool_value:false}}
fields:{key:"beam:option:allow_non_restored_state:v1"
value:{bool_value:false}} fields:{key:"beam:option:allow_unsafe_triggers:v1"
value:{bool_value:false}} fields:{key:"beam:option:app_name:v1"
value:{null_value:NULL_VALUE}} fields:{key:"beam:option:artifact_port:v1"
value:{string_value:"0"}}
fields:{key:"beam:option:auto_balance_write_files_sharding_enabled:v1"
value:{bool_value:false}} fields:{key:"beam:option:beam_services:v1"
value:{struct_value:{}}} fields:{key:"beam:option:block_on_run:v1"
value:{bool_value:true}} fields:{key:"beam:option:dataflow_endpoint:v1"
value:{string_value:"https://dataflow.googleapis.com"}}
fields:{key:"beam:option:direct_embed_docker_python:v1"
value:{bool_value:false}} fields:{key:"beam:option:direct_num_workers:v1"
value:{string_value:"1"}}
fields:{key:"beam:option:direct_runner_bundle_repeat:v1"
value:{string_value:"0"}}
fields:{key:"beam:option:direct_runner_use_stacked_bundle:v1"
value:{bool_value:true}} fields:{key:"beam:option:direct_running_mode:v1"
value:{string_value:"in_memory"}} fields:{key:"beam:option:disable_metrics:v1"
value:{bool_value:false}} fields:{key:"beam:option:dry_run:v1"
value:{bool_value:false}} fields:{key:"beam:option:enable_artifact_caching:v1"
value:{bool_value:false}} fields:{key:"beam:option:enable_hot_key_logging:v1"
value:{bool_value:false}}
fields:{key:"beam:option:enable_streaming_engine:v1" value:{bool_value:false}}
fields:{key:"beam:option:enforce_encodability:v1" value:{bool_value:true}}
fields:{key:"beam:option:enforce_immutability:v1" value:{bool_value:true}}
fields:{key:"beam:option:environment_cache_millis:v1"
value:{string_value:"0"}} fields:{key:"beam:option:environment_config:v1"
value:{string_value:"localhost:44127"}}
fields:{key:"beam:option:environment_type:v1" value:{string_value:"LOOPBACK"}}
fields:{key:"beam:option:expansion_port:v1" value:{string_value:"0"}}
fields:{key:"beam:option:experiments:v1"
value:{list_value:{values:{string_value:"beam_fn_api"}}}}
fields:{key:"beam:option:externalized_checkpoints_enabled:v1"
value:{bool_value:false}}
fields:{key:"beam:option:fail_on_checkpointing_errors:v1"
value:{bool_value:true}} fields:{key:"beam:option:faster_copy:v1"
value:{bool_value:false}}
fields:{key:"beam:option:finish_bundle_before_checkpointing:v1"
value:{bool_value:false}} fields:{key:"beam:option:flink_job_server_jar:v1"
value:{string_value:"<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/flink/1.15/job-server/build/libs/beam-runners-flink-1.15-job-server-2.40.0-SNAPSHOT.jar"}}>
fields:{key:"beam:option:flink_master:v1" value:{string_value:"[auto]"}}
fields:{key:"beam:option:flink_submit_uber_jar:v1" value:{bool_value:false}}
fields:{key:"beam:option:flink_version:v1" value:{string_value:"1.15"}}
fields:{key:"beam:option:gcs_performance_metrics:v1" value:{bool_value:false}}
fields:{key:"beam:option:hdfs_full_urls:v1" value:{bool_value:false}}
fields:{key:"beam:option:job_name:v1"
value:{string_value:"BeamApp-jenkins-0610134735-19f707e6"}}
fields:{key:"beam:option:job_port:v1" value:{string_value:"0"}}
fields:{key:"beam:option:job_server_java_launcher:v1"
value:{string_value:"java"}}
fields:{key:"beam:option:job_server_jvm_properties:v1" value:{list_value:{}}}
fields:{key:"beam:option:job_server_timeout:v1" value:{string_value:"60"}}
fields:{key:"beam:option:load_balance_bundles:v1" value:{bool_value:false}}
fields:{key:"beam:option:no_auth:v1" value:{bool_value:false}}
fields:{key:"beam:option:object_reuse:v1" value:{bool_value:false}}
fields:{key:"beam:option:options_id:v1" value:{number_value:2}}
fields:{key:"beam:option:output_executable_path:v1"
value:{null_value:NULL_VALUE}}
fields:{key:"beam:option:performance_runtime_type_check:v1"
value:{bool_value:false}} fields:{key:"beam:option:pickle_library:v1"
value:{string_value:"default"}}
fields:{key:"beam:option:pipeline_type_check:v1" value:{bool_value:true}}
fields:{key:"beam:option:profile_cpu:v1" value:{bool_value:false}}
fields:{key:"beam:option:profile_memory:v1" value:{bool_value:false}}
fields:{key:"beam:option:profile_sample_rate:v1" value:{number_value:1}}
fields:{key:"beam:option:project:v1"
value:{string_value:"apache-beam-testing"}}
fields:{key:"beam:option:re_iterable_group_by_key_result:v1"
value:{bool_value:false}}
fields:{key:"beam:option:requirements_cache_only_sources:v1"
value:{bool_value:false}} fields:{key:"beam:option:resource_hints:v1"
value:{list_value:{}}} fields:{key:"beam:option:retain_docker_containers:v1"
value:{bool_value:false}}
fields:{key:"beam:option:retain_externalized_checkpoints_on_cancellation:v1"
value:{bool_value:false}} fields:{key:"beam:option:runner:v1"
value:{null_value:NULL_VALUE}}
fields:{key:"beam:option:runner_determined_sharding:v1"
value:{bool_value:true}} fields:{key:"beam:option:runtime_type_check:v1"
value:{bool_value:false}} fields:{key:"beam:option:s3_disable_ssl:v1"
value:{bool_value:false}} fields:{key:"beam:option:save_main_session:v1"
value:{bool_value:false}} fields:{key:"beam:option:sdk_location:v1"
value:{string_value:"container"}}
fields:{key:"beam:option:sdk_worker_parallelism:v1" value:{string_value:"1"}}
fields:{key:"beam:option:spark_master_url:v1" value:{string_value:"local[4]"}}
fields:{key:"beam:option:spark_submit_uber_jar:v1" value:{bool_value:false}}
fields:{key:"beam:option:spark_version:v1" value:{string_value:"2"}}
fields:{key:"beam:option:streaming:v1" value:{bool_value:false}}
fields:{key:"beam:option:temp_location:v1"
value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-it"}}
fields:{key:"beam:option:type_check_additional:v1" value:{string_value:""}}
fields:{key:"beam:option:type_check_strictness:v1"
value:{string_value:"DEFAULT_TO_ANY"}} fields:{key:"beam:option:update:v1"
value:{bool_value:false}} fields:{key:"beam:option:use_storage_write_api:v1"
value:{bool_value:false}}
fields:{key:"beam:option:use_storage_write_api_at_least_once:v1"
value:{bool_value:false}} fields:{key:"beam:option:verify_row_values:v1"
value:{bool_value:false}}} logging_endpoint:{url:"localhost:40659"}
artifact_endpoint:{url:"localhost:38147"}
control_endpoint:{url:"localhost:46073"}
dependencies:{type_urn:"beam:artifact:type:file:v1"
type_payload:"\\n\\xce\\x01/tmp/beam-tempqp5h0jm6/artifactswv_8_dhf/3e904c3316a56988d1d404ce58f363a85098858631ec0b1741d13d90c3edecae/1-external_6beam:env:docker-beam-sdks-java-io-google-cloud-platform-expansion-service-2.40.0-SNAPSHO"
role_urn:"beam:artifact:role:staging_to:v1"
role_payload:"\\nybeam-sdks-java-io-google-cloud-platform-expansion-service-2.40.0-SNAPSHOT-X2l6a0iUNpijrH-SxXPBu6Gvrmjfub0W-3zIgAK7oR8.jar"}'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'2022/06/10 13:47:53 Initializing java harness: /opt/apache/beam/boot --id=2-1
--provision_endpoint=localhost:40401'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'2022/06/10 13:47:54 Downloaded:
/tmp/2-1/staged/beam-sdks-java-io-google-cloud-platform-expansion-service-2.40.0-SNAPSHOT-X2l6a0iUNpijrH-SxXPBu6Gvrmjfub0W-3zIgAK7oR8.jar
(sha256: 5f697a6b48943698a3ac7f92c573c1bba1afae68dfb9bd16fb7cc88002bba11f,
size: 120581763)'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'2022/06/10 13:47:54 Executing: java -Xmx76805122867 -XX:+UseParallelGC
-XX:+AlwaysActAsServerClassMachine -XX:-OmitStackTraceInFastThrow -cp
/opt/apache/beam/jars/slf4j-api.jar:/opt/apache/beam/jars/slf4j-jdk14.jar:/opt/apache/beam/jars/beam-sdks-java-harness.jar:/opt/apache/beam/jars/beam-sdks-java-io-kafka.jar:/opt/apache/beam/jars/kafka-clients.jar:/tmp/2-1/staged/beam-sdks-java-io-google-cloud-platform-expansion-service-2.40.0-SNAPSHOT-X2l6a0iUNpijrH-SxXPBu6Gvrmjfub0W-3zIgAK7oR8.jar
-javaagent:/opt/apache/beam/jars/jamm.jar org.apache.beam.fn.harness.FnHarness'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'SLF4J:
Class path contains multiple SLF4J bindings.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'SLF4J:
Found binding in
[jar:file:/opt/apache/beam/jars/slf4j-jdk14.jar!/org/slf4j/impl/StaticLoggerBinder.class]'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'SLF4J:
Found binding in
[jar:file:/tmp/2-1/staged/beam-sdks-java-io-google-cloud-platform-expansion-service-2.40.0-SNAPSHOT-X2l6a0iUNpijrH-SxXPBu6Gvrmjfub0W-3zIgAK7oR8.jar!/org/slf4j/impl/StaticLoggerBinder.class]'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'SLF4J:
See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'SLF4J:
Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory]'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'SDK Fn
Harness started'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Harness
ID 2-1'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Logging
location url: "localhost:40659"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b''
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Control
location url: "localhost:46073"'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b''
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Status
location null'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125
b'Pipeline options
{"beam:option:allow_non_deterministic_key_coders:v1":false,"beam:option:allow_non_restored_state:v1":false,"beam:option:allow_unsafe_triggers:v1":false,"beam:option:app_name:v1":null,"beam:option:artifact_port:v1":"0","beam:option:auto_balance_write_files_sharding_enabled:v1":false,"beam:option:beam_services:v1":{},"beam:option:block_on_run:v1":true,"beam:option:dataflow_endpoint:v1":"https://dataflow.googleapis.com","beam:option:direct_embed_docker_python:v1":false,"beam:option:direct_num_workers:v1":"1","beam:option:direct_runner_bundle_repeat:v1":"0","beam:option:direct_runner_use_stacked_bundle:v1":true,"beam:option:direct_running_mode:v1":"in_memory","beam:option:disable_metrics:v1":false,"beam:option:dry_run:v1":false,"beam:option:enable_artifact_caching:v1":false,"beam:option:enable_hot_key_logging:v1":false,"beam:option:enable_streaming_engine:v1":false,"beam:option:enforce_encodability:v1":true,"beam:option:enforce_immutability:v1":true,"beam:option:environment_cache_millis:v1":"0","beam:option:environment_config:v1":"localhost:44127","beam:option:environment_type:v1":"LOOPBACK","beam:option:expansion_port:v1":"0","beam:option:experiments:v1":["beam_fn_api"],"beam:option:externalized_checkpoints_enabled:v1":false,"beam:option:fail_on_checkpointing_errors:v1":true,"beam:option:faster_copy:v1":false,"beam:option:finish_bundle_before_checkpointing:v1":false,"beam:option:flink_job_server_jar:v1":";<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/runners/flink/1.15/job-server/build/libs/beam-runners-flink-1.15-job-server-2.40.0-SNAPSHOT.jar","beam>:option:flink_master:v1":"[auto]","beam:option:flink_submit_uber_jar:v1":false,"beam:option:flink_version:v1":"1.15","beam:option:gcs_performance_metrics:v1":false,"beam:option:hdfs_full_urls:v1":false,"beam:option:job_name:v1":"BeamApp-jenkins-0610134735-19f707e6","beam:option:job_port:v1":"0","beam:option:job_server_java_launcher:v1":"java","beam:option:job_server_jvm_properties:v1":[],"beam:option:job_server_timeout:v1":"60","beam:option:load_balance_bundles:v1":false,"beam:option:no_auth:v1":false,"beam:option:object_reuse:v1":false,"beam:option:options_id:v1":2,"beam:option:output_executable_path:v1":null,"beam:option:performance_runtime_type_check:v1":false,"beam:option:pickle_library:v1":"default","beam:option:pipeline_type_check:v1":true,"beam:option:profile_cpu:v1":false,"beam:option:profile_memory:v1":false,"beam:option:profile_sample_rate:v1":1,"beam:option:project:v1":"apache-beam-testing","beam:option:re_iterable_group_by_key_result:v1":false,"beam:option:requirements_cache_only_sources:v1":false,"beam:option:resource_hints:v1":[],"beam:option:retain_docker_containers:v1":false,"beam:option:retain_externalized_checkpoints_on_cancellation:v1":false,"beam:option:runner:v1":null,"beam:option:runner_determined_sharding:v1":true,"beam:option:runtime_type_check:v1":false,"beam:option:s3_disable_ssl:v1":false,"beam:option:save_main_session:v1":false,"beam:option:sdk_location:v1":"container","beam:option:sdk_worker_parallelism:v1":"1","beam:option:spark_master_url:v1":"local[4]","beam:option:spark_submit_uber_jar:v1":false,"beam:option:spark_version:v1":"2","beam:option:streaming:v1":false,"beam:option:temp_location:v1":"gs://temp-storage-for-end-to-end-tests/temp-it","beam:option:type_check_additional:v1":"","beam:option:type_check_strictness:v1":"DEFAULT_TO_ANY","beam:option:update:v1":false,"beam:option:use_storage_write_api:v1":false,"beam:option:use_storage_write_api_at_least_once:v1":false,"beam:option:verify_row_values:v1":false}'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Jun 10,
2022 1:48:00 PM
org.apache.flink.runtime.state.TaskExecutorLocalStateStoresManager shutdown'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO:
Shutting down TaskExecutorLocalStateStoresManager.'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Jun 10,
2022 1:48:00 PM org.apache.flink.runtime.blob.AbstractBlobCache close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO:
Shutting down BLOB cache'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'Jun 10,
2022 1:48:00 PM org.apache.flink.runtime.blob.AbstractBlobCache close'
INFO apache_beam.utils.subprocess_server:subprocess_server.py:125 b'INFO:
Shutting down BLOB cache'
Terminated
> Task :sdks:python:test-suites:portable:py37:xlangSpannerIOIT FAILED
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT
[gw0] node down: Not properly terminated
[gw0] [31mFAILED[0m
apache_beam/examples/complete/game/user_score_it_test.py::UserScoreIT::test_user_score_it
replacing crashed worker gw0
Terminated
Traceback (most recent call last):
File "<string>", line 1, in <module>
> Task :sdks:python:test-suites:dataflow:py37:postCommitIT FAILED
> Task :sdks:python:test-suites:dataflow:py37:spannerioIT FAILED
Remove shutdown hook failed
java.lang.IllegalStateException: Shutdown in progress
at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
at java.lang.Runtime.removeShutdownHook(Runtime.java:239)
at
org.gradle.process.internal.shutdown.ShutdownHooks.removeShutdownHook(ShutdownHooks.java:38)
at
org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:208)
at
org.gradle.process.internal.DefaultExecHandle.failed(DefaultExecHandle.java:369)
at
org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:87)
at
org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:42)
at
org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at
org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/portable/common.gradle'>
line: 352
* What went wrong:
Execution failed for task
':sdks:python:test-suites:portable:py37:xlangSpannerIOIT'.
> Process 'command 'sh'' finished with non-zero exit value 143
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 121
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:postCommitIT'.
> Process 'command 'sh'' finished with non-zero exit value 143
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:test-suites:dataflow:py37:spannerioIT'.
> A problem occurred starting process 'command 'sh''
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 27m 34s
226 actionable tasks: 159 executed, 61 from cache, 6 up-to-date
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=eed0b029-3acc-4b3d-b6db-6d043f798275,
currentDir=<https://ci-beam.apache.org/job/beam_PostCommit_Python37/ws/src}>
Attempting to read last messages from the daemon log...
Daemon pid: 964098
log file: /home/jenkins/.gradle/daemon/7.4/daemon-964098.out.log
----- Last 20 lines from daemon log file - daemon-964098.out.log -----
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 27m 34s
226 actionable tasks: 159 executed, 61 from cache, 6 up-to-date
Remove shutdown hook failed
java.lang.IllegalStateException: Shutdown in progress
at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82)
at java.lang.Runtime.removeShutdownHook(Runtime.java:239)
at
org.gradle.process.internal.shutdown.ShutdownHooks.removeShutdownHook(ShutdownHooks.java:38)
at
org.gradle.process.internal.DefaultExecHandle.setEndStateInfo(DefaultExecHandle.java:208)
at
org.gradle.process.internal.DefaultExecHandle.aborted(DefaultExecHandle.java:365)
at
org.gradle.process.internal.ExecHandleRunner.completed(ExecHandleRunner.java:108)
at
org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:84)
at
org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:42)
at
org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
at
org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:48)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may
have crashed)
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
