See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/6517/display/redirect?page=changes>
Changes: [noreply] Pass user specified destination type to UpdateSchemaDestination [noreply] [Go SDK] Stream decode values in single iterations (#22904) [noreply] Enable autosharding for BQ: #22818 [noreply] Update wordcount_minimal.py by removing pipeline_args.extend (#22786) [noreply] Add map state in the Go Sdk (#22897) [noreply] [BEAM-12164] Feat: Added support to Cloud Spanner Change Streams ------------------------------------------ [...truncated 2.36 MB...] 22/08/26 20:54:22 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:40029 for data ... 22/08/26 20:54:22 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected. 22/08/26 20:54:22 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1" payload: "\n\026apache/beam_go_sdk:dev" capabilities: "beam:protocol:progress_reporting:v1" capabilities: "beam:protocol:multi_core_bundle_processing:v1" capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1" capabilities: "beam:protocol:worker_status:v1" capabilities: "beam:protocol:monitoring_info_short_ids:v1" capabilities: "beam:version:sdk_base:go" capabilities: "beam:coder:bytes:v1" capabilities: "beam:coder:bool:v1" capabilities: "beam:coder:varint:v1" capabilities: "beam:coder:double:v1" capabilities: "beam:coder:string_utf8:v1" capabilities: "beam:coder:length_prefix:v1" capabilities: "beam:coder:kv:v1" capabilities: "beam:coder:iterable:v1" capabilities: "beam:coder:state_backed_iterable:v1" capabilities: "beam:coder:windowed_value:v1" capabilities: "beam:coder:global_window:v1" capabilities: "beam:coder:interval_window:v1" capabilities: "beam:coder:row:v1" capabilities: "beam:coder:nullable:v1" dependencies { type_urn: "beam:artifact:type:file:v1" type_payload: "\n\203\001/tmp/beam-artifact-staging/60bd8ed8664c2402a0eadcc7cffbc7c4d3d574714037cee767a0490553ca1d23/1-go-/tmp/worker-13-1661547257722046262" role_urn: "beam:artifact:role:go_worker_binary:v1" } 22/08/26 20:54:22 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown. 22/08/26 20:54:22 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint. 22/08/26 20:54:22 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 6e4cf794b36e00364c54e25d8f8f9f1efa5df9389c4ca7b90fc7e5942c707b66. Logs: 2022/08/26 20:54:22 Provision info: pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-158"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:38281"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test24226"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test24226"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"2h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0158-jenkins-0826205419-5efe9072"}} fields:{key:"beam:option:options_id:v1" value:{number_value:36}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:38997"} artifact_endpoint:{url:"localhost:38405"} control_endpoint:{url:"localhost:41693"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/60bd8ed8664c2402a0eadcc7cffbc7c4d3d574714037cee767a0490553ca1d23/1-go-/tmp/worker-13-1661547257722046262" role_urn:"beam:artifact:role:go_worker_binary:v1"} 2022/08/26 20:54:22 Initializing Go harness: /opt/apache/beam/boot --id=251-1 --provision_endpoint=localhost:32897 2022/08/26 20:54:22 Downloaded: /tmp/staged/1-worker-13-1661547257722046262 (sha256: 4de00ea48f215f17f46e36d89f39f150a3c867f4619c6639c21e85d336847728, size: 26499031) Error Setting Rlimit operation not permitted Worker exited successfully! 2022/08/26 20:54:22 control response channel closed 22/08/26 20:54:23 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata. 22/08/26 20:54:23 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found 22/08/26 20:54:24 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected. 22/08/26 20:54:24 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 252-1 22/08/26 20:54:24 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:33329 for control ... 22/08/26 20:54:24 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35167 for logging ... 22/08/26 20:54:24 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 252-2 22/08/26 20:54:24 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43215 for data ... 22/08/26 20:54:24 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected. 22/08/26 20:54:24 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1" payload: "\n\026apache/beam_go_sdk:dev" capabilities: "beam:protocol:progress_reporting:v1" capabilities: "beam:protocol:multi_core_bundle_processing:v1" capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1" capabilities: "beam:protocol:worker_status:v1" capabilities: "beam:protocol:monitoring_info_short_ids:v1" capabilities: "beam:version:sdk_base:go" capabilities: "beam:coder:bytes:v1" capabilities: "beam:coder:bool:v1" capabilities: "beam:coder:varint:v1" capabilities: "beam:coder:double:v1" capabilities: "beam:coder:string_utf8:v1" capabilities: "beam:coder:length_prefix:v1" capabilities: "beam:coder:kv:v1" capabilities: "beam:coder:iterable:v1" capabilities: "beam:coder:state_backed_iterable:v1" capabilities: "beam:coder:windowed_value:v1" capabilities: "beam:coder:global_window:v1" capabilities: "beam:coder:interval_window:v1" capabilities: "beam:coder:row:v1" capabilities: "beam:coder:nullable:v1" dependencies { type_urn: "beam:artifact:type:file:v1" type_payload: "\n\203\001/tmp/beam-artifact-staging/60bd8ed8664c2402a0eadcc7cffbc7c4d3d574714037cee767a0490553ca1d23/1-go-/tmp/worker-13-1661547257722046262" role_urn: "beam:artifact:role:go_worker_binary:v1" } 22/08/26 20:54:24 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown. 22/08/26 20:54:24 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint. 22/08/26 20:54:24 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container 342dd972fec05e912ce4a00d8e7f018433e029f426d6352413b6c54c691bcf89. Logs: 2022/08/26 20:54:24 Provision info: pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-158"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:38281"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test24226"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test24226"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"2h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0158-jenkins-0826205419-5efe9072"}} fields:{key:"beam:option:options_id:v1" value:{number_value:36}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:35167"} artifact_endpoint:{url:"localhost:33979"} control_endpoint:{url:"localhost:33329"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/60bd8ed8664c2402a0eadcc7cffbc7c4d3d574714037cee767a0490553ca1d23/1-go-/tmp/worker-13-1661547257722046262" role_urn:"beam:artifact:role:go_worker_binary:v1"} 2022/08/26 20:54:24 Initializing Go harness: /opt/apache/beam/boot --id=252-1 --provision_endpoint=localhost:44613 2022/08/26 20:54:24 Downloaded: /tmp/staged/1-worker-13-1661547257722046262 (sha256: 4de00ea48f215f17f46e36d89f39f150a3c867f4619c6639c21e85d336847728, size: 26499031) Error Setting Rlimit operation not permitted Worker exited successfully! 2022/08/26 20:54:24 control response channel closed 22/08/26 20:54:25 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata. 22/08/26 20:54:25 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found 22/08/26 20:54:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 253-1 22/08/26 20:54:25 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected. 22/08/26 20:54:25 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43031 for control ... 22/08/26 20:54:25 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:35827 for logging ... 22/08/26 20:54:25 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 253-2 22/08/26 20:54:25 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:43279 for data ... 22/08/26 20:54:25 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected. 22/08/26 20:54:25 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39701 for state ... 22/08/26 20:54:25 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1" payload: "\n\026apache/beam_go_sdk:dev" capabilities: "beam:protocol:progress_reporting:v1" capabilities: "beam:protocol:multi_core_bundle_processing:v1" capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1" capabilities: "beam:protocol:worker_status:v1" capabilities: "beam:protocol:monitoring_info_short_ids:v1" capabilities: "beam:version:sdk_base:go" capabilities: "beam:coder:bytes:v1" capabilities: "beam:coder:bool:v1" capabilities: "beam:coder:varint:v1" capabilities: "beam:coder:double:v1" capabilities: "beam:coder:string_utf8:v1" capabilities: "beam:coder:length_prefix:v1" capabilities: "beam:coder:kv:v1" capabilities: "beam:coder:iterable:v1" capabilities: "beam:coder:state_backed_iterable:v1" capabilities: "beam:coder:windowed_value:v1" capabilities: "beam:coder:global_window:v1" capabilities: "beam:coder:interval_window:v1" capabilities: "beam:coder:row:v1" capabilities: "beam:coder:nullable:v1" dependencies { type_urn: "beam:artifact:type:file:v1" type_payload: "\n\203\001/tmp/beam-artifact-staging/60bd8ed8664c2402a0eadcc7cffbc7c4d3d574714037cee767a0490553ca1d23/1-go-/tmp/worker-13-1661547257722046262" role_urn: "beam:artifact:role:go_worker_binary:v1" } 22/08/26 20:54:25 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown. 22/08/26 20:54:25 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint. 22/08/26 20:54:26 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container b25c0e5419497c84b04dd14cc093f3d73da3249a253a31830d52d76987137827. Logs: 2022/08/26 20:54:25 Provision info: pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-158"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:38281"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test24226"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test24226"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"2h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0158-jenkins-0826205419-5efe9072"}} fields:{key:"beam:option:options_id:v1" value:{number_value:36}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:35827"} artifact_endpoint:{url:"localhost:39753"} control_endpoint:{url:"localhost:43031"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/60bd8ed8664c2402a0eadcc7cffbc7c4d3d574714037cee767a0490553ca1d23/1-go-/tmp/worker-13-1661547257722046262" role_urn:"beam:artifact:role:go_worker_binary:v1"} 2022/08/26 20:54:25 Initializing Go harness: /opt/apache/beam/boot --id=253-1 --provision_endpoint=localhost:35393 2022/08/26 20:54:25 Downloaded: /tmp/staged/1-worker-13-1661547257722046262 (sha256: 4de00ea48f215f17f46e36d89f39f150a3c867f4619c6639c21e85d336847728, size: 26499031) Error Setting Rlimit operation not permitted Worker exited successfully! 2022/08/26 20:54:25 control response channel closed 22/08/26 20:54:26 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata. 22/08/26 20:54:26 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata. 22/08/26 20:54:26 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0158-jenkins-0826205419-5efe9072_9c3b372a-76ba-42be-b0d6-9db42d4a89fb: Pipeline translated successfully. Computing outputs 22/08/26 20:54:26 WARN software.amazon.awssdk.regions.internal.util.EC2MetadataUtils: Unable to retrieve the requested metadata. 22/08/26 20:54:27 WARN org.apache.beam.runners.fnexecution.environment.DockerCommand: Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found 22/08/26 20:54:27 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: Beam Fn Logging client connected. 22/08/26 20:54:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: Beam Fn Control client connected with id 254-1 22/08/26 20:54:27 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:45811 for control ... 22/08/26 20:54:27 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:34109 for logging ... 22/08/26 20:54:27 INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService: getProcessBundleDescriptor request with id 254-2 22/08/26 20:54:27 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:41991 for data ... 22/08/26 20:54:27 INFO org.apache.beam.runners.fnexecution.data.GrpcDataService: Beam Fn Data client connected. 22/08/26 20:54:27 INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:677: Connecting via grpc @ localhost:39893 for state ... 22/08/26 20:54:27 INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory: Closing environment urn: "beam:env:docker:v1" payload: "\n\026apache/beam_go_sdk:dev" capabilities: "beam:protocol:progress_reporting:v1" capabilities: "beam:protocol:multi_core_bundle_processing:v1" capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1" capabilities: "beam:protocol:worker_status:v1" capabilities: "beam:protocol:monitoring_info_short_ids:v1" capabilities: "beam:version:sdk_base:go" capabilities: "beam:coder:bytes:v1" capabilities: "beam:coder:bool:v1" capabilities: "beam:coder:varint:v1" capabilities: "beam:coder:double:v1" capabilities: "beam:coder:string_utf8:v1" capabilities: "beam:coder:length_prefix:v1" capabilities: "beam:coder:kv:v1" capabilities: "beam:coder:iterable:v1" capabilities: "beam:coder:state_backed_iterable:v1" capabilities: "beam:coder:windowed_value:v1" capabilities: "beam:coder:global_window:v1" capabilities: "beam:coder:interval_window:v1" capabilities: "beam:coder:row:v1" capabilities: "beam:coder:nullable:v1" dependencies { type_urn: "beam:artifact:type:file:v1" type_payload: "\n\203\001/tmp/beam-artifact-staging/60bd8ed8664c2402a0eadcc7cffbc7c4d3d574714037cee767a0490553ca1d23/1-go-/tmp/worker-13-1661547257722046262" role_urn: "beam:artifact:role:go_worker_binary:v1" } 22/08/26 20:54:27 INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService: 1 Beam Fn Logging clients still connected during shutdown. 22/08/26 20:54:27 WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer: Hanged up for unknown endpoint. 22/08/26 20:54:27 INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment: Closing Docker container e22aebc2f34ec6153269d74adeac6947e2dfd38ae305e0c42ddec530062dcd2b. Logs: 2022/08/26 20:54:27 Provision info: pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk-158"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:38281"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.42.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"spark"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test/test24226"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test/test24226"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"2h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:job_name:v1" value:{string_value:"go0testemitpardoaftergbk0158-jenkins-0826205419-5efe9072"}} fields:{key:"beam:option:options_id:v1" value:{number_value:36}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{null_value:NULL_VALUE}} fields:{key:"beam:option:spark_master:v1" value:{string_value:"local"}}} logging_endpoint:{url:"localhost:34109"} artifact_endpoint:{url:"localhost:39873"} control_endpoint:{url:"localhost:45811"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/60bd8ed8664c2402a0eadcc7cffbc7c4d3d574714037cee767a0490553ca1d23/1-go-/tmp/worker-13-1661547257722046262" role_urn:"beam:artifact:role:go_worker_binary:v1"} 2022/08/26 20:54:27 Initializing Go harness: /opt/apache/beam/boot --id=254-1 --provision_endpoint=localhost:39349 2022/08/26 20:54:27 Downloaded: /tmp/staged/1-worker-13-1661547257722046262 (sha256: 4de00ea48f215f17f46e36d89f39f150a3c867f4619c6639c21e85d336847728, size: 26499031) Error Setting Rlimit operation not permitted 2022/08/26 20:54:27 control response channel closed Worker exited successfully! 22/08/26 20:54:28 INFO org.apache.beam.runners.spark.SparkPipelineRunner: Job go0testemitpardoaftergbk0158-jenkins-0826205419-5efe9072_9c3b372a-76ba-42be-b0d6-9db42d4a89fb finished. 22/08/26 20:54:28 WARN org.apache.beam.runners.jobsubmission.InMemoryJobService: Failed to remove job staging directory for token go-testemitpardoaftergbk-158_e22c92e8-f3d2-4745-9370-26ded599a85b. java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/60bd8ed8664c2402a0eadcc7cffbc7c4d3d574714037cee767a0490553ca1d23/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/.> at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307) at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105) at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98) at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124) at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217) at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180) at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115) at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67) at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261) at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249) at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51) at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115) at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) 2022/08/26 20:54:28 Job state: DONE 22/08/26 20:54:28 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Getting job metrics for go0testemitpardoaftergbk0158-jenkins-0826205419-5efe9072_9c3b372a-76ba-42be-b0d6-9db42d4a89fb 22/08/26 20:54:28 INFO org.apache.beam.runners.jobsubmission.InMemoryJobService: Finished getting job metrics for go0testemitpardoaftergbk0158-jenkins-0826205419-5efe9072_9c3b372a-76ba-42be-b0d6-9db42d4a89fb --- PASS: TestEmitParDoAfterGBK (10.63s) FAIL FAIL github.com/apache/beam/sdks/v2/go/test/regression 130.188s FAIL > Task :sdks:go:test:sparkValidatesRunner FAILED FAILURE: Build failed with an exception. * Where: Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/ws/src/sdks/go/test/build.gradle'> line: 121 * What went wrong: Execution failed for task ':sdks:go:test:sparkValidatesRunner'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness. Please consult deprecation warnings for more details. BUILD FAILED in 14m 52s 114 actionable tasks: 76 executed, 33 from cache, 5 up-to-date Publishing build scan... https://gradle.com/s/ibcmfdyp6qlpw Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
