See <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/1305/display/redirect?page=changes>
Changes: [nielm] Fix SpannerIO service call metrics and improve tests. [andyye333] Add Pytorch support for batched keyed examples [andyye333] Add general support for non-batchable kwargs params; Add [noreply] [BEAM-12554] Create new instances of FileSink in sink_fn (#17708) [noreply] DataflowRunner: Experiment added to disable unbounded PCcollection [noreply] More flexible Python Callable type. (#17767) ------------------------------------------ [...truncated 6.98 MB...] [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Got serdes from serialized instances: Set() [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Got change log system streams: Map() [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Got intermediate streams: List() [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Setting up JVM metrics. [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Setting up message chooser. [samza-runner-job-invoker-5] INFO org.apache.samza.system.chooser.DefaultChooser - Building default chooser with: useBatching=false, useBootstrapping=false, usePriority=false [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Setting up metrics reporters. [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Got metrics reporters: Set() [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Got security manager: null [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Got checkpoint manager: null [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Got checkpointListeners : Map() [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Got offset manager: org.apache.samza.checkpoint.OffsetManager@7f11246c [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Got storage engines: Set(beamStore) [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Got thread pool size: 0 [samza-runner-job-invoker-5] INFO org.apache.samza.task.TaskFactoryUtil - Adapting StreamOperatorTaskFactory to inject container thread pool [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Got default storage engine base directory: <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/ws/src/state> [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Got base directory for non logged data stores: <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/ws/src/state> [samza-runner-job-invoker-5] WARN org.apache.samza.container.SamzaContainer$ - No override was provided for logged store base directory. This disables local state re-use on application restart. If you want to enable this feature, set LOGGED_STORE_BASE_DIR as an environment variable in all machines running the Samza container or configure job.logged.store.base.dir for your application [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Got base directory for logged data stores: <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/ws/src/state> [samza-runner-job-invoker-5] INFO org.apache.samza.storage.ContainerStorageManager - Starting with changelogSystemStreams = {} sideInputSystemStreams = {} [samza-runner-job-invoker-5] WARN org.apache.samza.storage.ContainerStorageManager - Logged and non-logged store base directory are configured to same path: <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/ws/src/state.> It is recommended to configurethem separately to ensure clean up of non-logged store data doesn't accidentally impact logged store data. [samza-runner-job-invoker-5] INFO org.apache.samza.storage.ContainerStorageManager - Created store beamStore for task Partition 0 in mode BulkLoad [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Got task SSPs: Set(SystemStreamPartition [n14, n14, 0], SystemStreamPartition [n10, n10, 0], SystemStreamPartition [n1, n1, 0], SystemStreamPartition [n8, n8, 0]) [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Got task side input SSPs: Set() [samza-runner-job-invoker-5] INFO org.apache.samza.table.TableManager - Added 0 tables [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Got table manager [samza-runner-job-invoker-5] INFO org.apache.samza.container.RunLoopFactory - Got window milliseconds: -1. [samza-runner-job-invoker-5] INFO org.apache.samza.container.RunLoopFactory - Got commit milliseconds: -1. [samza-runner-job-invoker-5] INFO org.apache.samza.container.RunLoopFactory - Got taskMaxConcurrency: 1. [samza-runner-job-invoker-5] INFO org.apache.samza.container.RunLoopFactory - Got asyncCommitEnabled: false. [samza-runner-job-invoker-5] INFO org.apache.samza.container.RunLoopFactory - Got callbackTimeout: -1. [samza-runner-job-invoker-5] INFO org.apache.samza.container.RunLoopFactory - Got maxIdleMs: 10. [samza-runner-job-invoker-5] INFO org.apache.samza.container.RunLoopFactory - Run loop in asynchronous mode. [samza-runner-job-invoker-5] INFO org.apache.samza.container.disk.NoThrottlingDiskQuotaPolicy - Using a no throttling disk quota policy [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Disk quotas disabled because polling interval is not set (container.disk.poll.interval.ms) [samza-runner-job-invoker-5] INFO org.apache.samza.container.SamzaContainer$ - Samza container setup complete. [samza-runner-job-invoker-5] WARN org.apache.samza.config.TaskConfig - Unable to parse user-configure value for task.shutdown.ms - null. Using default value 30000 [samza-runner-job-invoker-5] INFO org.apache.samza.processor.StreamProcessor - Starting the container: org.apache.samza.container.SamzaContainer@1f5fc98d for the stream processor: 1. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Starting container. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.metrics.JmxServer - According to Util.getLocalHost.getHostName we are apache-ci-beam-jenkins-13 [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.metrics.JmxServer - Not overriding system property com.sun.management.jmxremote.authenticate as already has value false [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.metrics.JmxServer - Not overriding system property com.sun.management.jmxremote.ssl as already has value false [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.metrics.JmxServer - Not overriding system property java.rmi.server.hostname as already has value apache-ci-beam-jenkins-13 [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.metrics.JmxServer - Started JmxServer registry port=39143 server port=39095 url=service:jmx:rmi://localhost:39095/jndi/rmi://localhost:39143/jmxrmi [Samza StreamProcessor Container Thread-0] INFO STARTUP_LOGGER - Started JmxServer registry port=39143 server port=39095 url=service:jmx:rmi://localhost:39095/jndi/rmi://localhost:39143/jmxrmi [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.metrics.JmxServer - If you are tunneling, you might want to try JmxServer registry port=39143 server port=39095 url=service:jmx:rmi://apache-ci-beam-jenkins-13:39095/jndi/rmi://apache-ci-beam-jenkins-13:39143/jmxrmi [Samza StreamProcessor Container Thread-0] INFO STARTUP_LOGGER - If you are tunneling, you might want to try JmxServer registry port=39143 server port=39095 url=service:jmx:rmi://apache-ci-beam-jenkins-13:39095/jndi/rmi://apache-ci-beam-jenkins-13:39143/jmxrmi [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Registering task instance metrics. JMX Server: JmxServer registry port=39143 server port=39095 url=service:jmx:rmi://apache-ci-beam-jenkins-13:39095/jndi/rmi://apache-ci-beam-jenkins-13:39143/jmxrmi [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Starting JVM metrics. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Starting metrics reporters. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Starting admin multiplexer. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Registering task instances with offsets. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Starting offset manager. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.checkpoint.OffsetManager - Successfully loaded last processed offsets: {} [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.checkpoint.OffsetManager - Successfully loaded starting offsets: Map(Partition 0 -> Map(SystemStreamPartition [n1, n1, 0] -> 0, SystemStreamPartition [n8, n8, 0] -> 0, SystemStreamPartition [n14, n14, 0] -> 0, SystemStreamPartition [n10, n10, 0] -> 0)) [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Registering task instances with producers. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Starting producer multiplexer. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Starting container storage manager. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.storage.ContainerStorageManager - Checkpointed changelog ssp offsets: {} [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.storage.ContainerStorageManager - Store Restore started [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.storage.TransactionalStateTaskRestoreManager - Got current changelog offsets for taskName: Partition 0 as: {} [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.storage.TransactionalStateTaskRestoreManager - Marking current directory: <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/ws/src/state/beamStore/Partition_0> for store: beamStore in task: Partition 0 for deletion since it is not a logged store. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.storage.TransactionalStateTaskRestoreManager - Store directories to be retained in Task: Partition 0 are: {} [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.storage.TransactionalStateTaskRestoreManager - Store directories to be deleted in Task: Partition 0 are: {beamStore=[<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/ws/src/state/beamStore/Partition_0]}> [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.storage.TransactionalStateTaskRestoreManager - Stores to be restored in Task: Partition 0 are: {} [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.storage.TransactionalStateTaskRestoreManager - Deleting persistent store directory: <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/ws/src/state/beamStore/Partition_0> for store: beamStore in task: Partition 0 [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.storage.TransactionalStateTaskRestoreManager - Creating missing persistent store current directory: <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/ws/src/state/beamStore/Partition_0> for store: beamStore in task: Partition 0 [Samza Restore Thread-0] INFO org.apache.samza.storage.ContainerStorageManager - Starting stores in task instance Partition 0 [Samza Restore Thread-0] INFO org.apache.samza.storage.TransactionalStateTaskRestoreManager - Stopped persistent store: beamStore in task: Partition 0 [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.storage.ContainerStorageManager - Re-created store beamStore in read-write mode for task Partition 0 because it a persistent store [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.storage.ContainerStorageManager - Store Restore complete [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Starting table manager in task instance Partition 0 [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Starting host statistics monitor [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Initializing stream tasks. [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 event time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 processing time timers in memory [Samza StreamProcessor Container Thread-0] WARN org.apache.beam.runners.fnexecution.environment.DockerCommand - Unable to pull docker image apache/beam_go_sdk:dev, cause: Received exit code 1 for command 'docker pull apache/beam_go_sdk:dev'. stderr: Error response from daemon: manifest for apache/beam_go_sdk:dev not found [grpc-default-executor-15] INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService - Beam Fn Control client connected with id 34-1 [grpc-default-executor-18] INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService - Beam Fn Logging client connected. [grpc-default-executor-18] INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:654 - Connecting via grpc @ localhost:37311 ... [grpc-default-executor-18] INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:654 - Connecting via grpc @ localhost:42839 ... [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 event time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 processing time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 event time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 processing time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 event time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 processing time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 event time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 processing time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 event time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 processing time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 event time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 processing time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 event time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 processing time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 event time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 processing time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 event time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 processing time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 event time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 processing time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Registering task instances with consumers. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Starting consumer multiplexer. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Entering run loop. [Samza StreamProcessor Container Thread-0] WARN org.apache.samza.processor.StreamProcessor - Received container start notification for container: org.apache.samza.container.SamzaContainer@1f5fc98d in stream processor: 1. [grpc-default-executor-18] INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService - getProcessBundleDescriptor request with id 34-5 [grpc-default-executor-18] INFO org.apache.beam.runners.fnexecution.data.GrpcDataService - Beam Fn Data client connected. [grpc-default-executor-18] INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:654 - Connecting via grpc @ localhost:42607 ... [grpc-default-executor-18] INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService - getProcessBundleDescriptor request with id 34-4 [grpc-default-executor-18] INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService - getProcessBundleDescriptor request with id 34-2 [grpc-default-executor-15] INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService - getProcessBundleDescriptor request with id 34-6 [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 event time timers in memory [grpc-default-executor-18] INFO org.apache.beam.runners.fnexecution.control.FnApiControlClientPoolService - getProcessBundleDescriptor request with id 34-3 [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 event time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 event time timers in memory [grpc-default-executor-17] INFO <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/ws/src/sdks/go/pkg/beam/core/runtime/harness/harness.go>:654 - Connecting via grpc @ localhost:44997 ... [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.samza.runtime.SamzaTimerInternalsFactory - Loaded 0 event time timers in memory [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.system.SystemConsumers - End of stream reached for partition: SystemStreamPartition [n14, n14, 0] [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.system.SystemConsumers - End of stream reached for partition: SystemStreamPartition [n10, n10, 0] [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.system.SystemConsumers - End of stream reached for partition: SystemStreamPartition [n1, n1, 0] [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.system.SystemConsumers - End of stream reached for partition: SystemStreamPartition [n8, n8, 0] [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.task.CoordinatorRequests - Shutdown has now been requested by tasks [Partition 0] [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.task.CoordinatorRequests - Shutdown requested. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Shutting down SamzaContainer. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Shutting down consumer multiplexer. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Shutting down task instance stream tasks. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Shutting down timer executor [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.fnexecution.control.DefaultJobBundleFactory - Closing environment urn: "beam:env:docker:v1" payload: "\n\026apache/beam_go_sdk:dev" capabilities: "beam:protocol:progress_reporting:v0" capabilities: "beam:protocol:multi_core_bundle_processing:v1" capabilities: "beam:transform:sdf_truncate_sized_restrictions:v1" capabilities: "beam:protocol:worker_status:v1" capabilities: "beam:version:sdk_base:go" capabilities: "beam:coder:bytes:v1" capabilities: "beam:coder:bool:v1" capabilities: "beam:coder:varint:v1" capabilities: "beam:coder:double:v1" capabilities: "beam:coder:string_utf8:v1" capabilities: "beam:coder:length_prefix:v1" capabilities: "beam:coder:kv:v1" capabilities: "beam:coder:iterable:v1" capabilities: "beam:coder:state_backed_iterable:v1" capabilities: "beam:coder:windowed_value:v1" capabilities: "beam:coder:global_window:v1" capabilities: "beam:coder:interval_window:v1" capabilities: "beam:coder:row:v1" capabilities: "beam:coder:nullable:v1" dependencies { type_urn: "beam:artifact:type:file:v1" type_payload: "\n\203\001/tmp/beam-artifact-staging/04540729e74e966a176580871a9813d74f76a6d055f0dd487604eb94838a0b5e/1-go-/tmp/worker-13-1654638525621650666" role_urn: "beam:artifact:role:go_worker_binary:v1" } [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.fnexecution.logging.GrpcLoggingService - 1 Beam Fn Logging clients still connected during shutdown. [Samza StreamProcessor Container Thread-0] WARN org.apache.beam.sdk.fn.data.BeamFnDataGrpcMultiplexer - Hanged up for unknown endpoint. [Samza StreamProcessor Container Thread-0] INFO org.apache.beam.runners.fnexecution.environment.DockerContainerEnvironment - Closing Docker container d1a689e3a5382151164e76deec99775fdd78d359252acee28ffd5206bd4e63fb. Logs: 2022/06/07 21:48:52 Provision info: pipeline_options:{fields:{key:"beam:option:app_name:v1" value:{string_value:"go-testemitpardoaftergbk"}} fields:{key:"beam:option:experiments:v1" value:{list_value:{values:{string_value:"beam_fn_api"}}}} fields:{key:"beam:option:go_options:v1" value:{struct_value:{fields:{key:"options" value:{struct_value:{fields:{key:"dataflow_worker_jar" value:{string_value:""}} fields:{key:"endpoint" value:{string_value:"localhost:49179"}} fields:{key:"environment_config" value:{string_value:"apache/beam_go_sdk:dev"}} fields:{key:"environment_type" value:{string_value:"DOCKER"}} fields:{key:"expansion_jar" value:{string_value:"[test:<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.40.0-SNAPSHOT.jar]"}}> fields:{key:"hookOrder" value:{string_value:"[\"default_remote_logging\"]"}} fields:{key:"hooks" value:{string_value:"{\"default_remote_logging\":null}"}} fields:{key:"project" value:{string_value:"apache-beam-testing"}} fields:{key:"region" value:{string_value:"us-central1"}} fields:{key:"runner" value:{string_value:"samza"}} fields:{key:"staging_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test"}} fields:{key:"temp_location" value:{string_value:"gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test"}} fields:{key:"test.paniconexit0" value:{string_value:"true"}} fields:{key:"test.timeout" value:{string_value:"2h0m0s"}} fields:{key:"test.v" value:{string_value:"true"}}}}}}}} fields:{key:"beam:option:parallelism:v1" value:{number_value:-1}} fields:{key:"beam:option:retain_docker_containers:v1" value:{bool_value:false}} fields:{key:"beam:option:runner:v1" value:{string_value:""}}} logging_endpoint:{url:"localhost:42839"} artifact_endpoint:{url:"localhost:35537"} control_endpoint:{url:"localhost:37311"} dependencies:{type_urn:"beam:artifact:type:file:v1" type_payload:"\n\x83\x01/tmp/beam-artifact-staging/04540729e74e966a176580871a9813d74f76a6d055f0dd487604eb94838a0b5e/1-go-/tmp/worker-13-1654638525621650666" role_urn:"beam:artifact:role:go_worker_binary:v1"} 2022/06/07 21:48:52 Initializing Go harness: /opt/apache/beam/boot --id=34-1 --provision_endpoint=localhost:46149 2022/06/07 21:48:52 Downloaded: /tmp/staged/1-worker-13-1654638525621650666 (sha256: eadcba6cd49688cf140bd6fafaa974f3f93685c8358087415af051b03083b28a, size: 25554920) Worker exited successfully! 2022/06/07 21:48:53 control response channel closed Remote logging shutting down. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Shutting down task instance table manager. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Shutting down container storage manager. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.storage.ContainerStorageManager - Shutdown complete [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Shutting down host statistics monitor. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Shutting down producer multiplexer. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Shutting down offset manager. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Shutting down metrics reporters. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Shutting down JVM metrics. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Shutting down admin multiplexer. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.container.SamzaContainer - Shutdown complete. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.processor.StreamProcessor - Container: org.apache.samza.container.SamzaContainer@1f5fc98d stopped. Stopping the stream processor: 1. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.processor.StreamProcessor - Ignoring onJobModelExpired invocation since the current state is STOPPING and not in [RUNNING, STARTED, IN_REBALANCE]. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.processor.StreamProcessor - Shutting down the executor service of the stream processor: 1. [Samza StreamProcessor Container Thread-0] WARN org.apache.samza.container.SamzaContainer - Shutdown is no-op since the container is already in state: STOPPED [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.processor.StreamProcessor - Waiting 30000 ms for the container: org.apache.samza.container.SamzaContainer@1f5fc98d to shutdown. [Samza StreamProcessor Container Thread-0] INFO org.apache.samza.processor.StreamProcessor - Shutdown status of container: org.apache.samza.container.SamzaContainer@1f5fc98d for stream processor: 1 is: true. [samza-runner-job-invoker-5] INFO org.apache.beam.runners.samza.SamzaPipelineResult - Pipeline finished. Final state: DONE [samza-runner-job-invoker-7] WARN org.apache.beam.runners.samza.SamzaPortablePipelineResult - Collecting monitoring infos is not implemented yet in Samza portable runner. [samza-runner-job-invoker-7] WARN org.apache.beam.runners.jobsubmission.InMemoryJobService - Failed to remove job staging directory for token go-testemitpardoaftergbk_2893735a-5030-4343-95b0-506c935352a4. java.io.FileNotFoundException: No files found for spec: /tmp/beam-artifact-staging/04540729e74e966a176580871a9813d74f76a6d055f0dd487604eb94838a0b5e/* in working directory <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/ws/src/.> at org.apache.beam.sdk.io.LocalFileSystem.matchOne(LocalFileSystem.java:307) at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:105) at org.apache.beam.sdk.io.LocalFileSystem.match(LocalFileSystem.java:98) at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:124) at org.apache.beam.sdk.io.FileSystems.matchResources(FileSystems.java:217) at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService$1.removeStagedArtifacts(ArtifactStagingService.java:180) at org.apache.beam.runners.fnexecution.artifact.ArtifactStagingService.removeStagedArtifacts(ArtifactStagingService.java:115) at org.apache.beam.runners.jobsubmission.JobServerDriver.lambda$createJobService$0(JobServerDriver.java:67) at org.apache.beam.runners.jobsubmission.InMemoryJobService.lambda$run$0(InMemoryJobService.java:261) at org.apache.beam.runners.jobsubmission.JobInvocation.setState(JobInvocation.java:249) at org.apache.beam.runners.jobsubmission.JobInvocation.access$200(JobInvocation.java:51) at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:115) at org.apache.beam.runners.jobsubmission.JobInvocation$1.onSuccess(JobInvocation.java:101) at org.apache.beam.vendor.guava.v26_0_jre.com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1058) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2022/06/07 21:48:54 Job state: DONE [grpc-default-executor-18] INFO org.apache.beam.runners.jobsubmission.InMemoryJobService - Getting job metrics for go0testemitpardoaftergbk-jenkins-0607214850-570bbefb_092deab1-e2f5-4e9f-a2ee-bcdd5aa778ca [grpc-default-executor-18] INFO org.apache.beam.runners.jobsubmission.InMemoryJobService - Finished getting job metrics for go0testemitpardoaftergbk-jenkins-0607214850-570bbefb_092deab1-e2f5-4e9f-a2ee-bcdd5aa778ca --- PASS: TestEmitParDoAfterGBK (9.13s) PASS ok github.com/apache/beam/sdks/v2/go/test/regression 211.999s FAIL $ TEST_EXIT_CODE=0 $ cd ../.. $ exit 1 $ exit 1 > Task :sdks:go:test:samzaValidatesRunner FAILED FAILURE: Build failed with an exception. * Where: Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/ws/src/sdks/go/test/build.gradle'> line: 95 * What went wrong: Execution failed for task ':sdks:go:test:samzaValidatesRunner'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings Execution optimizations have been disabled for 2 invalid unit(s) of work during this build to ensure correctness. Please consult deprecation warnings for more details. BUILD FAILED in 38m 10s 120 actionable tasks: 85 executed, 28 from cache, 7 up-to-date Publishing build scan... https://gradle.com/s/wezivumcuux7w Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
