See <https://ci-beam.apache.org/job/beam_PerformanceTests_TFRecordIOIT/7084/display/redirect?page=changes>
Changes: [noreply] [Website] change svg to png (#24268) [noreply] Fix error messages in cred rotation email (#24474) [noreply] Add integer to NUMERIC and BIGNUMERIC conversion support (#24447) [noreply] Update jackson dep. (#24445) [noreply] Reduce calls to FileSystem.match and API calls in FileSystem._list [noreply] Capture full response context to provide complete error information [noreply] Pandas 1.5 support (#23973) ------------------------------------------ [...truncated 364.27 KB...] work action null (Thread[Execution **** Thread 5,5,main]) completed. Took 0.0 secs. Resolve mutations for :sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** Thread 2,5,main]) started. Resolve mutations for :sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** Thread 2,5,main]) completed. Took 0.0 secs. :sdks:java:io:file-based-io-tests:integrationTest (Thread[Execution **** Thread 7,5,main]) started. producer locations for task group 0 (Thread[Execution **** Thread 2,5,main]) started. producer locations for task group 0 (Thread[Execution **** Thread 2,5,main]) completed. Took 0.0 secs. Gradle Test Executor 1 started executing tests. > Task :sdks:java:io:file-based-io-tests:integrationTest Custom actions are attached to task ':sdks:java:io:file-based-io-tests:integrationTest'. Build cache key for task ':sdks:java:io:file-based-io-tests:integrationTest' is 2511eabf174941ab7ae6a7031d873d92 Task ':sdks:java:io:file-based-io-tests:integrationTest' is not up-to-date because: Task.upToDateWhen is false. Starting process 'Gradle Test Executor 1'. Working directory: /home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TFRecordIOIT/src/sdks/java/io/file-based-io-tests Command: /usr/lib/jvm/java-8-openjdk-amd64/bin/java -DbeamTestPipelineOptions=["--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_TFRecordIOIT/7084/","--bigQueryDataset=beam_performance","--bigQueryTable=tfrecordioit_results","--influxMeasurement=tfrecordioit_results","--numberOfRecords=18000000","--expectedHash=543104423f8b6eb097acb9f111c19fe4","--datasetSize=1019380000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--****HarnessContainerImage=","--dataflowWorkerJar=/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TFRecordIOIT/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.45.0-SNAPSHOT.jar","--region=us-central1";] -Djava.security.manager=****.org.gradle.process.internal.****.child.BootstrapSecurityManager -Dorg.gradle.internal.****.tmpdir=/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TFRecordIOIT/src/sdks/java/io/file-based-io-tests/build/tmp/integrationTest/work -Dorg.gradle.native=false -Xmx2g -Dfile.encoding=UTF-8 -Duser.country=US -Duser.language=en -Duser.variant -ea -cp /home/jenkins/.gradle/caches/7.5.1/****Main/gradle-****.jar ****.org.gradle.process.internal.****.GradleWorkerMain 'Gradle Test Executor 1' Successfully started process 'Gradle Test Executor 1' org.apache.beam.sdk.io.tfrecord.TFRecordIOIT STANDARD_ERROR SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TFRecordIOIT/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.45.0-SNAPSHOT.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-jdk14/1.7.30/d35953dd2fe54ebe39fdf18cfd82fe6eb35b25ed/slf4j-jdk14-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-reload4j/1.7.36/db708f7d959dee1857ac524636e85ecf2e1781c1/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-simple/1.7.30/e606eac955f55ecf1d8edcccba04eb8ac98088dd/slf4j-simple-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/jenkins/.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.30/c21f55139d8141d2231214fb1feaf50a1edca95e/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.JDK14LoggerFactory] org.apache.beam.sdk.io.tfrecord.TFRecordIOIT > writeThenReadAll STANDARD_ERROR Dec 02, 2022 2:22:03 AM org.apache.beam.runners.dataflow.DataflowRunner validateSdkContainerImageOptions WARNING: Prefer --sdkContainerImage over deprecated legacy option --****HarnessContainerImage. Dec 02, 2022 2:22:04 AM org.apache.beam.sdk.extensions.gcp.options.GcpOptions$GcpTempLocationFactory tryCreateDefaultBucket INFO: No tempLocation specified, attempting to use default bucket: dataflow-staging-us-central1-844138762903 Dec 02, 2022 2:22:05 AM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse WARNING: Request failed with code 409, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://storage.googleapis.com/storage/v1/b?predefinedAcl=projectPrivate&predefinedDefaultObjectAcl=projectPrivate&project=apache-beam-testing. Dec 02, 2022 2:22:05 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create INFO: No stagingLocation provided, falling back to gcpTempLocation Dec 02, 2022 2:22:06 AM org.apache.beam.runners.dataflow.DataflowRunner fromOptions INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 376 files. Enable logging at DEBUG level to see which files will be staged. Dec 02, 2022 2:22:06 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services. Dec 02, 2022 2:22:10 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Uploading 377 files from PipelineOptions.filesToStage to staging location to prepare for execution. Dec 02, 2022 2:22:10 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading /tmp/test1502109829130941969.zip to gs://dataflow-staging-us-central1-844138762903/temp/staging/test-m0S-Gt3YlqHmJrzoHTp0yOEZaaWNCkMYQoRZMV-w4G8.jar Dec 02, 2022 2:22:10 AM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.45.0-SNAPSHOT-ZweCSXIFNS3bjV_ZnIgrpIRQDQtkpI0q13JBVnE9zjM.jar Dec 02, 2022 2:22:10 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading /tmp/test4836522183054590539.zip to gs://dataflow-staging-us-central1-844138762903/temp/staging/test-x4FhzzgRTyk8oQOgy6xZMlxBu1xgGJGgvGsX91v8HLA.jar Dec 02, 2022 2:22:11 AM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Staging files complete: 375 files cached, 2 files newly uploaded in 0 seconds Dec 02, 2022 2:22:11 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging portable pipeline proto to gs://dataflow-staging-us-central1-844138762903/temp/staging/ Dec 02, 2022 2:22:11 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <233804 bytes, hash 92572a69d74e37e1e114bbb2d059201d4a899e5b85b1b0618b7e6a6a9cd1f856> to gs://dataflow-staging-us-central1-844138762903/temp/staging/pipeline-klcqaddON-HhFLuy0FkgHUqJnluFsbBhi35qapzR-FY.pb Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Generate sequence/Read(BoundedCountingSource) as step s1 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Produce text lines as step s2 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Transform strings to bytes/Map as step s3 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Record time before writing as step s4 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Write content to files/Write/RewindowIntoGlobal/Window.Assign as step s5 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles as step s6 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten as step s7 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten as step s8 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum as step s9 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections as step s10 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) as step s11 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView as step s12 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) as step s13 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) as step s14 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map as step s15 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Finalize as step s16 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key as step s17 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign as step s18 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey as step s19 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable as step s20 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map as step s21 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.45.0-SNAPSHOT Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2022-12-01_18_22_14-12131931266466723510?project=apache-beam-testing Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2022-12-01_18_22_14-12131931266466723510 Dec 02, 2022 2:22:14 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel --region=us-central1 2022-12-01_18_22_14-12131931266466723510 Dec 02, 2022 2:22:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:37.233Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Dec 02, 2022 2:22:38 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:37.439Z: Worker configuration: e2-standard-2 in us-central1-f. Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.064Z: Expanding CoGroupByKey operations into optimizable parts. Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.178Z: Expanding GroupByKey operations into optimizable parts. Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.212Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.356Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.384Z: Unzipping flatten s10 for input s6.writtenRecords Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.413Z: Fusing unzipped copy of Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow), through flatten Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into producer Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.437Z: Fusing consumer Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) into Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.472Z: Fusing consumer Produce text lines into Generate sequence/Read(BoundedCountingSource) Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.506Z: Fusing consumer Transform strings to bytes/Map into Produce text lines Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.535Z: Fusing consumer Record time before writing into Transform strings to bytes/Map Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.568Z: Fusing consumer Write content to files/Write/RewindowIntoGlobal/Window.Assign into Record time before writing Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.604Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into Write content to files/Write/RewindowIntoGlobal/Window.Assign Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.642Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.701Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.738Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.768Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten into Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.802Z: Fusing consumer Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum into Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.833Z: Fusing consumer Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) into Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource) Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.865Z: Fusing consumer Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map into Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous) Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.898Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Finalize into Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.930Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key into Write content to files/Write/FinalizeTempFileBundles/Finalize Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.957Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:39.993Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:40.018Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:40.053Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:40.078Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow Dec 02, 2022 2:22:40 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:40.105Z: Fusing consumer Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map into Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable Dec 02, 2022 2:22:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:40.466Z: Executing operation Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create Dec 02, 2022 2:22:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:40.545Z: Starting 5 ****s in us-central1-f... Dec 02, 2022 2:22:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:41.462Z: Finished operation Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create Dec 02, 2022 2:22:42 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:22:41.624Z: Executing operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Transform strings to bytes/Map+Record time before writing+Write content to files/Write/RewindowIntoGlobal/Window.Assign+Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write Dec 02, 2022 2:23:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:23:23.053Z: Autoscaling: Raised the number of ****s to 5 based on the rate of progress in the currently running stage(s). Dec 02, 2022 2:23:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:23:55.311Z: Workers have started successfully. Dec 02, 2022 2:24:20 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:19.336Z: All ****s have finished the startup processes and began to receive work requests. Dec 02, 2022 2:24:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:46.676Z: Finished operation Generate sequence/Read(BoundedCountingSource)+Produce text lines+Transform strings to bytes/Map+Record time before writing+Write content to files/Write/RewindowIntoGlobal/Window.Assign+Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write Dec 02, 2022 2:24:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:46.825Z: Executing operation Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close Dec 02, 2022 2:24:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:46.897Z: Finished operation Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close Dec 02, 2022 2:24:48 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:46.979Z: Executing operation Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum+Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) Dec 02, 2022 2:24:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:49.453Z: Finished operation Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write content to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write content to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write content to files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum+Write content to files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow) Dec 02, 2022 2:24:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:49.607Z: Executing operation Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections/Unzipped-1 Dec 02, 2022 2:24:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:49.700Z: Finished operation Write content to files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections/Unzipped-1 Dec 02, 2022 2:24:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:49.873Z: Executing operation Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView Dec 02, 2022 2:24:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:50.000Z: Finished operation Write content to files/Write/GatherTempFileResults/View.AsList/CreateDataflowView Dec 02, 2022 2:24:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:50.143Z: Executing operation Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create Dec 02, 2022 2:24:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:50.317Z: Finished operation Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create Dec 02, 2022 2:24:50 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:50.474Z: Executing operation Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+Write content to files/Write/FinalizeTempFileBundles/Finalize+Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write Dec 02, 2022 2:24:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:52.858Z: Finished operation Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Create.Values/Read(CreateSource)+Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)+Write content to files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map+Write content to files/Write/FinalizeTempFileBundles/Finalize+Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key+Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write Dec 02, 2022 2:24:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:52.924Z: Executing operation Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close Dec 02, 2022 2:24:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:52.980Z: Finished operation Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close Dec 02, 2022 2:24:53 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:53.062Z: Executing operation Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map Dec 02, 2022 2:24:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:54.432Z: Finished operation Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write content to files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map Dec 02, 2022 2:24:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:54.609Z: Cleaning up. Dec 02, 2022 2:24:54 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2022-12-02T02:24:54.679Z: Stopping **** pool... Build timed out (after 100 minutes). Marking the build as aborted. FATAL: command execution failed hudson.remoting.ChannelClosedException: Channel "hudson.remoting.Channel@7ed7cc52:apache-beam-jenkins-7": Remote call on apache-beam-jenkins-7 failed. The channel is closing down or has closed down at hudson.remoting.Channel.call(Channel.java:993) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:285) at com.sun.proxy.$Proxy147.isAlive(Unknown Source) at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1215) at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1207) at hudson.Launcher$ProcStarter.join(Launcher.java:524) at hudson.plugins.gradle.Gradle.perform(Gradle.java:317) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:814) at hudson.model.Build$BuildExecution.build(Build.java:199) at hudson.model.Build$BuildExecution.doRun(Build.java:164) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:522) at hudson.model.Run.execute(Run.java:1896) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44) at hudson.model.ResourceController.execute(ResourceController.java:101) at hudson.model.Executor.run(Executor.java:442) Caused by: java.io.IOException at hudson.remoting.Channel.close(Channel.java:1491) at hudson.remoting.Channel.close(Channel.java:1447) at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:902) at hudson.slaves.SlaveComputer.access$100(SlaveComputer.java:111) at hudson.slaves.SlaveComputer$2.run(SlaveComputer.java:782) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:68) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure ERROR: apache-beam-jenkins-7 is offline; cannot locate jdk_1.8_latest --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
