See <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/523/display/redirect?page=changes>
Changes: [noreply] Merge pull request #12864: [BEAM-10650] Windmill implementation for [chamikaramj] Updates Pub/Sub runner API transformation to preserve None values ------------------------------------------ [...truncated 36.68 KB...] > Task :sdks:java:testing:load-tests:compileJava Note: Some input files use unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details. > Task :sdks:java:testing:load-tests:classes > Task :sdks:java:testing:load-tests:jar > Task :runners:google-cloud-dataflow-java:****:legacy-****:shadowJar > Task :sdks:java:testing:load-tests:run Oct 18, 2020 12:09:07 PM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$StagingLocationFactory create INFO: No stagingLocation provided, falling back to gcpTempLocation Oct 18, 2020 12:09:08 PM org.apache.beam.runners.dataflow.DataflowRunner fromOptions INFO: PipelineOptions.filesToStage was not specified. Defaulting to files from the classpath: will stage 180 files. Enable logging at DEBUG level to see which files will be staged. Oct 18, 2020 12:09:08 PM org.apache.beam.sdk.Pipeline validate WARNING: The following transforms do not have stable unique names: ParDo(TimeMonitor) Oct 18, 2020 12:09:09 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Executing pipeline on the Dataflow Service, which will have billing implications related to Google Compute Engine usage and other Google Cloud Services. Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Uploading 181 files from PipelineOptions.filesToStage to staging location to prepare for execution. Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes forFileToStage INFO: Staging custom dataflow-****.jar as beam-runners-google-cloud-dataflow-java-legacy-****-2.26.0-SNAPSHOT-eFeiYkh6Kxcy4JX92dJrmaVDpjrV3Fs3KrFWm7LeB4s.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.26.0-SNAPSHOT-Mnbf_mb5smKp_apFEi5uhhp8wcbSY3wxREE7HKQva54.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.26.0-SNAPSHOT-afhrbBMg34iSSR9pFz9A0_19K_aQHwUnS_I3Hcipx_I.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.26.0-SNAPSHOT-Iv-fmtlrVfzn2ft_ApaLhwnJ1wCETo6Ej2bkgWt_RdY.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/sdks/java/io/kafka/build/libs/beam-sdks-java-io-kafka-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kafka-2.26.0-SNAPSHOT-mVu-61o9WyC2IKwORF0rD4U1-jTB65mMoHsy4oVqHr0.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.26.0-SNAPSHOT-4SWr9Da1Q8Sbz5MZ-VUB0tGlX35jbdoNwojopmnRnmQ.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.26.0-SNAPSHOT-0mJD5Y0kwUyp5FAsh7Gn5mArmjmgisgCR24MfV31ooM.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.26.0-SNAPSHOT-CAmW9Z0TJc3HeHMXpKqWvPvAL7m-uamdBl2s6_vP3rA.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.26.0-SNAPSHOT-zqvKqAxW8PGUrXLXuOz302AZzlvv3GyHw4d63GJg-50.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/sdks/java/testing/load-tests/build/libs/beam-sdks-java-load-tests-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-load-tests-2.26.0-SNAPSHOT-McVGWhgIYLjg0Lk5kmbzv94ViX5YOmFD0tXwAI7BHoQ.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.26.0-SNAPSHOT-L7sX0ZG-d8Zxq1amMCsCYJ-gE5w5zQbeyVVC_QrVmBM.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.26.0-SNAPSHOT-1YPWDaeZGVUV9-01D31dgtt5bQYHwJV9FYstTf6fjFY.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.26.0-SNAPSHOT-vtdBOj4VgDZ8zZtgwpJXnb-kEier1uoyxh8hqEMPMBI.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/sdks/java/io/kinesis/build/libs/beam-sdks-java-io-kinesis-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-kinesis-2.26.0-SNAPSHOT-QecAXRo2NrSTwzLBSr8ze13zo_BWb2Ow5FX_chDd1OI.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.26.0-SNAPSHOT-ERT-pcXOYbJX7MZYUqqzv59cI3_9w337DSVAdPhyuCg.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.26.0-SNAPSHOT-mo6_8zQwU86MGI0uUYPPi8r75j7rCc_2Wd_cCmkENjY.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/sdks/java/io/synthetic/build/libs/beam-sdks-java-io-synthetic-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-synthetic-2.26.0-SNAPSHOT-AEUJF3rOf-BnzsASlLqE0bfcA5h6m6FcnX0jQkxnuFg.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.26.0-SNAPSHOT-lK0ucSrg5-GiWAq0m2Vb_YatSkfE9rntW6r8FoFViTs.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/runners/google-cloud-dataflow-java/****/legacy-****/build/libs/beam-runners-google-cloud-dataflow-java-legacy-****-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-****-2.26.0-SNAPSHOT-eFeiYkh6Kxcy4JX92dJrmaVDpjrV3Fs3KrFWm7LeB4s.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.26.0-SNAPSHOT-dGDlICcEF-ji5M1DdYYsmiNkHM2hue1oqIlrw7EDUAY.jar Oct 18, 2020 12:09:12 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/model/job-management/build/libs/beam-model-job-management-2.26.0-SNAPSHOT.jar> to gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.26.0-SNAPSHOT-R1FkOh9l__U2K8fKqVpvaM3jcput6O-szQYsMoYQPeo.jar Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.util.PackageUtil stageClasspathElements INFO: Staging files complete: 161 files cached, 20 files newly uploaded in 1 seconds Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s1 Oct 18, 2020 12:09:13 PM org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource split INFO: Split into 20 bundles of sizes: [org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@76db540e, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@10358c32, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2af4129d, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@3a175162, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@1640190a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2a2843ec, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2042ccce, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@20de05e5, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@36c0d0bd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@5e1fc2aa, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@688d411b, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@382c90c2, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@859ea42, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@28737371, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@2af46afd, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@760245e1, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@31ceba99, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@295bf2a, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@bbf9e07, org.apache.beam.sdk.io.synthetic.SyntheticUnboundedSource@6ff6efdc] Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Read input/StripIds as step s2 Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding ParDo(TimeMonitor) as step s3 Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding ParDo(ByteMonitor) as step s4 Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 0 as step s5 Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 1 as step s6 Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 2 as step s7 Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 3 as step s8 Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 4 as step s9 Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 5 as step s10 Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 6 as step s11 Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 7 as step s12 Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 8 as step s13 Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Step: 9 as step s14 Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding ParDo(TimeMonitor)2 as step s15 Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/loadtests/staging/ Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <91247 bytes, hash d2d9f724f89cceb0a698ff7640d8dfe39aed4ed84e9d2e4e15dea65cf5260fa4> to gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-0tn3JPiczrCmmP92QNjf45rtTthOnS5OFd6mXPUmD6Q.pb Oct 18, 2020 12:09:13 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.26.0-SNAPSHOT Oct 18, 2020 12:09:15 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-10-18_05_09_13-9723269108994568233?project=apache-beam-testing Oct 18, 2020 12:09:15 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2020-10-18_05_09_13-9723269108994568233 Oct 18, 2020 12:09:15 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2020-10-18_05_09_13-9723269108994568233 Oct 18, 2020 12:09:18 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2020-10-18T12:09:17.193Z: The workflow name is not a valid Cloud Label. Labels applied to Cloud resources (such as GCE Instances) for monitoring will be labeled with this modified job name: load0tests0java0dataflow0streaming0pardo01-jenkins-1018120-d1af. For the best monitoring experience, please name your job with a valid Cloud Label. For details, see: https://cloud.google.com/compute/docs/labeling-resources#restrictions Oct 18, 2020 12:09:20 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:20.570Z: Worker configuration: n1-standard-4 in us-central1-f. Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.063Z: Expanding CoGroupByKey operations into optimizable parts. Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.073Z: Expanding SplittableProcessKeyed operations into optimizable parts. Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.076Z: Expanding GroupByKey operations into streaming Read/Write steps Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.078Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.096Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.098Z: Fusing consumer Read input/StripIds into Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.100Z: Fusing consumer ParDo(TimeMonitor) into Read input/StripIds Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.102Z: Fusing consumer ParDo(ByteMonitor) into ParDo(TimeMonitor) Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.105Z: Fusing consumer Step: 0 into ParDo(ByteMonitor) Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.107Z: Fusing consumer Step: 1 into Step: 0 Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.109Z: Fusing consumer Step: 2 into Step: 1 Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.112Z: Fusing consumer Step: 3 into Step: 2 Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.115Z: Fusing consumer Step: 4 into Step: 3 Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.117Z: Fusing consumer Step: 5 into Step: 4 Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.120Z: Fusing consumer Step: 6 into Step: 5 Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.122Z: Fusing consumer Step: 7 into Step: 6 Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.124Z: Fusing consumer Step: 8 into Step: 7 Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.127Z: Fusing consumer Step: 9 into Step: 8 Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.129Z: Fusing consumer ParDo(TimeMonitor)2 into Step: 9 Oct 18, 2020 12:09:22 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:21.365Z: Starting 5 ****s... Oct 18, 2020 12:09:24 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:24.005Z: Executing operation Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds+Read input/StripIds+ParDo(TimeMonitor)+ParDo(ByteMonitor)+Step: 0+Step: 1+Step: 2+Step: 3+Step: 4+Step: 5+Step: 6+Step: 7+Step: 8+Step: 9+ParDo(TimeMonitor)2 Oct 18, 2020 12:09:50 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:50.137Z: Your project already contains 100 Dataflow-created metric descriptors, so new user metrics of the form custom.googleapis.com/* will not be created. However, all user metrics are also available in the metric dataflow.googleapis.com/job/user_counter. If you rely on the custom metrics, you can delete old / unused metric descriptors. See https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Oct 18, 2020 12:09:54 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:09:54.532Z: Worker configuration: n1-standard-4 in us-central1-f. Oct 18, 2020 12:10:04 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:10:04.171Z: Workers have started successfully. Oct 18, 2020 12:11:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:11:04.958Z: Finished operation Read input/DataflowRunner.StreamingUnboundedRead.ReadWithIds+Read input/StripIds+ParDo(TimeMonitor)+ParDo(ByteMonitor)+Step: 0+Step: 1+Step: 2+Step: 3+Step: 4+Step: 5+Step: 6+Step: 7+Step: 8+Step: 9+ParDo(TimeMonitor)2 Oct 18, 2020 12:11:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:11:05.082Z: Cleaning up. Oct 18, 2020 12:11:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:11:05.197Z: Worker pool stopped. Oct 18, 2020 12:11:05 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:11:05.313Z: Stopping **** pool... Oct 18, 2020 12:11:51 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-10-18T12:11:49.554Z: Worker pool stopped. Oct 18, 2020 12:11:57 PM org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState INFO: Job 2020-10-18_05_09_13-9723269108994568233 finished with status DONE. Load test results for test (ID): c9c7c73e-272a-4d62-bf79-77adfd30cb0c and timestamp: 2020-10-18T12:09:08.629000000Z: Metric: Value: dataflow_runtime_sec 18.725 dataflow_total_bytes_count 1.999998E9 Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD SUCCESSFUL in 3m 40s 73 actionable tasks: 47 executed, 26 from cache Publishing build scan... https://gradle.com/s/vhoushpiywytq Build step 'Invoke Gradle script' changed build result to SUCCESS No JDK named ‘JDK 1.8 (latest)’ found [beam_LoadTests_Java_ParDo_Dataflow_Streaming] $ /bin/bash -xe /tmp/jenkins1336684827001917580.sh + echo '*** Load test: ParDo 2GB 100 byte records 200 times ***' *** Load test: ParDo 2GB 100 byte records 200 times *** [Gradle] - Launching build. No JDK named ‘JDK 1.8 (latest)’ found [src] $ <https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src/gradlew> -PloadTest.mainClass=org.apache.beam.sdk.loadtests.ParDoLoadTest -Prunner=:runners:google-cloud-dataflow-java -PwithDataflowWorkerJar=false '-PloadTest.args=--project=apache-beam-testing --region=us-central1 --appName=load_tests_Java_Dataflow_streaming_ParDo_2 --tempLocation=gs://temp-storage-for-perf-tests/loadtests --publishToBigQuery=true --bigQueryDataset=load_test --bigQueryTable=java_dataflow_streaming_ParDo_2 --influxMeasurement=java_streaming_pardo_2 --publishToInfluxDB=true --sourceOptions={"numRecords":20000000,"keySizeBytes":10,"valueSizeBytes":90} --iterations=200 --numberOfCounters=1 --numberOfCounterOperations=0 --numWorkers=5 --autoscalingAlgorithm=NONE --streaming=true --influxDatabase=beam_test_metrics --influxHost=http://10.128.0.96:8086 --inputWindowDurationSec=1200 --runner=DataflowRunner' --continue --max-****s=12 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g -Pdocker-pull-licenses :sdks:java:testing:load-tests:run > Task :buildSrc:compileJava NO-SOURCE > Task :buildSrc:compileGroovy UP-TO-DATE > Task :buildSrc:pluginDescriptors UP-TO-DATE > Task :buildSrc:processResources UP-TO-DATE > Task :buildSrc:classes UP-TO-DATE > Task :buildSrc:jar UP-TO-DATE > Task :buildSrc:assemble UP-TO-DATE > Task :buildSrc:spotlessGroovy UP-TO-DATE > Task :buildSrc:spotlessGroovyCheck UP-TO-DATE > Task :buildSrc:spotlessGroovyGradle UP-TO-DATE > Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE > Task :buildSrc:spotlessCheck UP-TO-DATE The message received from the daemon indicates that the daemon has disappeared. Build request sent: Build{id=9b52327e-d940-4234-be08-586f1096c21f, currentDir=<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src}> Attempting to read last messages from the daemon log... Daemon pid: 31703 log file: /home/jenkins/.gradle/daemon/6.6.1/daemon-31703.out.log ----- Last 20 lines from daemon log file - daemon-31703.out.log ----- 2020-10-18T12:12:00.855+0000 [DEBUG] [org.gradle.launcher.daemon.server.DefaultDaemonConnection] thread 245: Received non-IO message from client: Build{id=9b52327e-d940-4234-be08-586f1096c21f, currentDir=<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src}> 2020-10-18T12:12:00.855+0000 [INFO] [org.gradle.launcher.daemon.server.DefaultIncomingConnectionHandler] Received command: Build{id=9b52327e-d940-4234-be08-586f1096c21f, currentDir=<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src}.> 2020-10-18T12:12:00.855+0000 [DEBUG] [org.gradle.launcher.daemon.server.DefaultIncomingConnectionHandler] Starting executing command: Build{id=9b52327e-d940-4234-be08-586f1096c21f, currentDir=<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src}> with connection: socket connection from /127.0.0.1:36943 to /127.0.0.1:44326. 2020-10-18T12:12:00.855+0000 [ERROR] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] Command execution: started DaemonCommandExecution[command = Build{id=9b52327e-d940-4234-be08-586f1096c21f, currentDir=<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src},> connection = DefaultDaemonConnection: socket connection from /127.0.0.1:36943 to /127.0.0.1:44326] after 0.0 minutes of idle 2020-10-18T12:12:00.856+0000 [INFO] [org.gradle.launcher.daemon.server.DaemonRegistryUpdater] Marking the daemon as busy, address: [30d85593-c1ab-4b0f-b30b-62a62be2cff3 port:36943, addresses:[localhost/127.0.0.1]] 2020-10-18T12:12:00.856+0000 [DEBUG] [org.gradle.launcher.daemon.registry.PersistentDaemonRegistry] Marking busy by address: [30d85593-c1ab-4b0f-b30b-62a62be2cff3 port:36943, addresses:[localhost/127.0.0.1]] 2020-10-18T12:12:00.856+0000 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire exclusive lock on daemon addresses registry. 2020-10-18T12:12:00.856+0000 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry. 2020-10-18T12:12:00.857+0000 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry. 2020-10-18T12:12:00.857+0000 [DEBUG] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] resetting idle timer 2020-10-18T12:12:00.857+0000 [DEBUG] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] daemon is running. Sleeping until state changes. 2020-10-18T12:12:00.857+0000 [INFO] [org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy] Daemon is about to start building Build{id=9b52327e-d940-4234-be08-586f1096c21f, currentDir=<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src}.> Dispatching build started information... 2020-10-18T12:12:00.857+0000 [DEBUG] [org.gradle.launcher.daemon.server.SynchronizedDispatchConnection] thread 27: dispatching org.gradle.launcher.daemon.protocol.BuildStarted@6852bcf8 2020-10-18T12:12:00.857+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment] Configuring env variables: {PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games, INFLUXDB_USER=****, RUN_DISPLAY_URL=https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/523/display/redirect, HUDSON_HOME=/home/jenkins/jenkins-home, RUN_CHANGES_DISPLAY_URL=https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/523/display/redirect?page=changes, JOB_URL=https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/, HUDSON_COOKIE=7747af44-3821-4df6-84ef-8f4870dd5980, MAIL=/var/mail/jenkins, JENKINS_SERVER_COOKIE=14925adb602d4986, LOGNAME=jenkins, PWD=<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src,> JENKINS_URL=https://ci-beam.apache.org/, SHELL=/bin/bash, BUILD_TAG=jenkins-beam_LoadTests_Java_ParDo_Dataflow_Streaming-523, ROOT_BUILD_CAUSE=TIMERTRIGGER, BUILD_CAUSE_TIMERTRIGGER=true, OLDPWD=<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/src,> GIT_CHECKOUT_DIR=src, JENKINS_HOME=/home/jenkins/jenkins-home, sha1=master, NODE_NAME=apache-beam-jenkins-3, BUILD_DISPLAY_NAME=#523, JOB_DISPLAY_URL=https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/display/redirect, GIT_BRANCH=origin/master, SETUPTOOLS_USE_DISTUTILS=stdlib, SHLVL=1, WORKSPACE_TMP=<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/@tmp,> GIT_PREVIOUS_COMMIT=8b9dc0fef3a541a3502b5dba812da2cb05ab4a5b, INFLUXDB_USER_PASSWORD=****, BUILD_ID=523, LANG=en_US.UTF-8, XDG_SESSION_ID=2138, JOB_NAME=beam_LoadTests_Java_ParDo_Dataflow_Streaming, SPARK_LOCAL_IP=127.0.0.1, BUILD_CAUSE=TIMERTRIGGER, GIT_PREVIOUS_SUCCESSFUL_COMMIT=8b9dc0fef3a541a3502b5dba812da2cb05ab4a5b, NODE_LABELS=apache-beam-jenkins-3 beam, HUDSON_URL=https://ci-beam.apache.org/, WORKSPACE=<https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/ws/,> ROOT_BUILD_CAUSE_TIMERTRIGGER=true, _=/usr/bin/java, GIT_COMMIT=7c357e459d4bbf628dfa98d054930c05aef23e8b, EXECUTOR_NUMBER=1, HUDSON_SERVER_COOKIE=14925adb602d4986, SSH_CLIENT=3.94.149.17 55304 22, JOB_BASE_NAME=beam_LoadTests_Java_ParDo_Dataflow_Streaming, USER=jenkins, SSH_CONNECTION=3.94.149.17 55304 10.128.0.144 22, BUILD_NUMBER=523, BUILD_URL=https://ci-beam.apache.org/job/beam_LoadTests_Java_ParDo_Dataflow_Streaming/523/, GIT_URL=https://github.com/apache/beam.git, XDG_RUNTIME_DIR=/run/user/1017, HOME=/home/jenkins} 2020-10-18T12:12:00.858+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.LogToClient] About to start relaying all logs to the client via the connection. 2020-10-18T12:12:00.858+0000 [INFO] [org.gradle.launcher.daemon.server.exec.LogToClient] The client will now receive all logging from the daemon (pid: 31703). The daemon log file: /home/jenkins/.gradle/daemon/6.6.1/daemon-31703.out.log 2020-10-18T12:12:00.859+0000 [INFO] [org.gradle.launcher.daemon.server.exec.LogAndCheckHealth] Starting 2nd build in daemon [uptime: 6 mins 39.933 secs, performance: 100%, GC rate: 0.01/s, heap usage: 2% of 2.9 GB] 2020-10-18T12:12:00.859+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.ExecuteBuild] The daemon has started executing the build. 2020-10-18T12:12:00.860+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.ExecuteBuild] Executing build with daemon context: DefaultDaemonContext[uid=b402f4fb-322c-4b7c-8eb3-2779c65cbb6f,javaHome=/usr/lib/jvm/java-8-openjdk-amd64,daemonRegistryDir=/home/jenkins/.gradle/daemon,pid=31703,idleTimeout=10800000,priority=NORMAL,daemonOpts=-Xmx4g,-Dfile.encoding=UTF-8,-Duser.country=US,-Duser.language=en,-Duser.variant] Daemon vm is shutting down... The daemon has exited normally or was terminated in response to a user interrupt. ----- End of the daemon log ----- FAILURE: Build failed with an exception. * What went wrong: Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed) * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org > Task :buildSrc:pluginUnderTestMetadata Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure No JDK named ‘JDK 1.8 (latest)’ found --------------------------------------------------------------------- To unsubscribe, e-mail: builds-unsubscr...@beam.apache.org For additional commands, e-mail: builds-h...@beam.apache.org