See <https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/936/display/redirect>
Changes: ------------------------------------------ [...truncated 3.51 MB...] Mar 30, 2020 11:17:09 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CalculateUserScores/ExtractUserScore/MapElements/Map as step s19 Mar 30, 2020 11:17:09 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CalculateUserScores/ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey as step s20 Mar 30, 2020 11:17:09 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding CalculateUserScores/ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues as step s21 Mar 30, 2020 11:17:09 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/ConvertToRow as step s22 [INFO] Downloaded from test.release: https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-sdks-java-harness/2.21.0-SNAPSHOT/beam-sdks-java-harness-2.21.0-20200330.071430-32.jar (56 MB at 14 MB/s) Mar 30, 2020 11:17:10 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/BigQueryIO.Write/PrepareWrite/ParDo(Anonymous) as step s23 [INFO] [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ word-count-beam --- Mar 30, 2020 11:17:10 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/CreateTables/ParDo(CreateTables) as step s24 Mar 30, 2020 11:17:10 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/ShardTableWrites as step s25 [WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent! [INFO] skip non existing resourceDirectory /tmp/groovy-generated-8057430337473299476-tmpdir/word-count-beam/src/main/resources [INFO] [INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ word-count-beam --- Mar 30, 2020 11:17:10 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/TagWithUniqueIds as step s26 Mar 30, 2020 11:17:10 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/Window.Into()/Window.Assign as step s27 Mar 30, 2020 11:17:10 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey as step s28 Mar 30, 2020 11:17:10 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/ExpandIterable as step s29 Mar 30, 2020 11:17:10 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/GlobalWindow/Window.Assign as step s30 Mar 30, 2020 11:17:10 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/StreamingWrite as step s31 Mar 30, 2020 11:17:10 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging pipeline description to gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/ [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- exec-maven-plugin:1.6.0:java (default-cli) @ word-count-beam --- Mar 30, 2020 11:17:10 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <81842 bytes, hash 8OvzS5Ry0A9jxOGPBpI49A> to gs://temp-storage-for-release-validation-tests/nightly-snapshot-validation/tmp/staging/pipeline-8OvzS5Ry0A9jxOGPBpI49A.pb Mar 30, 2020 11:17:11 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Dataflow SDK version: 2.21.0-SNAPSHOT fullTopicName projects/apache-beam-testing/topics/java_mobile_gaming_topic Injecting to topic: projects/apache-beam-testing/topics/java_mobile_gaming_topic Starting Injector [+(AuburnKookaburra, num members: 7, starting at: 1585567031761, expires in: 27, robot: Robot-16)] [+(AntiqueBrassKangaroo, num members: 14, starting at: 1585567031762, expires in: 35, robot: Robot-0)] [+(ArmyGreenWombat, num members: 19, starting at: 1585567031762, expires in: 30, robot: null)] [+(FuchsiaQuokka, num members: 8, starting at: 1585567031762, expires in: 30, robot: null)] [+(AliceBlueBilby, num members: 13, starting at: 1585567031762, expires in: 25, robot: null)] [+(RubyPossum, num members: 6, starting at: 1585567031763, expires in: 33, robot: Robot-1)] [+(BattleshipGreyWallaby, num members: 8, starting at: 1585567031763, expires in: 35, robot: null)] [+(AquaWallaby, num members: 10, starting at: 1585567031763, expires in: 31, robot: null)] [+(BananaBilby, num members: 7, starting at: 1585567031764, expires in: 30, robot: null)] [+(AmaranthKangaroo, num members: 9, starting at: 1585567031764, expires in: 21, robot: null)] [+(AntiqueBrassEmu, num members: 17, starting at: 1585567031764, expires in: 24, robot: Robot-13)] [+(AntiqueBrassBandicoot, num members: 11, starting at: 1585567031765, expires in: 36, robot: null)] [+(AppleGreenMarmot, num members: 19, starting at: 1585567031765, expires in: 21, robot: null)] [+(ApricotMarmot, num members: 16, starting at: 1585567031765, expires in: 20, robot: Robot-16)] [+(AppleGreenCockatoo, num members: 11, starting at: 1585567031765, expires in: 25, robot: null)] DELAY(396127, 1) {timestamp_ms=1585566635000} late data for: user3_AuburnKookaburra,AuburnKookaburra,7,1585566635000,2020-03-30 04:17:11.769 Mar 30, 2020 11:17:12 AM org.apache.beam.runners.dataflow.options.DataflowPipelineOptions$DefaultGcpRegionFactory create WARNING: Region will default to us-central1. Future releases of Beam will require the user to set the region explicitly. https://cloud.google.com/compute/docs/regions-zones/regions-zones Mar 30, 2020 11:17:13 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobs/us-central1/2020-03-30_04_17_12-7445495913101283443?project=apache-beam-testing Mar 30, 2020 11:17:13 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Submitted job: 2020-03-30_04_17_12-7445495913101283443 Mar 30, 2020 11:17:13 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2020-03-30_04_17_12-7445495913101283443 Mar 30, 2020 11:17:17 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:16.047Z: Checking permissions granted to controller Service Account. Mar 30, 2020 11:17:23 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:22.798Z: Worker configuration: n1-standard-4 in us-central1-a. Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.467Z: Expanding CoGroupByKey operations into optimizable parts. Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.483Z: Expanding SplittableProcessKeyed operations into optimizable parts. Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.485Z: Expanding GroupByKey operations into streaming Read/Write steps Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.496Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.535Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.537Z: Fusing consumer CalculateUserScores/LeaderboardUserGlobalWindow/Window.Assign into ParseGameEvent Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.539Z: Fusing consumer CalculateTeamScores/LeaderboardTeamFixedWindows/Window.Assign into ParseGameEvent Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.541Z: Fusing consumer PubsubIO.Read/MapElements/Map into PubsubIO.Read/PubsubUnboundedSource Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.542Z: Fusing consumer ParseGameEvent into PubsubIO.Read/MapElements/Map Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.544Z: Fusing consumer CalculateTeamScores/ExtractTeamScore/MapElements/Map into CalculateTeamScores/LeaderboardTeamFixedWindows/Window.Assign Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.546Z: Fusing consumer CalculateTeamScores/ExtractTeamScore/Combine.perKey(SumInteger)/GroupByKey/WriteStream into CalculateTeamScores/ExtractTeamScore/MapElements/Map Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.548Z: Fusing consumer CalculateTeamScores/ExtractTeamScore/Combine.perKey(SumInteger)/Combine.GroupedValues into CalculateTeamScores/ExtractTeamScore/Combine.perKey(SumInteger)/GroupByKey/ReadStream Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.549Z: Fusing consumer WriteTeamScoreSums/ConvertToRow into CalculateTeamScores/ExtractTeamScore/Combine.perKey(SumInteger)/Combine.GroupedValues Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.551Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/PrepareWrite/ParDo(Anonymous) into WriteTeamScoreSums/ConvertToRow Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.553Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/CreateTables/ParDo(CreateTables) into WriteTeamScoreSums/BigQueryIO.Write/PrepareWrite/ParDo(Anonymous) Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.555Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/ShardTableWrites into WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/CreateTables/ParDo(CreateTables) Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.557Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/TagWithUniqueIds into WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/ShardTableWrites Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.558Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/Window.Into()/Window.Assign into WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/TagWithUniqueIds Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.560Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/WriteStream into WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/Window.Into()/Window.Assign Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.562Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/MergeBuckets into WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/ReadStream Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.564Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/ExpandIterable into WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/MergeBuckets Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.565Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/GlobalWindow/Window.Assign into WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/ExpandIterable Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.567Z: Fusing consumer WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/StreamingWrite into WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/GlobalWindow/Window.Assign Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.569Z: Fusing consumer CalculateUserScores/ExtractUserScore/MapElements/Map into CalculateUserScores/LeaderboardUserGlobalWindow/Window.Assign Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.571Z: Fusing consumer CalculateUserScores/ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/WriteStream into CalculateUserScores/ExtractUserScore/MapElements/Map Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.573Z: Fusing consumer CalculateUserScores/ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues into CalculateUserScores/ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/ReadStream Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.574Z: Fusing consumer WriteUserScoreSums/ConvertToRow into CalculateUserScores/ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.576Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/PrepareWrite/ParDo(Anonymous) into WriteUserScoreSums/ConvertToRow Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.578Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/CreateTables/ParDo(CreateTables) into WriteUserScoreSums/BigQueryIO.Write/PrepareWrite/ParDo(Anonymous) Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.580Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/ShardTableWrites into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/CreateTables/ParDo(CreateTables) Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.582Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/TagWithUniqueIds into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/ShardTableWrites Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.583Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/Window.Into()/Window.Assign into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/TagWithUniqueIds Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.585Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/WriteStream into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/Window.Into()/Window.Assign Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.587Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/MergeBuckets into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/ReadStream Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.589Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/ExpandIterable into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/MergeBuckets Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.591Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/GlobalWindow/Window.Assign into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/ExpandIterable Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.593Z: Fusing consumer WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/StreamingWrite into WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/GlobalWindow/Window.Assign Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.603Z: The pubsub read for: projects/apache-beam-testing/subscriptions/java_mobile_gaming_topic.subscription-7474408433002216028 is configured to compute input data watermarks based on custom timestamp attribute timestamp_ms. Cloud Dataflow has created an additional tracking subscription to do this, which will be cleaned up automatically. For details, see: https://cloud.google.com/dataflow/model/pubsub-io#timestamps-ids Mar 30, 2020 11:17:25 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:23.873Z: Starting 1 workers... Mar 30, 2020 11:17:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:25.527Z: Pub/Sub resources set up for topic 'projects/apache-beam-testing/topics/java_mobile_gaming_topic'. Mar 30, 2020 11:17:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:26.410Z: Executing operation CalculateTeamScores/ExtractTeamScore/Combine.perKey(SumInteger)/GroupByKey/ReadStream+CalculateTeamScores/ExtractTeamScore/Combine.perKey(SumInteger)/Combine.GroupedValues+WriteTeamScoreSums/ConvertToRow+WriteTeamScoreSums/BigQueryIO.Write/PrepareWrite/ParDo(Anonymous)+WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/CreateTables/ParDo(CreateTables)+WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/ShardTableWrites+WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/TagWithUniqueIds+WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/Window.Into()/Window.Assign+WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/WriteStream Mar 30, 2020 11:17:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:26.413Z: Executing operation PubsubIO.Read/PubsubUnboundedSource+PubsubIO.Read/MapElements/Map+ParseGameEvent+CalculateUserScores/LeaderboardUserGlobalWindow/Window.Assign+CalculateTeamScores/LeaderboardTeamFixedWindows/Window.Assign+CalculateTeamScores/ExtractTeamScore/MapElements/Map+CalculateTeamScores/ExtractTeamScore/Combine.perKey(SumInteger)/GroupByKey/WriteStream+CalculateUserScores/ExtractUserScore/MapElements/Map+CalculateUserScores/ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/WriteStream Mar 30, 2020 11:17:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:26.413Z: Executing operation CalculateUserScores/ExtractUserScore/Combine.perKey(SumInteger)/GroupByKey/ReadStream+CalculateUserScores/ExtractUserScore/Combine.perKey(SumInteger)/Combine.GroupedValues+WriteUserScoreSums/ConvertToRow+WriteUserScoreSums/BigQueryIO.Write/PrepareWrite/ParDo(Anonymous)+WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/CreateTables/ParDo(CreateTables)+WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/ShardTableWrites+WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/TagWithUniqueIds+WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/Window.Into()/Window.Assign+WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/WriteStream Mar 30, 2020 11:17:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:26.413Z: Executing operation WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/ReadStream+WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/MergeBuckets+WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/ExpandIterable+WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/GlobalWindow/Window.Assign+WriteTeamScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/StreamingWrite Mar 30, 2020 11:17:27 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:26.413Z: Executing operation WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/ReadStream+WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/GroupByKey/MergeBuckets+WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/Reshuffle/ExpandIterable+WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/GlobalWindow/Window.Assign+WriteUserScoreSums/BigQueryIO.Write/StreamingInserts/StreamingWriteTables/StreamingWrite Mar 30, 2020 11:17:32 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2020-03-30T11:17:30.048Z: Your project already contains 100 Dataflow-created metric descriptors and Stackdriver will not create new Dataflow custom metrics for this job. Each unique user-defined metric name (independent of the DoFn in which it is defined) produces a new metric descriptor. To delete old / unused metric descriptors see https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list and https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete Mar 30, 2020 11:17:56 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:17:55.986Z: Checking permissions granted to controller Service Account. bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__ Waiting on bqjob_r11ac5f9f2fab2ed5_000001712b29a5c7_1 ... (0s) Current status: RUNNING Waiting on bqjob_r11ac5f9f2fab2ed5_000001712b29a5c7_1 ... (0s) Current status: DONE +-----------------------------------+ | table_id | +-----------------------------------+ | hourly_team_score_python_dataflow | | hourly_team_score_python_direct | | leaderboard_DirectRunner_team | | leaderboard_DirectRunner_user | +-----------------------------------+ Waiting for pipeline to produce more results... Mar 30, 2020 11:18:05 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:18:05.044Z: Worker configuration: n1-standard-4 in us-central1-a. Mar 30, 2020 11:18:10 AM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2020-03-30T11:18:09.791Z: Workers have started successfully. bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__ Waiting on bqjob_r740cc7131c5f9ddb_000001712b2a9e08_1 ... (0s) Current status: RUNNING Waiting on bqjob_r740cc7131c5f9ddb_000001712b2a9e08_1 ... (0s) Current status: DONE +-----------------------------------+ | table_id | +-----------------------------------+ | hourly_team_score_python_dataflow | | hourly_team_score_python_direct | | leaderboard_DirectRunner_team | | leaderboard_DirectRunner_user | +-----------------------------------+ Waiting for pipeline to produce more results... FATAL: command execution failed java.io.IOException: Backing channel 'JNLP4-connect connection from 140.111.238.35.bc.googleusercontent.com/35.238.111.140:59242' is disconnected. at hudson.remoting.RemoteInvocationHandler.channelOrFail(RemoteInvocationHandler.java:214) at hudson.remoting.RemoteInvocationHandler.invoke(RemoteInvocationHandler.java:283) at com.sun.proxy.$Proxy135.isAlive(Unknown Source) at hudson.Launcher$RemoteLauncher$ProcImpl.isAlive(Launcher.java:1150) at hudson.Launcher$RemoteLauncher$ProcImpl.join(Launcher.java:1142) at hudson.Launcher$ProcStarter.join(Launcher.java:470) at hudson.plugins.gradle.Gradle.perform(Gradle.java:317) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:741) at hudson.model.Build$BuildExecution.build(Build.java:206) at hudson.model.Build$BuildExecution.doRun(Build.java:163) at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504) at hudson.model.Run.execute(Run.java:1815) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Caused by: java.nio.channels.ClosedChannelException at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer.onReadClosed(ChannelApplicationLayer.java:209) at org.jenkinsci.remoting.protocol.ApplicationLayer.onRecvClosed(ApplicationLayer.java:222) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.onRecvClosed(ProtocolStack.java:816) at org.jenkinsci.remoting.protocol.FilterLayer.onRecvClosed(FilterLayer.java:287) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.onRecvClosed(SSLEngineFilterLayer.java:181) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.switchToNoSecure(SSLEngineFilterLayer.java:283) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processWrite(SSLEngineFilterLayer.java:503) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.processQueuedWrites(SSLEngineFilterLayer.java:248) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doSend(SSLEngineFilterLayer.java:200) at org.jenkinsci.remoting.protocol.impl.SSLEngineFilterLayer.doCloseSend(SSLEngineFilterLayer.java:213) at org.jenkinsci.remoting.protocol.ProtocolStack$Ptr.doCloseSend(ProtocolStack.java:784) at org.jenkinsci.remoting.protocol.ApplicationLayer.doCloseWrite(ApplicationLayer.java:173) at org.jenkinsci.remoting.protocol.impl.ChannelApplicationLayer$ByteBufferCommandTransport.closeWrite(ChannelApplicationLayer.java:314) at hudson.remoting.Channel.close(Channel.java:1452) at hudson.remoting.Channel.close(Channel.java:1405) at hudson.slaves.SlaveComputer.closeChannel(SlaveComputer.java:847) at hudson.slaves.SlaveComputer.access$800(SlaveComputer.java:108) at hudson.slaves.SlaveComputer$3.run(SlaveComputer.java:756) at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28) at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:59) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure ERROR: apache-beam-jenkins-1 is offline; cannot locate JDK 1.8 (latest) --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
