See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/1041/display/redirect?page=changes>
Changes: [vaclav.plajt] [BEAM-6062] MetricsContainerStepMap#toString overwritten to show Beam ------------------------------------------ [...truncated 255.98 KB...] INFO: Adding Query7/Query7.Monitor as step s2 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7.Snoop as step s3 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/justBids/IsBid/ParDo(Anonymous) as step s4 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/justBids/AsBid as step s5 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/Window.Into()/Window.Assign as step s6 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/BidToPrice as step s7 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map as step s8 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce as step s9 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey as step s10 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues as step s11 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map as step s12 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map as step s13 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map as step s14 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections as step s15 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey as step s16 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues as step s17 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map as step s18 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) as step s19 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly as step s20 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) as step s21 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/Combine.GloballyAsSingletonView/CreateDataflowView as step s22 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7/Query7.Select as step s23 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7.Debug as step s24 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7/Query7.Stamp as step s25 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7.Format as step s26 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Query7.DevNull as step s27 Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/ Nov 15, 2018 12:46:43 PM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <88169 bytes, hash e-xgp3wqxBFZ9jF6CEhLgQ> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-e-xgp3wqxBFZ9jF6CEhLgQ.pb Dataflow SDK version: 2.9.0-SNAPSHOT Nov 15, 2018 12:46:44 PM org.apache.beam.runners.dataflow.DataflowRunner run Submitted job: 2018-11-15_04_46_43-7270627279516513033 INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-11-15_04_46_43-7270627279516513033?project=apache-beam-testing Nov 15, 2018 12:46:44 PM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2018-11-15_04_46_43-7270627279516513033 Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2018-11-15T12:46:44.121Z: The requested max number of workers (4) is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE). Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:46.693Z: Checking permissions granted to controller Service Account. Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:50.241Z: Worker configuration: n1-standard-1 in us-central1-b. Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:50.717Z: Expanding CoGroupByKey operations into optimizable parts. Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:50.822Z: Expanding GroupByKey operations into optimizable parts. Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:50.873Z: Lifting ValueCombiningMappingFns into MergeBucketsMappingFns Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.081Z: Fusing adjacent ParDo, Read, Write, and Flatten operations Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.123Z: Unzipping flatten s15 for input s13.org.apache.beam.sdk.values.PCollection.<init>:402#403d1808e3d3838 Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.164Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial, through flatten Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Flatten.PCollections, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.211Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.265Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.341Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey) Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.407Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) into Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.458Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.513Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.560Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.604Z: Unzipping flatten s15-u45 for input partial-s16-out1-c43 Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.653Z: Fusing unzipped copy of Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify, through flatten s15-u45, into producer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.706Z: Fusing consumer Query7/Query7/BidToPrice into Query7/Query7/Window.Into()/Window.Assign Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.762Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.812Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.866Z: Fusing consumer Query7.Format into Query7/Query7.Stamp Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.906Z: Fusing consumer Query7/Query7.Stamp into Query7/Query7.Debug Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:51.954Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:52.017Z: Fusing consumer Query7/Query7/justBids/IsBid/ParDo(Anonymous) into Query7/Query7.Snoop Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:52.060Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:52.107Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map into Query7/Query7/BidToPrice Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:52.156Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:52.223Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:52.275Z: Fusing consumer Query7/Query7/justBids/AsBid into Query7/Query7/justBids/IsBid/ParDo(Anonymous) Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:52.322Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:52.368Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:52.408Z: Fusing consumer Query7/Query7.Monitor into Query7.ReadBounded Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:52.439Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:52.486Z: Fusing consumer Query7/Query7/Window.Into()/Window.Assign into Query7/Query7/justBids/AsBid Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:52.531Z: Fusing consumer Query7/Query7.Debug into Query7/Query7/Query7.Select Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:52.582Z: Fusing consumer Query7/Query7.Snoop into Query7/Query7.Monitor Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:52.635Z: Fusing consumer Query7.DevNull into Query7.Format Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:52.683Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:52.734Z: Fusing consumer Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write into Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:53.354Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Create Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:53.410Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Create Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:53.461Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:53.476Z: Starting 4 workers in us-central1-b... Nov 15, 2018 12:47:02 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:46:53.781Z: Executing operation Query7.ReadBounded+Query7/Query7.Monitor+Query7/Query7.Snoop+Query7/Query7/justBids/IsBid/ParDo(Anonymous)+Query7/Query7/justBids/AsBid+Query7/Query7/Window.Into()/Window.Assign+Query7/Query7/BidToPrice+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/WithKeys/AddKeys/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/AddNonce+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PrepareCold/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Write Nov 15, 2018 12:47:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:47:10.379Z: Autoscaling: Raised the number of workers to 0 based on the rate of progress in the currently running step(s). Nov 15, 2018 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:47:38.759Z: Autoscaling: Raised the number of workers to 1 based on the rate of progress in the currently running step(s). Nov 15, 2018 12:47:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:47:38.851Z: Resized worker pool to 1, though goal was 4. This could be a quota issue. Nov 15, 2018 12:47:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:47:39.535Z: Workers have started successfully. Nov 15, 2018 12:47:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:47:39.582Z: Workers have started successfully. Nov 15, 2018 12:48:11 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process WARNING: 2018-11-15T12:48:10.628Z: Autoscaling: Startup of the worker pool in zone us-central1-b reached 2 workers, but the goal was 4 workers. The service will retry. QUOTA_EXCEEDED: Quota 'CPUS' exceeded. Limit: 750.0 in region us-central1. Nov 15, 2018 12:49:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:49:29.849Z: Autoscaling: Raised the number of workers to 2 based on the rate of progress in the currently running step(s). Nov 15, 2018 12:49:31 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:49:29.887Z: Resized worker pool to 2, though goal was 4. This could be a quota issue. Nov 15, 2018 12:50:12 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:50:11.630Z: Autoscaling: Raised the number of workers to 4 based on the rate of progress in the currently running step(s). Nov 15, 2018 12:50:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:50:38.217Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Close Nov 15, 2018 12:50:39 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:50:38.310Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PreCombineHot/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/StripNonce/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/Window.Remerge/Identity/Map+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Partial+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Reify+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Write Nov 15, 2018 12:50:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:50:40.949Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Close Nov 15, 2018 12:50:41 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:50:41.040Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/GroupByKey/Read+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Combine.perKeyWithFanout(MaxLong)/PostCombine/Combine.GroupedValues/Extract+Query7/Query7/Combine.GloballyAsSingletonView/Combine.globally(MaxLong)/Values/Values/Map+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/ParDo(UseWindowHashAsKeyAndWindowAsSortKey)+Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Write Nov 15, 2018 12:50:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:50:42.505Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Close Nov 15, 2018 12:50:42 PM org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process INFO: 2018-11-15T12:50:42.604Z: Executing operation Query7/Query7/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Read+Query7/Query7/Combine.GloballyAsSingletonView/ParDo(IsmRecordForSingularValuePerWindow) Nov 15, 2018 12:50:44 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish WARNING: No terminal state was returned. State value RUNNING 2018-11-15T12:50:44.940Z Waiting for main pipeline to 'finish' 2018-11-15T12:50:44.981Z RUNNING Query7 Nov 15, 2018 12:50:45 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.elements, from namespace Query7.Results Nov 15, 2018 12:50:45 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.bytes, from namespace Query7.Results Nov 15, 2018 12:50:45 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getDistributionMetric SEVERE: Failed to get distribution metric result.startTime for namespace Query7.Results Nov 15, 2018 12:50:45 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getDistributionMetric SEVERE: Failed to get distribution metric result.endTime for namespace Query7.Results Nov 15, 2018 12:50:46 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getDistributionMetric SEVERE: Failed to get distribution metric result.startTimestamp for namespace Query7.Results Nov 15, 2018 12:50:46 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getDistributionMetric SEVERE: Failed to get distribution metric result.endTimestamp for namespace Query7.Results 2018-11-15T12:50:46.266Z new perf {"snapshots":null,"runtimeSec":161.414,"numEvents":10000000,"eventsPerSec":61952.494826966686,"eventBytesPerSec":7952004.764146853,"numResults":-1,"resultsPerSec":-1.0,"resultBytesPerSec":-1.0,"startupDelaySec":-170.493,"processingDelaySec":-1.0,"shutdownDelaySec":-1.0,"timeDilation":-1.0,"errors":null,"jobId":null} 2018-11-15T12:51:01.325Z RUNNING Query7 2018-11-15T12:51:02.468Z new perf {"snapshots":null,"runtimeSec":186.494,"numEvents":10000000,"eventsPerSec":53621.028022349245,"eventBytesPerSec":6882606.931054082,"numResults":49,"resultsPerSec":0.2627430373095113,"resultBytesPerSec":26.483425740238292,"startupDelaySec":-170.493,"processingDelaySec":173.985,"shutdownDelaySec":0.329,"timeDilation":5.1103520756699945,"errors":null,"jobId":null} 2018-11-15T12:51:17.562Z RUNNING Query7 2018-11-15T12:51:18.920Z new perf {"snapshots":null,"runtimeSec":196.804,"numEvents":10000000,"eventsPerSec":50811.975366354345,"eventBytesPerSec":6522046.792748115,"numResults":100,"resultsPerSec":0.5081197536635434,"resultBytesPerSec":51.335338712627795,"startupDelaySec":-170.493,"processingDelaySec":173.985,"shutdownDelaySec":6.218,"timeDilation":4.996849657527286,"errors":null,"jobId":null} 2018-11-15T12:51:33.967Z RUNNING Query7 2018-11-15T12:51:35.325Z no activity 2018-11-15T12:51:50.406Z RUNNING Query7 2018-11-15T12:51:51.940Z no activity 2018-11-15T12:52:07.005Z RUNNING Query7 2018-11-15T12:52:08.272Z no activity 2018-11-15T12:52:23.332Z RUNNING Query7 2018-11-15T12:52:24.901Z no activity 2018-11-15T12:52:39.946Z RUNNING Query7 2018-11-15T12:52:41.255Z no activity 2018-11-15T12:52:56.301Z RUNNING Query7 2018-11-15T12:52:57.612Z no activity 2018-11-15T12:53:12.666Z RUNNING Query7 2018-11-15T12:53:13.746Z no activity 2018-11-15T12:53:28.779Z RUNNING Query7 2018-11-15T12:53:29.827Z no activity 2018-11-15T12:53:44.876Z RUNNING Query7 2018-11-15T12:53:45.986Z no activity 2018-11-15T12:54:01.106Z DONE Query7 > Task :beam-sdks-java-nexmark:run FAILED :beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 4,5,main]) completed. Took 7 mins 42.919 secs. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':beam-sdks-java-nexmark:run'. > Process 'command '/usr/local/asfpackages/java/jdk1.8.0_191/bin/java'' > finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 11m 27s 76 actionable tasks: 71 executed, 5 from cache Publishing build scan... https://gradle.com/s/7ewisnl3jplvw Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: builds-unsubscr...@beam.apache.org For additional commands, e-mail: builds-h...@beam.apache.org
