See <https://builds.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/660/display/redirect?page=changes>
Changes: [yifanzou] [BEAM-5700] remove the extra licenses from python bigquery IT ------------------------------------------ [...truncated 1.28 MB...] Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding Create.Values/Read(CreateSource)/Read(CreateSource)/Read(BoundedToUnboundedSourceAdapter)/StripIds as step s2 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/PrepareWrite/ParDo(Anonymous) as step s3 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/JobIdCreationRoot/Read(CreateSource)/Read(CreateSource)/Read(BoundedToUnboundedSourceAdapter)/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s4 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.core.construction.UnboundedReadFromBoundedSource$BoundedToUnboundedSourceAdapter split WARNING: BoundedSource [0, 1) cannot estimate its size, skips the initial splits. Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/JobIdCreationRoot/Read(CreateSource)/Read(CreateSource)/Read(BoundedToUnboundedSourceAdapter)/StripIds as step s5 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/CreateJobId as step s6 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map as step s7 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey as step s8 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues as step s9 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map as step s10 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/View.AsSingleton/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization) as step s11 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/Combine.globally(Concatenate)/WithKeys/AddKeys/Map as step s12 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/Combine.globally(Concatenate)/Combine.perKey(Concatenate)/GroupByKey as step s13 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/Combine.globally(Concatenate)/Combine.perKey(Concatenate)/Combine.GroupedValues as step s14 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/Combine.globally(Concatenate)/Values/Values/Map as step s15 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ParDo(StreamingPCollectionViewWriter) as step s16 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/View.AsSingleton/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateDataflowView as step s17 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/Create.Values/Read(CreateSource)/Read(CreateSource)/Read(BoundedToUnboundedSourceAdapter)/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s18 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.core.construction.UnboundedReadFromBoundedSource$BoundedToUnboundedSourceAdapter split WARNING: BoundedSource [0, 1) cannot estimate its size, skips the initial splits. Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/Create.Values/Read(CreateSource)/Read(CreateSource)/Read(BoundedToUnboundedSourceAdapter)/StripIds as step s19 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/GetTempFilePrefix as step s20 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/TempFilePrefixView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map as step s21 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/TempFilePrefixView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey as step s22 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/TempFilePrefixView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues as step s23 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/TempFilePrefixView/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Values/Values/Map as step s24 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/TempFilePrefixView/Combine.GloballyAsSingletonView/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization) as step s25 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/TempFilePrefixView/Combine.GloballyAsSingletonView/View.CreatePCollectionView/Combine.globally(Concatenate)/WithKeys/AddKeys/Map as step s26 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/TempFilePrefixView/Combine.GloballyAsSingletonView/View.CreatePCollectionView/Combine.globally(Concatenate)/Combine.perKey(Concatenate)/GroupByKey as step s27 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/TempFilePrefixView/Combine.GloballyAsSingletonView/View.CreatePCollectionView/Combine.globally(Concatenate)/Combine.perKey(Concatenate)/Combine.GroupedValues as step s28 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/TempFilePrefixView/Combine.GloballyAsSingletonView/View.CreatePCollectionView/Combine.globally(Concatenate)/Values/Values/Map as step s29 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/TempFilePrefixView/Combine.GloballyAsSingletonView/View.CreatePCollectionView/ParDo(StreamingPCollectionViewWriter) as step s30 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/TempFilePrefixView/Combine.GloballyAsSingletonView/View.CreatePCollectionView/CreateDataflowView as step s31 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/rewindowIntoGlobal/Window.Assign as step s32 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/WriteBundlesToFiles as step s33 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/GroupByDestination as step s34 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/WriteGroupedRecords as step s35 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/FlattenFiles as step s36 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/View.AsIterable/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization) as step s37 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/View.AsIterable/View.CreatePCollectionView/Combine.globally(Concatenate)/WithKeys/AddKeys/Map as step s38 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/View.AsIterable/View.CreatePCollectionView/Combine.globally(Concatenate)/Combine.perKey(Concatenate)/GroupByKey as step s39 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/View.AsIterable/View.CreatePCollectionView/Combine.globally(Concatenate)/Combine.perKey(Concatenate)/Combine.GroupedValues as step s40 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/View.AsIterable/View.CreatePCollectionView/Combine.globally(Concatenate)/Values/Values/Map as step s41 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/View.AsIterable/View.CreatePCollectionView/ParDo(StreamingPCollectionViewWriter) as step s42 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/View.AsIterable/View.CreatePCollectionView/CreateDataflowView as step s43 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/Create.Values/Read(CreateSource)/Read(CreateSource)/Read(BoundedToUnboundedSourceAdapter)/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s44 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.core.construction.UnboundedReadFromBoundedSource$BoundedToUnboundedSourceAdapter split WARNING: BoundedSource [0, 1) cannot estimate its size, skips the initial splits. Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/Create.Values/Read(CreateSource)/Read(CreateSource)/Read(BoundedToUnboundedSourceAdapter)/StripIds as step s45 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyResults/ParDo(Anonymous) as step s46 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/WritePartitionUntriggered as step s47 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsReshuffle/Window.Into()/Window.Assign as step s48 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsReshuffle/GroupByKey as step s49 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsReshuffle/ExpandIterable as step s50 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsWriteTables/ParMultiDo(WriteTables) as step s51 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsWriteTables/WithKeys/AddKeys/Map as step s52 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsWriteTables/Window.Into()/Window.Assign as step s53 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsWriteTables/GroupByKey as step s54 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsWriteTables/Values/Values/Map as step s55 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/MultiPartitionsWriteTables/ParDo(GarbageCollectTemporaryFiles) as step s56 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.VoidKeyToMultimapMaterialization/ParDo(VoidKeyToMultimapMaterialization) as step s57 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.CreatePCollectionView/Combine.globally(Concatenate)/WithKeys/AddKeys/Map as step s58 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.CreatePCollectionView/Combine.globally(Concatenate)/Combine.perKey(Concatenate)/GroupByKey as step s59 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.CreatePCollectionView/Combine.globally(Concatenate)/Combine.perKey(Concatenate)/Combine.GroupedValues as step s60 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.CreatePCollectionView/Combine.globally(Concatenate)/Values/Values/Map as step s61 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.CreatePCollectionView/ParDo(StreamingPCollectionViewWriter) as step s62 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/View.AsIterable/View.CreatePCollectionView/CreateDataflowView as step s63 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/Create.Values/Read(CreateSource)/Read(CreateSource)/Read(BoundedToUnboundedSourceAdapter)/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s64 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.core.construction.UnboundedReadFromBoundedSource$BoundedToUnboundedSourceAdapter split WARNING: BoundedSource [0, 1) cannot estimate its size, skips the initial splits. Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/Create.Values/Read(CreateSource)/Read(CreateSource)/Read(BoundedToUnboundedSourceAdapter)/StripIds as step s65 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/ReifyRenameInput/ParDo(Anonymous) as step s66 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/WriteRenameUntriggered as step s67 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionsReshuffle/Window.Into()/Window.Assign as step s68 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionsReshuffle/GroupByKey as step s69 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionsReshuffle/ExpandIterable as step s70 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionWriteTables/ParMultiDo(WriteTables) as step s71 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionWriteTables/WithKeys/AddKeys/Map as step s72 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionWriteTables/Window.Into()/Window.Assign as step s73 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionWriteTables/GroupByKey as step s74 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionWriteTables/Values/Values/Map as step s75 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/SinglePartitionWriteTables/ParDo(GarbageCollectTemporaryFiles) as step s76 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/CreateEmptyFailedInserts/Read(CreateSource)/Read(CreateSource)/Read(BoundedToUnboundedSourceAdapter)/DataflowRunner.StreamingUnboundedRead.ReadWithIds as step s77 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.core.construction.UnboundedReadFromBoundedSource$BoundedToUnboundedSourceAdapter split WARNING: BoundedSource [0, 0) cannot estimate its size, skips the initial splits. Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding savePerfsToBigQuery/BatchLoads/CreateEmptyFailedInserts/Read(CreateSource)/Read(CreateSource)/Read(BoundedToUnboundedSourceAdapter)/StripIds as step s78 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep INFO: Adding DropInputs as step s79 Oct 11, 2018 12:19:21 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: Staging pipeline description to gs://temp-storage-for-perf-tests/nexmark/staging/ Oct 11, 2018 12:19:22 AM org.apache.beam.runners.dataflow.util.PackageUtil tryStagePackage INFO: Uploading <263812 bytes, hash _y2FfmY3xosmSK5fechvrw> to gs://temp-storage-for-perf-tests/nexmark/staging/pipeline-_y2FfmY3xosmSK5fechvrw.pb Dataflow SDK version: 2.9.0-SNAPSHOT Oct 11, 2018 12:19:24 AM org.apache.beam.runners.dataflow.DataflowRunner run Submitted job: 2018-10-10_17_19_22-10403671556006359986 INFO: To access the Dataflow monitoring console, please navigate to https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-10-10_17_19_22-10403671556006359986?project=apache-beam-testing Oct 11, 2018 12:19:24 AM org.apache.beam.runners.dataflow.DataflowRunner run INFO: To cancel the job using the 'gcloud' tool, run: > gcloud dataflow jobs --project=apache-beam-testing cancel > --region=us-central1 2018-10-10_17_19_22-10403671556006359986 ========================================================================================== Run started 2018-10-11T00:06:52.443Z and ran for PT752.220S Default configuration: {"debug":true,"query":0,"sourceType":"DIRECT","sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1} Configurations: Conf Description 0000 query:0; exportSummaryToBigQuery:true; numEvents:10000000 0001 query:1; exportSummaryToBigQuery:true; numEvents:10000000 0002 query:2; exportSummaryToBigQuery:true; numEvents:10000000 0003 query:3; exportSummaryToBigQuery:true; numEvents:10000000 0004 query:4; exportSummaryToBigQuery:true; numEvents:1000000 0005 query:5; exportSummaryToBigQuery:true; numEvents:10000000 0006 query:6; exportSummaryToBigQuery:true; numEvents:1000000 0007 query:7; exportSummaryToBigQuery:true; numEvents:10000000 0008 query:8; exportSummaryToBigQuery:true; numEvents:10000000 0009 query:9; exportSummaryToBigQuery:true; numEvents:1000000 0010 query:10; exportSummaryToBigQuery:true; numEvents:10000000 0011 query:11; exportSummaryToBigQuery:true; numEvents:10000000 0012 query:12; exportSummaryToBigQuery:true; numEvents:10000000 Performance: Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline) 0000 24.0 416250.4 10000000 0001 20.5 488973.6 9200000 0002 11.3 881756.5 83527 0003 40.8 245104.0 59380 0004 53.3 18764.5 140 0005 131.0 76337.0 4567 0006 39.1 25589.8 14087 0007 123.5 80963.8 100 0008 -1.0 -1.0 -1 *** Job was unexpectedly updated *** 0009 39.7 25173.7 32521 0010 -1.0 -1.0 -1 *** Job was unexpectedly updated *** 0011 203.1 49238.3 1839657 0012 181.2 55173.9 570114 ========================================================================================== Exception in thread "main" java.lang.RuntimeException: Execution was not successful at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:174) at org.apache.beam.sdk.nexmark.Main.main(Main.java:477) > Task :beam-sdks-java-nexmark:run FAILED :beam-sdks-java-nexmark:run (Thread[Task worker for ':' Thread 4,5,main]) completed. Took 12 mins 33.479 secs. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':beam-sdks-java-nexmark:run'. > Process 'command '/usr/local/asfpackages/java/jdk1.8.0_172/bin/java'' > finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/4.10.2/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 14m 40s 65 actionable tasks: 15 executed, 50 up-to-date Publishing build scan... https://gradle.com/s/u7bj7yew3pt52 Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure
