See
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/1776/display/redirect>
------------------------------------------
[...truncated 338.13 KB...]
INFO: Adding Map documents to Strings/Map as step s3
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/WithKeys/AddKeys/Map as step s4
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Combine.perKey(Hashing)/GroupByKey as step
s5
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues as step s6
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Values/Values/Map as step s7
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) as step s8
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/View.AsIterable/CreateDataflowView as step
s9
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/CreateVoid/Read(CreateSource) as step s10
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/ProduceDefault as step s11
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding Calculate hashcode/Flatten.PCollections as step s12
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Window.Into()/Window.Assign as step s13
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) as step
s14
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
as step s15
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign as step s16
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey as step s17
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map as
step s18
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/RewindowActuals/Window.Assign as step
s19
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map as step s20
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
PAssert$0/GroupGlobally/RemoveActualsTriggering/Flatten.PCollections as step s21
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Create.Values/Read(CreateSource) as
step s22
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign as step
s23
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding
PAssert$0/GroupGlobally/RemoveDummyTriggering/Flatten.PCollections as step s24
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/FlattenDummyAndContents as step s25
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/NeverTrigger/Flatten.PCollections as
step s26
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/GroupDummyAndContents as step s27
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/Values/Values/Map as step s28
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GroupGlobally/ParDo(Concat) as step s29
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/GetPane/Map as step s30
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/RunChecks as step s31
Jun 17, 2019 1:06:09 AM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding PAssert$0/VerifyAssertions/ParDo(DefaultConclude) as step s32
Jun 17, 2019 1:06:09 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to
gs://temp-storage-for-perf-tests/mongodbioit0testwriteandread-jenkins-0617010605-b6cbfbe3/output/results/staging/
Jun 17, 2019 1:06:09 AM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading <67108 bytes, hash O91KM1UaknocLQy7jcZPCg> to
gs://temp-storage-for-perf-tests/mongodbioit0testwriteandread-jenkins-0617010605-b6cbfbe3/output/results/staging/pipeline-O91KM1UaknocLQy7jcZPCg.pb
org.apache.beam.sdk.io.mongodb.MongoDBIOIT > testWriteAndRead STANDARD_OUT
Dataflow SDK version: 2.14.0-SNAPSHOT
org.apache.beam.sdk.io.mongodb.MongoDBIOIT > testWriteAndRead STANDARD_ERROR
Jun 17, 2019 1:06:11 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-16_18_06_09-12888579051115498066?project=apache-beam-testing
org.apache.beam.sdk.io.mongodb.MongoDBIOIT > testWriteAndRead STANDARD_OUT
Submitted job: 2019-06-16_18_06_09-12888579051115498066
org.apache.beam.sdk.io.mongodb.MongoDBIOIT > testWriteAndRead STANDARD_ERROR
Jun 17, 2019 1:06:11 AM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
--region=us-central1 2019-06-16_18_06_09-12888579051115498066
Jun 17, 2019 1:06:11 AM org.apache.beam.runners.dataflow.TestDataflowRunner
run
INFO: Running Dataflow job 2019-06-16_18_06_09-12888579051115498066 with 1
expected assertions.
Jun 17, 2019 1:06:11 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:10.106Z: Autoscaling is enabled for job
2019-06-16_18_06_09-12888579051115498066. The number of workers will be between
1 and 1000.
Jun 17, 2019 1:06:11 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:10.150Z: Autoscaling was automatically enabled for
job 2019-06-16_18_06_09-12888579051115498066.
Jun 17, 2019 1:06:18 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:15.525Z: Checking permissions granted to controller
Service Account.
Jun 17, 2019 1:06:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:20.602Z: Worker configuration: n1-standard-1 in
us-central1-b.
Jun 17, 2019 1:06:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:21.170Z: Expanding CoGroupByKey operations into
optimizable parts.
Jun 17, 2019 1:06:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:21.314Z: Expanding GroupByKey operations into
optimizable parts.
Jun 17, 2019 1:06:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:21.362Z: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
Jun 17, 2019 1:06:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:21.584Z: Fusing adjacent ParDo, Read, Write, and
Flatten operations
Jun 17, 2019 1:06:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:21.622Z: Elided trivial flatten
Jun 17, 2019 1:06:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:21.654Z: Elided trivial flatten
Jun 17, 2019 1:06:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:21.697Z: Elided trivial flatten
Jun 17, 2019 1:06:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:21.731Z: Fusing consumer Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) into Calculate
hashcode/Values/Values/Map
Jun 17, 2019 1:06:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:21.778Z: Unzipping flatten s25 for input
s20.org.apache.beam.sdk.values.PCollection.<init>:402#2249e1908bcf01f3
Jun 17, 2019 1:06:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:21.810Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify, through flatten
PAssert$0/GroupGlobally/FlattenDummyAndContents, into producer
PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map
Jun 17, 2019 1:06:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:21.854Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) into
PAssert$0/GroupGlobally/Window.Into()/Window.Assign
Jun 17, 2019 1:06:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:21.883Z: Fusing consumer
PAssert$0/GroupGlobally/Values/Values/Map into
PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow
Jun 17, 2019 1:06:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:21.923Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(Concat) into
PAssert$0/GroupGlobally/Values/Values/Map
Jun 17, 2019 1:06:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:21.970Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow into
PAssert$0/GroupGlobally/GroupDummyAndContents/Read
Jun 17, 2019 1:06:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.012Z: Unzipping flatten s12 for input
s11.org.apache.beam.sdk.values.PCollection.<init>:402#a00997a449310e6c
Jun 17, 2019 1:06:22 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.058Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/Window.Into()/Window.Assign, through flatten Calculate
hashcode/Flatten.PCollections, into producer Calculate hashcode/ProduceDefault
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.099Z: Fusing consumer
PAssert$0/GroupGlobally/Window.Into()/Window.Assign into Calculate
hashcode/Values/Values/Map
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.137Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow into
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Read
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.180Z: Fusing consumer
PAssert$0/GroupGlobally/RewindowActuals/Window.Assign into
PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.223Z: Fusing consumer
PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map into
PAssert$0/GroupGlobally/RewindowActuals/Window.Assign
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.271Z: Fusing consumer
PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.317Z: Unzipping flatten s12-u47 for input
s14.org.apache.beam.sdk.values.PCollection.<init>:402#bf09ad1bfb1f38d6-c45
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.360Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map, through flatten
Calculate hashcode/Flatten.PCollections/Unzipped-1, into producer
PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.407Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map into
PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.452Z: Unzipping flatten s12-u47-u52 for input
s15.org.apache.beam.sdk.values.PCollection.<init>:402#bbafda9391edc490-c50
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.497Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign, through
flatten Calculate hashcode/Flatten.PCollections/Unzipped-2, into producer
PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.534Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign into
PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.579Z: Fusing consumer PAssert$0/RunChecks into
PAssert$0/GetPane/Map
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.624Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map into
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.670Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write into
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.708Z: Unzipping flatten s25-u40 for input
s27-reify-value18-c38
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.753Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GroupDummyAndContents/Write, through flatten
PAssert$0/GroupGlobally/FlattenDummyAndContents/Unzipped-1, into producer
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.792Z: Fusing consumer PAssert$0/GetPane/Map into
PAssert$0/GroupGlobally/ParDo(Concat)
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.825Z: Unzipping flatten s12-u47-u52-u57 for input
s16.output-c55
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.870Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify, through flatten
Calculate hashcode/Flatten.PCollections/Unzipped-3, into producer
PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.897Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify into
PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:22.944Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:23.024Z: Fusing consumer Calculate
hashcode/WithKeys/AddKeys/Map into Map documents to Strings/Map
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:23.067Z: Fusing consumer Collect read time metrics
into Read all documents/Read(BoundedMongoDbSource)
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:23.113Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:23.160Z: Fusing consumer Calculate
hashcode/Values/Values/Map into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:23.207Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:23.244Z: Fusing consumer Map documents to
Strings/Map into Collect read time metrics
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:23.289Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:23.333Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate
hashcode/WithKeys/AddKeys/Map
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:23.375Z: Fusing consumer Calculate
hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:23.419Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify into
PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:23.464Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/Write into
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:23.505Z: Fusing consumer
PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign into
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:23.962Z: Executing operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Create
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:24.008Z: Executing operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Create
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:24.056Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:24.068Z: Starting 1 workers in us-central1-b...
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:24.117Z: Finished operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Create
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:24.135Z: Finished operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Create
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:24.135Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:24.316Z: Executing operation Read all
documents/Read(BoundedMongoDbSource)+Collect read time metrics+Map documents to
Strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Jun 17, 2019 1:06:24 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:06:24.359Z: Executing operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Jun 17, 2019 1:07:14 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:07:12.378Z: Workers have started successfully.
Jun 17, 2019 1:07:41 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:07:40.374Z: Autoscaling: Raised the number of workers
to 1 based on the rate of progress in the currently running step(s).
Jun 17, 2019 1:07:44 AM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-17T01:07:43.752Z: Workers have started successfully.
STDERR:
2019-06-17 01:25:43,489 ccb9b33a MainThread beam_integration_benchmark(1/1)
ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 760, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 609, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
line 160, in Run
job_type=job_type)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,>
line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2019-06-17 01:25:43,490 ccb9b33a MainThread beam_integration_benchmark(1/1)
INFO Cleaning up benchmark beam_integration_benchmark
2019-06-17 01:25:43,491 ccb9b33a MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-beam-performancetests-mongodbio-it-1776>
delete -f
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/.test-infra/kubernetes/mongodb/load-balancer/mongo.yml>
--ignore-not-found
2019-06-17 01:25:45,661 ccb9b33a MainThread beam_integration_benchmark(1/1)
ERROR Exception running benchmark
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 903, in RunBenchmarkTask
RunBenchmark(spec, collector)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 760, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 609, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
line 160, in Run
job_type=job_type)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,>
line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2019-06-17 01:25:45,661 ccb9b33a MainThread beam_integration_benchmark(1/1)
ERROR Benchmark 1/1 beam_integration_benchmark (UID:
beam_integration_benchmark0) failed. Execution will continue.
2019-06-17 01:25:45,662 ccb9b33a MainThread beam_integration_benchmark(1/1)
INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed
Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-06-17 01:25:45,662 ccb9b33a MainThread beam_integration_benchmark(1/1)
INFO Complete logs can be found at:
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/runs/ccb9b33a/pkb.log>
2019-06-17 01:25:45,662 ccb9b33a MainThread beam_integration_benchmark(1/1)
INFO Completion statuses can be found at:
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/runs/ccb9b33a/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
