See
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/1767/display/redirect?page=changes>
Changes:
[ehudm] [BEAM-7432] Set input and output files for sdist
[ehudm] [BEAM-7531] Remove and disallow KMS key setting
[ehudm] Reduce gradle build verbosity
[mxm] [BEAM-7551] Checkpoint Flink's ImpulseSourceFunction
[kanterov] Merge pull request #8858: [BEAM-7542] Fix faulty cast in
BigQueryUtils
[robertwb] Remove unused **kwargs arguments for various transforms.
[github] Adding a Link to The Doc for Cost Estimation
------------------------------------------
[...truncated 343.96 KB...]
INFO: Uploading <67110 bytes, hash HxCXi5AxpbaI5CER9aYVZQ> to
gs://temp-storage-for-perf-tests/mongodbioit0testwriteandread-jenkins-0614192445-dd7db53a/output/results/staging/pipeline-HxCXi5AxpbaI5CER9aYVZQ.pb
org.apache.beam.sdk.io.mongodb.MongoDBIOIT > testWriteAndRead STANDARD_OUT
Dataflow SDK version: 2.14.0-SNAPSHOT
org.apache.beam.sdk.io.mongodb.MongoDBIOIT > testWriteAndRead STANDARD_ERROR
Jun 14, 2019 7:24:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2019-06-14_12_24_50-11263638480099673953?project=apache-beam-testing
org.apache.beam.sdk.io.mongodb.MongoDBIOIT > testWriteAndRead STANDARD_OUT
Submitted job: 2019-06-14_12_24_50-11263638480099673953
org.apache.beam.sdk.io.mongodb.MongoDBIOIT > testWriteAndRead STANDARD_ERROR
Jun 14, 2019 7:24:51 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
--region=us-central1 2019-06-14_12_24_50-11263638480099673953
Jun 14, 2019 7:24:51 PM org.apache.beam.runners.dataflow.TestDataflowRunner
run
INFO: Running Dataflow job 2019-06-14_12_24_50-11263638480099673953 with 1
expected assertions.
Jun 14, 2019 7:24:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:24:50.238Z: Autoscaling is enabled for job
2019-06-14_12_24_50-11263638480099673953. The number of workers will be between
1 and 1000.
Jun 14, 2019 7:24:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:24:50.274Z: Autoscaling was automatically enabled for
job 2019-06-14_12_24_50-11263638480099673953.
Jun 14, 2019 7:24:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:24:53.622Z: Checking permissions granted to controller
Service Account.
Jun 14, 2019 7:25:00 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:24:59.798Z: Worker configuration: n1-standard-1 in
us-central1-f.
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:00.362Z: Expanding CoGroupByKey operations into
optimizable parts.
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:00.493Z: Expanding GroupByKey operations into
optimizable parts.
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:00.541Z: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:00.793Z: Fusing adjacent ParDo, Read, Write, and
Flatten operations
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:00.840Z: Elided trivial flatten
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:00.889Z: Elided trivial flatten
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:00.942Z: Elided trivial flatten
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:00.992Z: Fusing consumer Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow) into Calculate
hashcode/Values/Values/Map
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.038Z: Unzipping flatten s25 for input
s20.org.apache.beam.sdk.values.PCollection.<init>:402#2249e1908bcf01f3
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.088Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify, through flatten
PAssert$0/GroupGlobally/FlattenDummyAndContents, into producer
PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.135Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) into
PAssert$0/GroupGlobally/Window.Into()/Window.Assign
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.194Z: Fusing consumer
PAssert$0/GroupGlobally/Values/Values/Map into
PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.248Z: Fusing consumer
PAssert$0/GroupGlobally/ParDo(Concat) into
PAssert$0/GroupGlobally/Values/Values/Map
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.297Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow into
PAssert$0/GroupGlobally/GroupDummyAndContents/Read
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.344Z: Unzipping flatten s12 for input
s11.org.apache.beam.sdk.values.PCollection.<init>:402#a00997a449310e6c
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.395Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/Window.Into()/Window.Assign, through flatten Calculate
hashcode/Flatten.PCollections, into producer Calculate hashcode/ProduceDefault
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.437Z: Fusing consumer
PAssert$0/GroupGlobally/Window.Into()/Window.Assign into Calculate
hashcode/Values/Values/Map
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.476Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow into
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Read
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.517Z: Fusing consumer
PAssert$0/GroupGlobally/RewindowActuals/Window.Assign into
PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.562Z: Fusing consumer
PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map into
PAssert$0/GroupGlobally/RewindowActuals/Window.Assign
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.601Z: Fusing consumer
PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.647Z: Unzipping flatten s12-u47 for input
s14.org.apache.beam.sdk.values.PCollection.<init>:402#bf09ad1bfb1f38d6-c45
Jun 14, 2019 7:25:01 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.695Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map, through flatten
Calculate hashcode/Flatten.PCollections/Unzipped-1, into producer
PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.742Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map into
PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.789Z: Unzipping flatten s12-u47-u52 for input
s15.org.apache.beam.sdk.values.PCollection.<init>:402#bbafda9391edc490-c50
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.828Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign, through
flatten Calculate hashcode/Flatten.PCollections/Unzipped-2, into producer
PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.891Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign into
PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.940Z: Fusing consumer PAssert$0/RunChecks into
PAssert$0/GetPane/Map
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:01.982Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map into
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.014Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write into
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.060Z: Unzipping flatten s25-u40 for input
s27-reify-value18-c38
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.108Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GroupDummyAndContents/Write, through flatten
PAssert$0/GroupGlobally/FlattenDummyAndContents/Unzipped-1, into producer
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.154Z: Fusing consumer PAssert$0/GetPane/Map into
PAssert$0/GroupGlobally/ParDo(Concat)
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.211Z: Unzipping flatten s12-u47-u52-u57 for input
s16.output-c55
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.238Z: Fusing unzipped copy of
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify, through flatten
Calculate hashcode/Flatten.PCollections/Unzipped-3, into producer
PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.282Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify into
PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.329Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.375Z: Fusing consumer Calculate
hashcode/WithKeys/AddKeys/Map into Map documents to Strings/Map
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.418Z: Fusing consumer Collect read time metrics
into Read all documents/Read(BoundedMongoDbSource)
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.459Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.517Z: Fusing consumer Calculate
hashcode/Values/Values/Map into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.564Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.609Z: Fusing consumer Map documents to
Strings/Map into Collect read time metrics
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.655Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract into Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.707Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial into Calculate
hashcode/WithKeys/AddKeys/Map
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.742Z: Fusing consumer Calculate
hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.771Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify into
PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.818Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/Write into
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:02.854Z: Fusing consumer
PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign into
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:03.365Z: Executing operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Create
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:03.408Z: Executing operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Create
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:03.455Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:03.465Z: Starting 1 workers in us-central1-f...
Jun 14, 2019 7:25:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:03.530Z: Finished operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Create
Jun 14, 2019 7:25:05 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:03.553Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Jun 14, 2019 7:25:05 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:03.555Z: Finished operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Create
Jun 14, 2019 7:25:05 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:03.803Z: Executing operation Read all
documents/Read(BoundedMongoDbSource)+Collect read time metrics+Map documents to
Strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Jun 14, 2019 7:25:05 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:03.849Z: Executing operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Jun 14, 2019 7:25:52 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:25:50.383Z: Workers have started successfully.
Jun 14, 2019 7:26:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:26:20.398Z: Autoscaling: Raised the number of workers
to 1 based on the rate of progress in the currently running step(s).
Jun 14, 2019 7:26:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:26:21.580Z: Workers have started successfully.
Jun 14, 2019 7:27:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:27:06.420Z: Finished operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Jun 14, 2019 7:28:20 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:20.331Z: Finished operation Read all
documents/Read(BoundedMongoDbSource)+Collect read time metrics+Map documents to
Strings/Map+Calculate hashcode/WithKeys/AddKeys/Map+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Partial+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify+Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write
Jun 14, 2019 7:28:20 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:20.424Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Close
Jun 14, 2019 7:28:20 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:20.474Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Close
Jun 14, 2019 7:28:20 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:20.564Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate
hashcode/Values/Values/Map+Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write
Jun 14, 2019 7:28:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:23.452Z: Finished operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues+Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues/Extract+Calculate
hashcode/Values/Values/Map+Calculate
hashcode/View.AsIterable/ParDo(ToIsmRecordForGlobalWindow)+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write
Jun 14, 2019 7:28:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:23.633Z: Executing operation Calculate
hashcode/View.AsIterable/CreateDataflowView
Jun 14, 2019 7:28:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:23.701Z: Finished operation Calculate
hashcode/View.AsIterable/CreateDataflowView
Jun 14, 2019 7:28:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:23.904Z: Executing operation Calculate
hashcode/CreateVoid/Read(CreateSource)+Calculate
hashcode/ProduceDefault+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write
Jun 14, 2019 7:28:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:25.566Z: Finished operation Calculate
hashcode/CreateVoid/Read(CreateSource)+Calculate
hashcode/ProduceDefault+PAssert$0/GroupGlobally/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)+PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map+PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Reify+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Write
Jun 14, 2019 7:28:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:25.670Z: Executing operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Close
Jun 14, 2019 7:28:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:25.706Z: Finished operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Close
Jun 14, 2019 7:28:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:25.818Z: Executing operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$0/GroupGlobally/RewindowActuals/Window.Assign+PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Jun 14, 2019 7:28:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:26.196Z: Finished operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Read+PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/GroupByWindow+PAssert$0/GroupGlobally/GatherAllOutputs/Values/Values/Map+PAssert$0/GroupGlobally/RewindowActuals/Window.Assign+PAssert$0/GroupGlobally/KeyForDummy/AddKeys/Map+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Jun 14, 2019 7:28:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:26.287Z: Executing operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Close
Jun 14, 2019 7:28:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:26.337Z: Finished operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Close
Jun 14, 2019 7:28:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:26.438Z: Executing operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Read+PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
Jun 14, 2019 7:28:29 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:27.916Z: Finished operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Read+PAssert$0/GroupGlobally/GroupDummyAndContents/GroupByWindow+PAssert$0/GroupGlobally/Values/Values/Map+PAssert$0/GroupGlobally/ParDo(Concat)+PAssert$0/GetPane/Map+PAssert$0/RunChecks+PAssert$0/VerifyAssertions/ParDo(DefaultConclude)
Jun 14, 2019 7:28:29 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:28.205Z: Cleaning up.
Jun 14, 2019 7:28:29 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-14T19:28:28.328Z: Stopping worker pool...
STDERR: Note:
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/sdks/java/io/common/src/test/java/org/apache/beam/sdk/io/common/IOITHelperTest.java>
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note:
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/sdks/java/io/mongodb/src/test/java/org/apache/beam/sdk/io/mongodb/MongoDBGridFSIOTest.java>
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
2019-06-14 19:30:30,067 df5e769c MainThread beam_integration_benchmark(1/1)
ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 760, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 609, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
line 160, in Run
job_type=job_type)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,>
line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2019-06-14 19:30:30,069 df5e769c MainThread beam_integration_benchmark(1/1)
INFO Cleaning up benchmark beam_integration_benchmark
2019-06-14 19:30:30,069 df5e769c MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-beam-performancetests-mongodbio-it-1767>
delete -f
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/.test-infra/kubernetes/mongodb/load-balancer/mongo.yml>
--ignore-not-found
2019-06-14 19:30:30,709 df5e769c MainThread beam_integration_benchmark(1/1)
ERROR Exception running benchmark
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 903, in RunBenchmarkTask
RunBenchmark(spec, collector)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 760, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 609, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
line 160, in Run
job_type=job_type)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,>
line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2019-06-14 19:30:30,709 df5e769c MainThread beam_integration_benchmark(1/1)
ERROR Benchmark 1/1 beam_integration_benchmark (UID:
beam_integration_benchmark0) failed. Execution will continue.
2019-06-14 19:30:30,710 df5e769c MainThread beam_integration_benchmark(1/1)
INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed
Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-06-14 19:30:30,710 df5e769c MainThread beam_integration_benchmark(1/1)
INFO Complete logs can be found at:
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/runs/df5e769c/pkb.log>
2019-06-14 19:30:30,710 df5e769c MainThread beam_integration_benchmark(1/1)
INFO Completion statuses can be found at:
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/runs/df5e769c/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
