See
<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT/1794/display/redirect>
------------------------------------------
[...truncated 437.49 KB...]
INFO: 2019-06-02T19:02:21.991Z: Fusing consumer Write xml
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into
Gather element count
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:22.070Z: Fusing consumer Gather element count into
Gather byte count
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:22.162Z: Fusing consumer
PAssert$0/GroupGlobally/Window.Into()/Window.Assign into Calculate
hashcode/ProduceDefault
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:22.244Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
into Calculate hashcode/ProduceDefault
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:22.331Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/Window.Into()/Window.Assign into
PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:22.415Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous) into
PAssert$0/GroupGlobally/Window.Into()/Window.Assign
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:22.506Z: Fusing consumer
PAssert$0/GroupGlobally/GatherAllOutputs/WithKeys/AddKeys/Map into
PAssert$0/GroupGlobally/GatherAllOutputs/Reify.Window/ParDo(Anonymous)
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:22.587Z: Fusing consumer
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey+View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/Combine.GroupedValues/Partial
into
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/WithKeys/AddKeys/Map
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:22.675Z: Fusing consumer Calculate
hashcode/ProduceDefault into Calculate hashcode/CreateVoid/Read(CreateSource)
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:22.765Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify into
PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:22.859Z: Fusing consumer
PAssert$0/GroupGlobally/GroupDummyAndContents/Write into
PAssert$0/GroupGlobally/GroupDummyAndContents/Reify
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:22.944Z: Fusing consumer
PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign into
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:23.862Z: Executing operation Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Create
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:23.943Z: Executing operation Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:24.033Z: Executing operation Write xml
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:24.062Z: Starting 1 workers in us-central1-f...
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:24.123Z: Executing operation Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:24.204Z: Executing operation Read xml
files/ReadAllViaFileBasedSource/Reshuffle/Reshuffle/GroupByKey/Create
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:24.289Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:24.378Z: Executing operation
PAssert$0/GroupGlobally/GatherAllOutputs/GroupByKey/Create
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:24.457Z: Executing operation Write xml
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:24.537Z: Executing operation
PAssert$0/GroupGlobally/GroupDummyAndContents/Create
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:24.627Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Jun 02, 2019 7:02:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:24.712Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Jun 02, 2019 7:02:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:25.767Z: Executing operation Generate
sequence/Read(BoundedCountingSource)+Create xml records/Map+Gather write start
time+Gather byte count+Gather element count+Write xml
files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write
xml files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write
xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write
xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write+Write xml
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write
xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
Jun 02, 2019 7:02:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:02:25.857Z: Executing operation
PAssert$0/GroupGlobally/Create.Values/Read(CreateSource)+PAssert$0/GroupGlobally/WindowIntoDummy/Window.Assign+PAssert$0/GroupGlobally/GroupDummyAndContents/Reify+PAssert$0/GroupGlobally/GroupDummyAndContents/Write
Jun 02, 2019 7:05:46 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:05:45.841Z: Workers have started successfully.
Jun 02, 2019 7:05:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:05:53.097Z: Autoscaling: Raised the number of workers
to 1 based on the rate of progress in the currently running step(s).
Jun 02, 2019 7:06:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:06:17.214Z: Workers have started successfully.
Jun 02, 2019 7:07:46 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:07:45.390Z: Autoscaling: Resizing worker pool from 1 to
5.
Jun 02, 2019 7:08:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:08:16.077Z: Autoscaling: Resizing worker pool from 5 to
8.
Jun 02, 2019 7:08:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:08:40.571Z: Autoscaling: Resizing worker pool from 8 to
10.
Jun 02, 2019 7:09:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:09:17.954Z: Autoscaling: Raised the number of workers
to 7 based on the rate of progress in the currently running step(s).
Jun 02, 2019 7:09:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:09:18.063Z: Resized worker pool to 7, though goal was
12. This could be a quota issue.
Jun 02, 2019 7:09:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:09:23.175Z: Autoscaling: Resizing worker pool from 10
to 12.
Jun 02, 2019 7:09:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:09:33.927Z: Autoscaling: Raised the number of workers
to 10 based on the rate of progress in the currently running step(s).
Jun 02, 2019 7:09:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:09:34.043Z: Resized worker pool to 10, though goal was
14. This could be a quota issue.
Jun 02, 2019 7:09:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:09:48.333Z: Autoscaling: Resizing worker pool from 12
to 14.
Jun 02, 2019 7:09:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:09:49.197Z: Autoscaling: Raised the number of workers
to 11 based on the rate of progress in the currently running step(s).
Jun 02, 2019 7:09:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:09:49.304Z: Resized worker pool to 11, though goal was
14. This could be a quota issue.
Jun 02, 2019 7:10:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:10:13.617Z: Autoscaling: Resizing worker pool from 14
to 16.
Jun 02, 2019 7:10:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:10:15.974Z: Autoscaling: Raised the number of workers
to 12 based on the rate of progress in the currently running step(s).
Jun 02, 2019 7:10:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:10:16.079Z: Resized worker pool to 12, though goal was
16. This could be a quota issue.
Jun 02, 2019 7:10:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:10:21.561Z: Autoscaling: Raised the number of workers
to 14 based on the rate of progress in the currently running step(s).
Jun 02, 2019 7:10:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:10:21.666Z: Resized worker pool to 14, though goal was
16. This could be a quota issue.
Jun 02, 2019 7:10:38 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:10:38.219Z: Autoscaling: Raised the number of workers
to 16 based on the rate of progress in the currently running step(s).
Jun 02, 2019 7:13:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2019-06-02T19:13:07.000Z: java.io.IOException: Failed closing
channel to
gs://temp-storage-for-perf-tests/beam_PerformanceTests_XmlIOIT/1794/_1559501688123/.temp-beam-2019-06-02_18-54-48-0/93a8636e-11f3-4e7d-b7f1-6abdd18694af
at
org.apache.beam.sdk.io.FileBasedSink$Writer.close(FileBasedSink.java:1015)
at
org.apache.beam.sdk.io.WriteFiles$WriteUnshardedTempFilesFn.finishBundle(WriteFiles.java:561)
Caused by: java.io.IOException:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 503,
"errors" : [ {
"domain" : "global",
"message" : "Backend Error",
"reason" : "backendError"
} ],
"message" : "Backend Error"
}
at
com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:367)
at
com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:238)
at
org.apache.beam.sdk.io.FileBasedSink$Writer.close(FileBasedSink.java:1013)
at
org.apache.beam.sdk.io.WriteFiles$WriteUnshardedTempFilesFn.finishBundle(WriteFiles.java:561)
at
org.apache.beam.sdk.io.WriteFiles$WriteUnshardedTempFilesFn$DoFnInvoker.invokeFinishBundle(Unknown
Source)
at
org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.finishBundle(SimpleDoFnRunner.java:224)
at
org.apache.beam.runners.dataflow.worker.SimpleParDoFn.finishBundle(SimpleParDoFn.java:412)
at
org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.finish(ParDoOperation.java:56)
at
org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:85)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:412)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:381)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:306)
at
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 503,
"errors" : [ {
"domain" : "global",
"message" : "Backend Error",
"reason" : "backendError"
} ],
"message" : "Backend Error"
}
at
com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at
com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at
com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:512)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:549)
at
com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:301)
... 4 more
Jun 02, 2019 7:13:08 PM
org.apache.beam.runners.dataflow.TestDataflowRunner$ErrorMonitorMessagesHandler
process
INFO: Dataflow job 2019-06-02_12_01_25-2286102871983299253 threw exception.
Failure message was: java.io.IOException: Failed closing channel to
gs://temp-storage-for-perf-tests/beam_PerformanceTests_XmlIOIT/1794/_1559501688123/.temp-beam-2019-06-02_18-54-48-0/93a8636e-11f3-4e7d-b7f1-6abdd18694af
at
org.apache.beam.sdk.io.FileBasedSink$Writer.close(FileBasedSink.java:1015)
at
org.apache.beam.sdk.io.WriteFiles$WriteUnshardedTempFilesFn.finishBundle(WriteFiles.java:561)
Caused by: java.io.IOException:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 503,
"errors" : [ {
"domain" : "global",
"message" : "Backend Error",
"reason" : "backendError"
} ],
"message" : "Backend Error"
}
at
com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.waitForCompletionAndThrowIfUploadFailed(AbstractGoogleAsyncWriteChannel.java:367)
at
com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel.close(AbstractGoogleAsyncWriteChannel.java:238)
at
org.apache.beam.sdk.io.FileBasedSink$Writer.close(FileBasedSink.java:1013)
at
org.apache.beam.sdk.io.WriteFiles$WriteUnshardedTempFilesFn.finishBundle(WriteFiles.java:561)
at
org.apache.beam.sdk.io.WriteFiles$WriteUnshardedTempFilesFn$DoFnInvoker.invokeFinishBundle(Unknown
Source)
at
org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.finishBundle(SimpleDoFnRunner.java:224)
at
org.apache.beam.runners.dataflow.worker.SimpleParDoFn.finishBundle(SimpleParDoFn.java:412)
at
org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.finish(ParDoOperation.java:56)
at
org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:85)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:412)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:381)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:306)
at
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
at
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
at
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 410 Gone
{
"code" : 503,
"errors" : [ {
"domain" : "global",
"message" : "Backend Error",
"reason" : "backendError"
} ],
"message" : "Backend Error"
}
at
com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:150)
at
com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:113)
at
com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:40)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:512)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:432)
at
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:549)
at
com.google.cloud.hadoop.util.AbstractGoogleAsyncWriteChannel$UploadOperation.call(AbstractGoogleAsyncWriteChannel.java:301)
... 4 more
Jun 02, 2019 7:14:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:14:19.226Z: Executing operation Write xml
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Close
Jun 02, 2019 7:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:14:19.515Z: Executing operation Write xml
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read+Write xml
files/WriteFiles/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow+Write
xml files/WriteFiles/WriteUnshardedBundlesToTempFiles/WriteUnwritten+Write xml
files/WriteFiles/WriteUnshardedBundlesToTempFiles/DropShardNum+Write xml
files/WriteFiles/GatherTempFileResults/Add void key/AddKeys/Map+Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle/Window.Into()/Window.Assign+Write
xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Reify+Write
xml files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Write
Jun 02, 2019 7:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:14:20.147Z: Executing operation Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Close
Jun 02, 2019 7:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:14:20.414Z: Executing operation Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/Read+Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle/GroupByKey/GroupByWindow+Write
xml files/WriteFiles/GatherTempFileResults/Reshuffle/ExpandIterable+Write xml
files/WriteFiles/GatherTempFileResults/Drop key/Values/Map+Write xml
files/WriteFiles/GatherTempFileResults/Gather bundles+Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Pair with random
key+Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write
xml
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write
xml
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Jun 02, 2019 7:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:14:21.243Z: Executing operation Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Jun 02, 2019 7:14:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:14:21.525Z: Executing operation Write xml
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write
xml
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write
xml
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write
xml
files/WriteFiles/GatherTempFileResults/Reshuffle.ViaRandomKey/Values/Values/Map+Write
xml files/WriteFiles/FinalizeTempFileBundles/Finalize+Write xml
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with
random key+Write xml
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Write
xml
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Write
xml
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
Jun 02, 2019 7:14:36 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:14:35.045Z: Executing operation Write xml
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Close
Jun 02, 2019 7:14:36 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2019-06-02T19:14:35.311Z: Executing operation Write xml
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read+Write
xml
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow+Write
xml
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable+Write
xml
files/WriteFiles/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map+Gather
write end time+Get file names/Values/Map+Find files/Match filepatterns+Find
files/Reshuffle.ViaRandomKey/Pair with random key+Find
files/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign+Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify+Find
files/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Write
STDERR:
2019-06-02 19:14:40,445 e3a0b914 MainThread beam_integration_benchmark(1/1)
ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 760, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 609, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
line 160, in Run
job_type=job_type)
File
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,>
line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2019-06-02 19:14:40,447 e3a0b914 MainThread beam_integration_benchmark(1/1)
INFO Cleaning up benchmark beam_integration_benchmark
2019-06-02 19:14:40,449 e3a0b914 MainThread beam_integration_benchmark(1/1)
ERROR Exception running benchmark
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 903, in RunBenchmarkTask
RunBenchmark(spec, collector)
File
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 760, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 609, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
line 160, in Run
job_type=job_type)
File
"<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,>
line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2019-06-02 19:14:40,449 e3a0b914 MainThread beam_integration_benchmark(1/1)
ERROR Benchmark 1/1 beam_integration_benchmark (UID:
beam_integration_benchmark0) failed. Execution will continue.
2019-06-02 19:14:40,450 e3a0b914 MainThread beam_integration_benchmark(1/1)
INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed
Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2019-06-02 19:14:40,450 e3a0b914 MainThread beam_integration_benchmark(1/1)
INFO Complete logs can be found at:
<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/runs/e3a0b914/pkb.log>
2019-06-02 19:14:40,450 e3a0b914 MainThread beam_integration_benchmark(1/1)
INFO Completion statuses can be found at:
<https://builds.apache.org/job/beam_PerformanceTests_XmlIOIT/ws/runs/e3a0b914/completion_statuses.json>
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
