See
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/322/display/redirect?page=changes>
Changes:
[robertwb] [BEAM-4565] Fix hot key fanout in the face of combiner lifting.
[robertwb] Fix and test global combine with fanout.
[ankurgoenka] Fixing key lookup in
FlinkBatchSideInputHandlerFactory.SideInputHandler
[echauchot] [BEAM-4283] Fix naming of the BigQuery fields
[iemejia] [BEAM-3314] Set correctly host and port on RedisIO
[iemejia] [BEAM-3314] Fix error-prone warnings and add extra test for Read
[migryz] Point gearpump to master
------------------------------------------
[...truncated 246.85 KB...]
INFO: 2018-06-21T18:17:48.546Z: Fusing consumer Write Avro records to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
into Write Avro records to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Reify.ReifyView/ParDo(Anonymous)
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:48.582Z: Fusing consumer Values/Values/Map into
Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Values/Values/Map
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:48.612Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/Combine.GroupedValues into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Read
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:48.645Z: Fusing consumer Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Write into Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Reify
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:48.680Z: Fusing consumer Read all files/Read all via
FileBasedSource/Read ranges into Read all files/Read all via
FileBasedSource/Reshuffle/Values/Values/Map
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:48.726Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
into Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:48.754Z: Fusing consumer Read all files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify into Read all files/Read
all via FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:48.795Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/GroupByWindow
into Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Read
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:48.841Z: Fusing consumer Read all files/Read all via
FileBasedSource/Reshuffle/Values/Values/Map into Read all files/Read all via
FileBasedSource/Reshuffle/Reshuffle/ExpandIterable
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:48.889Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Reify
into Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:48.930Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/Window.Into()/Window.Assign
into Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:48.969Z: Fusing consumer Read all files/Read all via
FileBasedSource/Reshuffle/Reshuffle/Window.Into()/Window.Assign into Read all
files/Read all via FileBasedSource/Reshuffle/Pair with random key
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.006Z: Fusing consumer Read all files/Read all via
FileBasedSource/Split into ranges into Read all
files/FileIO.ReadMatches/ParDo(ToReadableFile)
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.043Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Pair with random key
into Write Avro records to files/Write/FinalizeTempFileBundles/Finalize
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.074Z: Fusing consumer Read all
files/FileIO.MatchAll/Reshuffle.ViaRandomKey/Values/Values/Map into Read all
files/FileIO.MatchAll/Reshuffle.ViaRandomKey/Reshuffle/ExpandIterable
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.108Z: Fusing consumer Read all files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow into Read all
files/Read all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Read
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.139Z: Fusing consumer Read all files/Read all via
FileBasedSource/Reshuffle/Reshuffle/ExpandIterable into Read all files/Read all
via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/GroupByWindow
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.172Z: Fusing consumer Read all files/Read all via
FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Write into Read all files/Read
all via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Reify
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.213Z: Fusing consumer Read all files/Read all via
FileBasedSource/Reshuffle/Pair with random key into Read all files/Read all via
FileBasedSource/Split into ranges
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.252Z: Fusing consumer Write Avro records to
files/Write/FinalizeTempFileBundles/Finalize into Write Avro records to
files/Write/GatherTempFileResults/Reify.ReifyViewInGlobalWindow/Values/Values/Map
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.273Z: Unzipping flatten s9 for input
s5.writtenRecords
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.304Z: Fusing unzipped copy of Write Avro records
to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow),
through flatten Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/Flatten.PCollections, into
producer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.343Z: Fusing consumer Write Avro records to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)
into Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.375Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write into Write
Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.409Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify into Write
Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.441Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow into
Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Read
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.468Z: Fusing consumer Write Avro records to
files/Write/RewindowIntoGlobal/Window.Assign into Produce Avro records
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.491Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/DropShardNum into Write Avro
records to files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.542Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles into Write
Avro records to files/Write/RewindowIntoGlobal/Window.Assign
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.583Z: Fusing consumer Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnwritten into Write Avro
records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/GroupByWindow
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.616Z: Fusing consumer Produce text lines into
Generate sequence/Read(BoundedCountingSource)
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.647Z: Fusing consumer Produce Avro records into
Produce text lines
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.686Z: Fusing consumer
PAssert$0/VerifyAssertions/ParDo(DefaultConclude) into PAssert$0/RunChecks
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.716Z: Fusing consumer
PAssert$0/WindowToken/Window.Assign into
PAssert$0/Create.Values/Read(CreateSource)
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:49.753Z: Fusing consumer PAssert$0/RunChecks into
PAssert$0/WindowToken/Window.Assign
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:50.258Z: Executing operation Write Avro records to
files/Write/FinalizeTempFileBundles/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:50.289Z: Executing operation Read all
files/FileIO.MatchAll/Reshuffle.ViaRandomKey/Reshuffle/GroupByKey/Create
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:50.331Z: Executing operation Read all files/Read all
via FileBasedSource/Reshuffle/Reshuffle/GroupByKey/Create
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:50.332Z: Starting 1 workers in us-central1-b...
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:50.393Z: Executing operation Calculate
hashcode/Combine.perKey(Hashing)/GroupByKey/Create
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:50.446Z: Executing operation
PAssert$0/CreateActual/GatherPanes/GroupByKey/Create
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:50.492Z: Executing operation
PAssert$0/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:50.532Z: Executing operation Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Create
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:50.588Z: Executing operation
PAssert$0/CreateActual/View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:50.637Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/Combine.globally(Singleton)/Combine.perKey(Singleton)/GroupByKey/Create
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:50.712Z: Executing operation
View.AsSingleton/Combine.GloballyAsSingletonView/BatchViewOverrides.GroupByWindowHashAsKeyAndWindowAsSortKey/BatchViewOverrides.GroupByKeyAndSortValuesOnly/Create
Jun 21, 2018 6:17:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:51.156Z: Executing operation Generate
sequence/Read(BoundedCountingSource)+Produce text lines+Produce Avro
records+Write Avro records to
files/Write/RewindowIntoGlobal/Window.Assign+Write Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/WriteUnshardedBundles+Write Avro
records to
files/Write/GatherTempFileResults/View.AsList/ParDo(ToIsmRecordForGlobalWindow)+Write
Avro records to
files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Reify+Write Avro
records to files/Write/WriteUnshardedBundlesToTempFiles/GroupUnwritten/Write
Jun 21, 2018 6:18:00 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:17:58.661Z: Autoscaling: Raised the number of workers
to 0 based on the rate of progress in the currently running step(s).
Jun 21, 2018 6:18:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2018-06-21T18:18:06.921Z: Startup of the worker pool in zone
us-central1-b failed to bring up any of the desired 1 workers. Please check for
errors in your job parameters, check quota and retry later, or please try in a
different zone/region.
Jun 21, 2018 6:18:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2018-06-21T18:18:06.957Z: Workflow failed.
Jun 21, 2018 6:18:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:18:07.186Z: Cleaning up.
Jun 21, 2018 6:18:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2018-06-21T18:18:07.315Z: Stopping worker pool...
Jun 21, 2018 6:18:07 PM
org.apache.beam.runners.dataflow.TestDataflowRunner$ErrorMonitorMessagesHandler
process
INFO: Dataflow job 2018-06-21_11_17_37-7098979971903966146 threw exception.
Failure message was: Startup of the worker pool in zone us-central1-b failed to
bring up any of the desired 1 workers. Please check for errors in your job
parameters, check quota and retry later, or please try in a different
zone/region.
Jun 21, 2018 6:18:07 PM
org.apache.beam.runners.dataflow.TestDataflowRunner$ErrorMonitorMessagesHandler
process
INFO: Dataflow job 2018-06-21_11_17_37-7098979971903966146 threw exception.
Failure message was: Workflow failed.
Jun 21, 2018 6:18:26 PM
org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish
INFO: Job 2018-06-21_11_17_37-7098979971903966146 failed with status FAILED.
Jun 21, 2018 6:18:26 PM org.apache.beam.runners.dataflow.TestDataflowRunner
checkForPAssertSuccess
INFO: Dataflow job 2018-06-21_11_17_37-7098979971903966146 terminated in
failure state FAILED without reporting a failed assertion
Jun 21, 2018 6:18:26 PM org.apache.beam.runners.dataflow.TestDataflowRunner
run
WARNING: Dataflow job 2018-06-21_11_17_37-7098979971903966146 did not
output a success or failure metric.
Gradle Test Executor 1 finished executing tests.
> Task :beam-sdks-java-io-file-based-io-tests:integrationTest FAILED
org.apache.beam.sdk.io.avro.AvroIOIT > writeThenReadAll FAILED
java.lang.RuntimeException: Startup of the worker pool in zone
us-central1-b failed to bring up any of the desired 1 workers. Please check for
errors in your job parameters, check quota and retry later, or please try in a
different zone/region.Workflow failed.
at
org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:134)
at
org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:90)
at
org.apache.beam.runners.dataflow.TestDataflowRunner.run(TestDataflowRunner.java:55)
at org.apache.beam.sdk.Pipeline.run(Pipeline.java:311)
at org.apache.beam.sdk.testing.TestPipeline.run(TestPipeline.java:348)
at org.apache.beam.sdk.testing.TestPipeline.run(TestPipeline.java:329)
at
org.apache.beam.sdk.io.avro.AvroIOIT.writeThenReadAll(AvroIOIT.java:117)
Finished generating test XML results (0.019 secs) into:
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/sdks/java/io/file-based-io-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.03 secs) into:
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest>
:beam-sdks-java-io-file-based-io-tests:integrationTest (Thread[Task worker for
':' Thread 12,5,main]) completed. Took 59.053 secs.
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 5.0.
See
https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warnings
73 actionable tasks: 1 executed, 72 up-to-date
Publishing build scan...
https://gradle.com/s/hz5gw5tyi4ujq
STDERR:
1 test completed, 1 failed
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':beam-sdks-java-io-file-based-io-tests:integrationTest'.
> There were failing tests. See the report at:
> file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --debug option to get more log output. Run with --scan to get full
insights.
* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task
':beam-sdks-java-io-file-based-io-tests:integrationTest'.
at
org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:110)
at
org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:77)
at
org.gradle.api.internal.tasks.execution.OutputDirectoryCreatingTaskExecuter.execute(OutputDirectoryCreatingTaskExecuter.java:51)
at
org.gradle.api.internal.tasks.execution.SkipCachedTaskExecuter.execute(SkipCachedTaskExecuter.java:105)
at
org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:59)
at
org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54)
at
org.gradle.api.internal.tasks.execution.ResolveBuildCacheKeyExecuter.execute(ResolveBuildCacheKeyExecuter.java:66)
at
org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:59)
at
org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101)
at
org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44)
at
org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91)
at
org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62)
at
org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:59)
at
org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
at
org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
at
org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)
at
org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.run(EventFiringTaskExecuter.java:51)
at
org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317)
at
org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309)
at
org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185)
at
org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97)
at
org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
at
org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:46)
at
org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$ExecuteTaskAction.execute(DefaultTaskExecutionGraph.java:262)
at
org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$ExecuteTaskAction.execute(DefaultTaskExecutionGraph.java:246)
at
org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:136)
at
org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:130)
at
org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.execute(DefaultTaskPlanExecutor.java:201)
at
org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.executeWithTask(DefaultTaskPlanExecutor.java:192)
at
org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.run(DefaultTaskPlanExecutor.java:130)
at
org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
at
org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
at
org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
Caused by: org.gradle.api.GradleException: There were failing tests. See the
report at:
file://<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/sdks/java/io/file-based-io-tests/build/reports/tests/integrationTest/index.html>
at
org.gradle.api.tasks.testing.AbstractTestTask.handleTestFailures(AbstractTestTask.java:612)
at
org.gradle.api.tasks.testing.AbstractTestTask.executeTests(AbstractTestTask.java:484)
at org.gradle.api.tasks.testing.Test.executeTests(Test.java:603)
at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:73)
at
org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:46)
at
org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:39)
at
org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:26)
at
org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:794)
at
org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:761)
at
org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:131)
at
org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317)
at
org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309)
at
org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185)
at
org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97)
at
org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
at
org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:120)
at
org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:99)
... 32 more
* Get more help at https://help.gradle.org
BUILD FAILED in 1m 9s
2018-06-21 18:18:28,258 0f78a6a9 MainThread beam_integration_benchmark(1/1)
ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 667, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File
"<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 547, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File
"<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
line 159, in Run
job_type=job_type)
File
"<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,>
line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-06-21 18:18:28,259 0f78a6a9 MainThread beam_integration_benchmark(1/1)
INFO Cleaning up benchmark beam_integration_benchmark
2018-06-21 18:18:28,260 0f78a6a9 MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-322>
delete -f
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/LargeITCluster/hdfs-multi-datanode-cluster.yml>
--ignore-not-found
2018-06-21 18:20:05,749 0f78a6a9 MainThread beam_integration_benchmark(1/1)
ERROR Exception running benchmark
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 801, in RunBenchmarkTask
RunBenchmark(spec, collector)
File
"<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 667, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File
"<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 547, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File
"<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
line 159, in Run
job_type=job_type)
File
"<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,>
line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-06-21 18:20:05,750 0f78a6a9 MainThread beam_integration_benchmark(1/1)
ERROR Benchmark 1/1 beam_integration_benchmark (UID:
beam_integration_benchmark0) failed. Execution will continue.
2018-06-21 18:20:05,775 0f78a6a9 MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed
Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-06-21 18:20:05,776 0f78a6a9 MainThread INFO Complete logs can be found
at:
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/runs/0f78a6a9/pkb.log>
2018-06-21 18:20:05,776 0f78a6a9 MainThread INFO Completion statuses can be
found at:
<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/runs/0f78a6a9/completion_statuses.json>
Build step 'Execute shell' marked build as failure
