See <https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/696/display/redirect?page=changes>
Changes: [daniel.o.programmer] [BEAM-4780] Updating to DockerJobBundleFactory in ReferenceRunner. [timrobertson100] [BEAM-4861] Autocreate absent parent directories when doing an HDFS [timrobertson100] [BEAM-4861] Improve clarity around exceptions thrown in rename [vin1990] Endpoint host port configuration reset issue [vin1990] Endpoint host port configuration reset issue [vin1990] Fixes https://issues.apache.org/jira/browse/BEAM-3446. BaseReadFn to [vin1990] package private and typo fix [vin1990] Using mget with configurable batch size to increase efficiency of read [vin1990] fixing equality of batch size [vin1990] fixing order of test arguments, to fix [vin1990] Aggregating on per window, applyiong redis get on per window batch and [herohde] Fix Go index parsing in the face of runner-gernerated keys [herohde] Fix use of wrong key in Dataflow side input [timrobertson100] [BEAM-4861] Typo in comment [mxm] [BEAM-3089] Use Flink cluster parallelism if no parallelism provided [mxm] [flink] Revert default checkpointing mode to EXACTLY_ONCE [mxm] [flink] Use default value for checkpoint timeout [mxm] [flink] Set default master url to [auto] [mxm] [BEAM-3089] Test default values of FlinkPipelineOptions ------------------------------------------ [...truncated 208.30 KB...] > Task :beam-runners-core-java:classes UP-TO-DATE Skipping task ':beam-runners-core-java:classes' as it has no actions. :beam-runners-core-java:classes (Thread[Task worker for ':' Thread 15,5,main]) completed. Took 0.0 secs. > Task :beam-runners-google-cloud-dataflow-java:classes UP-TO-DATE Skipping task ':beam-runners-google-cloud-dataflow-java:classes' as it has no actions. :beam-runners-google-cloud-dataflow-java:classes (Thread[Task worker for ':' Thread 10,5,main]) completed. Took 0.0 secs. :beam-runners-core-java:shadowJar (Thread[Task worker for ':' Thread 15,5,main]) started. :beam-runners-google-cloud-dataflow-java:shadowJar (Thread[Task worker for ':' Thread 10,5,main]) started. > Task :beam-runners-core-java:shadowJar Build cache key for task ':beam-runners-core-java:shadowJar' is d0f7cfa3b14a858e43302fb16dd7abb7 Caching disabled for task ':beam-runners-core-java:shadowJar': Caching has not been enabled for the task Task ':beam-runners-core-java:shadowJar' is not up-to-date because: Input property 'configurations' file <https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.8.0-SNAPSHOT.jar> has changed. Input property 'configurations' file <https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.8.0-SNAPSHOT.jar> has changed. Input property 'configurations' file <https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/model/job-management/build/libs/beam-model-job-management-2.8.0-SNAPSHOT.jar> has changed. ******************* GRADLE SHADOW STATS Total Jars: 2 (includes project) Total Time: 2.044s [2044ms] Average Time/Jar: 1.022s [1022.0ms] ******************* :beam-runners-core-java:shadowJar (Thread[Task worker for ':' Thread 15,5,main]) completed. Took 2.361 secs. :beam-sdks-java-harness:compileJava (Thread[Task worker for ':' Thread 15,5,main]) started. > Task :beam-runners-google-cloud-dataflow-java:shadowJar Build cache key for task ':beam-runners-google-cloud-dataflow-java:shadowJar' is 2281a45dec6c347449cbce9943dbd17f Caching disabled for task ':beam-runners-google-cloud-dataflow-java:shadowJar': Caching has not been enabled for the task Task ':beam-runners-google-cloud-dataflow-java:shadowJar' is not up-to-date because: Input property 'configurations' file <https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.8.0-SNAPSHOT.jar> has changed. Input property 'configurations' file <https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/model/job-management/build/libs/beam-model-job-management-2.8.0-SNAPSHOT.jar> has changed. ******************* GRADLE SHADOW STATS Total Jars: 2 (includes project) Total Time: 2.059s [2059ms] Average Time/Jar: 1.0295s [1029.5ms] ******************* :beam-runners-google-cloud-dataflow-java:shadowJar (Thread[Task worker for ':' Thread 10,5,main]) completed. Took 2.373 secs. > Task :beam-sdks-java-harness:compileJava UP-TO-DATE Build cache key for task ':beam-sdks-java-harness:compileJava' is 3c828f04afed8a2eccefac8fe4f6f067 Skipping task ':beam-sdks-java-harness:compileJava' as it is up-to-date. :beam-sdks-java-harness:compileJava (Thread[Task worker for ':' Thread 15,5,main]) completed. Took 0.571 secs. :beam-sdks-java-harness:classes (Thread[Task worker for ':' Thread 15,5,main]) started. > Task :beam-sdks-java-harness:classes UP-TO-DATE Skipping task ':beam-sdks-java-harness:classes' as it has no actions. :beam-sdks-java-harness:classes (Thread[Task worker for ':' Thread 15,5,main]) completed. Took 0.0 secs. :beam-sdks-java-harness:jar (Thread[Task worker for ':' Thread 15,5,main]) started. > Task :beam-sdks-java-harness:jar UP-TO-DATE Build cache key for task ':beam-sdks-java-harness:jar' is 4a07dfcfa99b023191a684da42957e62 Caching disabled for task ':beam-sdks-java-harness:jar': Caching has not been enabled for the task Skipping task ':beam-sdks-java-harness:jar' as it is up-to-date. :beam-sdks-java-harness:jar (Thread[Task worker for ':' Thread 15,5,main]) completed. Took 0.004 secs. :beam-sdks-java-harness:shadowJar (Thread[Task worker for ':' Thread 15,5,main]) started. > Task :beam-sdks-java-harness:shadowJar Build cache key for task ':beam-sdks-java-harness:shadowJar' is 8fee50317cfad05da2123990a8b2d705 Caching disabled for task ':beam-sdks-java-harness:shadowJar': Caching has not been enabled for the task Task ':beam-sdks-java-harness:shadowJar' is not up-to-date because: Input property 'configurations' file <https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.8.0-SNAPSHOT.jar> has changed. Input property 'configurations' file <https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/runners/core-java/build/libs/beam-runners-core-java-2.8.0-SNAPSHOT.jar> has changed. Input property 'configurations' file <https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.8.0-SNAPSHOT.jar> has changed. ******************* GRADLE SHADOW STATS Total Jars: 43 (includes project) Total Time: 34.559s [34559ms] Average Time/Jar: 0.8036976744186s [803.6976744186ms] ******************* :beam-sdks-java-harness:shadowJar (Thread[Task worker for ':' Thread 15,5,main]) completed. Took 34.998 secs. :beam-runners-java-fn-execution:compileJava (Thread[Task worker for ':' Thread 15,5,main]) started. > Task :beam-runners-java-fn-execution:compileJava UP-TO-DATE Build cache key for task ':beam-runners-java-fn-execution:compileJava' is 1b1112ee2795e221b0f72b4b06e0a81d Skipping task ':beam-runners-java-fn-execution:compileJava' as it is up-to-date. :beam-runners-java-fn-execution:compileJava (Thread[Task worker for ':' Thread 15,5,main]) completed. Took 4.776 secs. :beam-runners-java-fn-execution:classes (Thread[Task worker for ':' Thread 15,5,main]) started. > Task :beam-runners-java-fn-execution:classes UP-TO-DATE Skipping task ':beam-runners-java-fn-execution:classes' as it has no actions. :beam-runners-java-fn-execution:classes (Thread[Task worker for ':' Thread 15,5,main]) completed. Took 0.0 secs. :beam-runners-java-fn-execution:shadowJar (Thread[Task worker for ':',5,main]) started. > Task :beam-runners-java-fn-execution:shadowJar Build cache key for task ':beam-runners-java-fn-execution:shadowJar' is 7c44c3968180f91a3c1c70f590f05342 Caching disabled for task ':beam-runners-java-fn-execution:shadowJar': Caching has not been enabled for the task Task ':beam-runners-java-fn-execution:shadowJar' is not up-to-date because: Input property 'configurations' file <https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.8.0-SNAPSHOT.jar> has changed. Input property 'configurations' file <https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.8.0-SNAPSHOT.jar> has changed. Input property 'configurations' file <https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.8.0-SNAPSHOT.jar> has changed. ******************* GRADLE SHADOW STATS Total Jars: 2 (includes project) Total Time: 2.126s [2126ms] Average Time/Jar: 1.063s [1063.0ms] ******************* :beam-runners-java-fn-execution:shadowJar (Thread[Task worker for ':',5,main]) completed. Took 2.299 secs. :beam-runners-direct-java:compileJava (Thread[Task worker for ':',5,main]) started. > Task :beam-runners-direct-java:compileJava Build cache key for task ':beam-runners-direct-java:compileJava' is 60ba5015eff18c8c5368637cad47aa39 Task ':beam-runners-direct-java:compileJava' is not up-to-date because: Task has failed previously. Custom actions are attached to task ':beam-runners-direct-java:compileJava'. All input files are considered out-of-date for incremental task ':beam-runners-direct-java:compileJava'. Compiling with error-prone compiler > Task :beam-runners-direct-java:compileJava FAILED :beam-runners-direct-java:compileJava (Thread[Task worker for ':',5,main]) completed. Took 7.802 secs. Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0. See https://docs.gradle.org/4.8/userguide/command_line_interface.html#sec:command_line_warnings 58 actionable tasks: 11 executed, 47 up-to-date Publishing build scan... https://gradle.com/s/6zyfah52kys7a Build cache (/home/jenkins/.gradle/caches/build-cache-1) remove files older than Wed Sep 12 18:58:14 UTC 2018. Build cache (/home/jenkins/.gradle/caches/build-cache-1) removing 1247 cache entries (216 MB reclaimed). Build cache (/home/jenkins/.gradle/caches/build-cache-1) cleaned up in 0.205 secs. STDERR: <https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/runners/direct-java/src/main/java/org/apache/beam/runners/direct/portable/ReferenceRunner.java>:268: error: cannot find symbol return DockerJobBundleFactory.create(jobInfo); ^ symbol: method create(JobInfo) location: class DockerJobBundleFactory Note: Some input files use or override a deprecated API. Note: Recompile with -Xlint:deprecation for details. Note: Some input files use unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details. 1 error FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':beam-runners-direct-java:compileJava'. > Compilation failed with exit code 1; see the compiler error output for > details. * Try: Run with --debug option to get more log output. Run with --scan to get full insights. * Exception is: org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':beam-runners-direct-java:compileJava'. at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:110) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:77) at org.gradle.api.internal.tasks.execution.OutputDirectoryCreatingTaskExecuter.execute(OutputDirectoryCreatingTaskExecuter.java:51) at org.gradle.api.internal.tasks.execution.SkipCachedTaskExecuter.execute(SkipCachedTaskExecuter.java:105) at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:59) at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54) at org.gradle.api.internal.tasks.execution.ResolveBuildCacheKeyExecuter.execute(ResolveBuildCacheKeyExecuter.java:66) at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:59) at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101) at org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44) at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91) at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62) at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:59) at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54) at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43) at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.run(EventFiringTaskExecuter.java:51) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309) at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185) at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97) at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:46) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$ExecuteTaskAction.execute(DefaultTaskExecutionGraph.java:262) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$ExecuteTaskAction.execute(DefaultTaskExecutionGraph.java:246) at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:136) at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:130) at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.execute(DefaultTaskPlanExecutor.java:201) at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.executeWithTask(DefaultTaskPlanExecutor.java:192) at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.run(DefaultTaskPlanExecutor.java:130) at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63) at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46) at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55) Caused by: org.gradle.api.internal.tasks.compile.CompilationFailedException: Compilation failed with exit code 1; see the compiler error output for details. at net.ltgt.gradle.errorprone.ErrorProneCompiler.execute(ErrorProneCompiler.java:74) at net.ltgt.gradle.errorprone.ErrorProneCompiler.execute(ErrorProneCompiler.java:23) at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.delegateAndHandleErrors(NormalizingJavaCompiler.java:86) at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.execute(NormalizingJavaCompiler.java:47) at org.gradle.api.internal.tasks.compile.NormalizingJavaCompiler.execute(NormalizingJavaCompiler.java:33) at org.gradle.api.internal.tasks.compile.CleaningJavaCompilerSupport.execute(CleaningJavaCompilerSupport.java:40) at org.gradle.api.internal.tasks.compile.CleaningJavaCompilerSupport.execute(CleaningJavaCompilerSupport.java:27) at org.gradle.api.tasks.compile.JavaCompile.performCompilation(JavaCompile.java:161) at org.gradle.api.tasks.compile.JavaCompile.compile(JavaCompile.java:146) at org.gradle.api.tasks.compile.JavaCompile.compile(JavaCompile.java:118) at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:73) at org.gradle.api.internal.project.taskfactory.IncrementalTaskAction.doExecute(IncrementalTaskAction.java:50) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:39) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:26) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:131) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309) at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185) at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97) at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:120) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:99) ... 32 more * Get more help at https://help.gradle.org BUILD FAILED in 1m 32s 2018-09-19 18:59:43,601 26259051 MainThread beam_integration_benchmark(1/1) ERROR Error during benchmark beam_integration_benchmark Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,> line 695, in RunBenchmark DoRunPhase(spec, collector, detailed_timer) File "<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,> line 575, in DoRunPhase samples = spec.BenchmarkRun(spec) File "<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,> line 160, in Run job_type=job_type) File "<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,> line 90, in SubmitJob assert retcode == 0, "Integration Test Failed." AssertionError: Integration Test Failed. 2018-09-19 18:59:43,602 26259051 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark 2018-09-19 18:59:43,602 26259051 MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-beam-performancetests-mongodbio-it-696> delete -f <https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/.test-infra/kubernetes/mongodb/load-balancer/mongo.yml> --ignore-not-found 2018-09-19 18:59:44,149 26259051 MainThread beam_integration_benchmark(1/1) ERROR Exception running benchmark Traceback (most recent call last): File "<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,> line 832, in RunBenchmarkTask RunBenchmark(spec, collector) File "<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,> line 695, in RunBenchmark DoRunPhase(spec, collector, detailed_timer) File "<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,> line 575, in DoRunPhase samples = spec.BenchmarkRun(spec) File "<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,> line 160, in Run job_type=job_type) File "<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,> line 90, in SubmitJob assert retcode == 0, "Integration Test Failed." AssertionError: Integration Test Failed. 2018-09-19 18:59:44,150 26259051 MainThread beam_integration_benchmark(1/1) ERROR Benchmark 1/1 beam_integration_benchmark (UID: beam_integration_benchmark0) failed. Execution will continue. 2018-09-19 18:59:44,201 26259051 MainThread INFO Benchmark run statuses: --------------------------------------------------------------------------------- Name UID Status Failed Substatus --------------------------------------------------------------------------------- beam_integration_benchmark beam_integration_benchmark0 FAILED --------------------------------------------------------------------------------- Success rate: 0.00% (0/1) 2018-09-19 18:59:44,202 26259051 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/runs/26259051/pkb.log> 2018-09-19 18:59:44,203 26259051 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/runs/26259051/completion_statuses.json> Build step 'Execute shell' marked build as failure
