See <https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT/473/display/redirect?page=changes>
Changes: [Pablo] Make experiments as set attr of RuntimeValueProvider ------------------------------------------ [...truncated 50.06 KB...] > Task :beam-sdks-java-build-tools:install > Task :beam-sdks-python:setupVirtualenv New python executable in <https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT/ws/runs/19099ac3/beam/sdks/python/build/gradleenv/bin/python2> Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT/ws/runs/19099ac3/beam/sdks/python/build/gradleenv/bin/python> Installing setuptools, pkg_resources, pip, wheel... > Task :beam-sdks-java-core:compileJava FROM-CACHE > Task :beam-sdks-java-core:processResources > Task :beam-sdks-java-core:classes > Task :beam-model-pipeline:shadowJar > Task :beam-model-fn-execution:extractIncludeProto > Task :beam-model-job-management:extractIncludeProto > Task :beam-model-pipeline:jar > Task :beam-model-job-management:generateProto > Task :beam-model-pipeline:extractIncludeTestProto > Task :beam-model-pipeline:extractTestProto > Task :beam-model-pipeline:generateTestProto NO-SOURCE > Task :beam-model-pipeline:compileTestJava NO-SOURCE > Task :beam-model-pipeline:processTestResources NO-SOURCE > Task :beam-model-pipeline:testClasses UP-TO-DATE > Task :beam-model-pipeline:packageTests > Task :beam-model-fn-execution:generateProto > Task :beam-model-pipeline:install > Task :beam-model-job-management:compileJava FROM-CACHE > Task :beam-model-job-management:classes > Task :beam-model-fn-execution:compileJava FROM-CACHE > Task :beam-model-fn-execution:classes > Task :beam-sdks-python:setupVirtualenv done. Running virtualenv with interpreter /usr/bin/python2 Collecting tox==3.0.0 Using cached https://files.pythonhosted.org/packages/e6/41/4dcfd713282bf3213b0384320fa8841e4db032ddcb80bc08a540159d42a8/tox-3.0.0-py2.py3-none-any.whl Collecting grpcio-tools==1.3.5 Using cached https://files.pythonhosted.org/packages/05/f6/0296e29b1bac6f85d2a8556d48adf825307f73109a3c2c17fb734292db0a/grpcio_tools-1.3.5-cp27-cp27mu-manylinux1_x86_64.whl Collecting pluggy<1.0,>=0.3.0 (from tox==3.0.0) Using cached https://files.pythonhosted.org/packages/82/05/43e3947125a2137cba4746135c75934ceed1863f27e050fc560052104a71/pluggy-0.6.0-py2-none-any.whl Requirement not upgraded as not directly required: six in /usr/local/lib/python2.7/dist-packages (from tox==3.0.0) (1.11.0) Requirement not upgraded as not directly required: virtualenv>=1.11.2 in /usr/lib/python2.7/dist-packages (from tox==3.0.0) (15.0.1) Collecting py>=1.4.17 (from tox==3.0.0) Using cached https://files.pythonhosted.org/packages/67/a5/f77982214dd4c8fd104b066f249adea2c49e25e8703d284382eb5e9ab35a/py-1.5.3-py2.py3-none-any.whl Collecting grpcio>=1.3.5 (from grpcio-tools==1.3.5) Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl Collecting protobuf>=3.2.0 (from grpcio-tools==1.3.5) Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl Requirement not upgraded as not directly required: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio>=1.3.5->grpcio-tools==1.3.5) (1.1.6) Collecting futures>=2.2.0 (from grpcio>=1.3.5->grpcio-tools==1.3.5) Using cached https://files.pythonhosted.org/packages/2d/99/b2c4e9d5a30f6471e410a146232b4118e697fa3ffc06d6a65efde84debd0/futures-3.2.0-py2-none-any.whl Requirement not upgraded as not directly required: setuptools in /usr/local/lib/python2.7/dist-packages (from protobuf>=3.2.0->grpcio-tools==1.3.5) (39.0.1) > Task :beam-model-job-management:shadowJar > Task :beam-model-job-management:jar > Task :beam-model-job-management:extractIncludeTestProto > Task :beam-model-job-management:generateTestProto NO-SOURCE > Task :beam-model-job-management:compileTestJava NO-SOURCE > Task :beam-model-job-management:testClasses UP-TO-DATE > Task :beam-model-job-management:packageTests > Task :beam-model-job-management:install > Task :beam-model-fn-execution:shadowJar > Task :beam-model-fn-execution:jar > Task :beam-model-fn-execution:extractIncludeTestProto > Task :beam-model-fn-execution:generateTestProto NO-SOURCE > Task :beam-model-fn-execution:compileTestJava NO-SOURCE > Task :beam-model-fn-execution:testClasses UP-TO-DATE > Task :beam-model-fn-execution:packageTests > Task :beam-model-fn-execution:install > Task :beam-sdks-python:setupVirtualenv FAILED Installing collected packages: pluggy, py, tox, protobuf, futures, grpcio, grpcio-tools Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/pluggy-0.6.0.dist-info' Consider using the `--user` option or check the permissions. > Task :beam-sdks-java-extensions-sql:generateFmppSources > Task :beam-model-fn-execution:shadowTestJar > Task :beam-sdks-java-core:shadowJar Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0. See https://docs.gradle.org/4.7/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 43s 139 actionable tasks: 132 executed, 5 from cache, 2 up-to-date Publishing build scan... https://gradle.com/s/x4j3t7yohepy6 STDERR: FAILURE: Build failed with an exception. * Where: Build file '<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT/ws/runs/19099ac3/beam/sdks/python/build.gradle'> line: 36 * What went wrong: Execution failed for task ':beam-sdks-python:setupVirtualenv'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Exception is: org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':beam-sdks-python:setupVirtualenv'. at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:103) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:73) at org.gradle.api.internal.tasks.execution.OutputDirectoryCreatingTaskExecuter.execute(OutputDirectoryCreatingTaskExecuter.java:51) at org.gradle.api.internal.tasks.execution.SkipCachedTaskExecuter.execute(SkipCachedTaskExecuter.java:105) at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:59) at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54) at org.gradle.api.internal.tasks.execution.ResolveBuildCacheKeyExecuter.execute(ResolveBuildCacheKeyExecuter.java:66) at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:59) at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101) at org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44) at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91) at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62) at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:59) at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54) at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43) at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34) at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker$1.run(DefaultTaskGraphExecuter.java:256) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309) at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185) at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97) at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31) at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:249) at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:238) at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:104) at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:98) at org.gradle.execution.taskgraph.DefaultTaskExecutionPlan.execute(DefaultTaskExecutionPlan.java:663) at org.gradle.execution.taskgraph.DefaultTaskExecutionPlan.executeWithTask(DefaultTaskExecutionPlan.java:596) at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.run(DefaultTaskPlanExecutor.java:98) at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63) at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46) at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55) Caused by: org.gradle.process.internal.ExecException: Process 'command 'sh'' finished with non-zero exit value 1 at org.gradle.process.internal.DefaultExecHandle$ExecResultImpl.assertNormalExitValue(DefaultExecHandle.java:389) at org.gradle.process.internal.DefaultExecAction.execute(DefaultExecAction.java:36) at org.gradle.api.internal.file.DefaultFileOperations.exec(DefaultFileOperations.java:192) at org.gradle.api.internal.project.DefaultProject.exec(DefaultProject.java:1087) at org.gradle.groovy.scripts.DefaultScript.exec(DefaultScript.java:253) at org.gradle.internal.metaobject.BeanDynamicObject$MetaClassAdapter.invokeMethod(BeanDynamicObject.java:479) at org.gradle.internal.metaobject.BeanDynamicObject.tryInvokeMethod(BeanDynamicObject.java:191) at org.gradle.groovy.scripts.BasicScript$ScriptDynamicObject.tryInvokeMethod(BasicScript.java:130) at org.gradle.internal.metaobject.ConfigureDelegate.invokeMethod(ConfigureDelegate.java:78) at build_5yyvyzzl6i4rvt28c3cwv2e93$_run_closure2$_closure16.doCall(<https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT/ws/runs/19099ac3/beam/sdks/python/build.gradle>:36) at org.gradle.api.internal.AbstractTask$ClosureTaskAction.execute(AbstractTask.java:732) at org.gradle.api.internal.AbstractTask$ClosureTaskAction.execute(AbstractTask.java:705) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:124) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309) at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185) at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97) at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:113) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:95) ... 31 more * Get more help at https://help.gradle.org 2018-05-09 06:01:34,332 19099ac3 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark 2018-05-09 06:01:34,344 19099ac3 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT/ws/runs/19099ac3/beam/gradlew> integrationTest --tests=org.apache.beam.sdk.io.text.TextIOIT -p sdks/java/io/file-based-io-tests -DintegrationTestRunner=dataflow -Dfilesystem=gcs -DintegrationTestPipelineOptions=["--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_Compressed_TextIOIT/473/","--numberOfRecords=1000000","--compressionType=GZIP","--runner=TestDataflowRunner"] --stacktrace --info 2018-05-09 06:08:11,919 19099ac3 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark 2018-05-09 06:08:11,920 19099ac3 MainThread beam_integration_benchmark(1/1) INFO Tearing down resources for benchmark beam_integration_benchmark 2018-05-09 06:08:11,967 19099ac3 MainThread INFO Publishing 2 samples to <https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT/ws/runs/19099ac3/perfkit-bq-pubyiYXCG.json> 2018-05-09 06:08:11,968 19099ac3 MainThread INFO Publishing 2 samples to beam_performance.compressed_textioit_pkb_results 2018-05-09 06:08:11,968 19099ac3 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.compressed_textioit_pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT/ws/runs/19099ac3/perfkit-bq-pubyiYXCG.json> 2018-05-09 06:08:15,927 19099ac3 MainThread INFO -------------------------PerfKitBenchmarker Complete Results------------------------- {'metadata': {'dpb_cluster_id': None, 'dpb_cluster_shape': 'n1-standard-1', 'dpb_cluster_size': 2, 'dpb_dataflow_runner': 'DataflowRunner', 'dpb_dataflow_sdk': None, 'dpb_service': 'dataflow', 'perfkitbenchmarker_version': 'v1.12.0-580-g36cfc82', 'run_number': 0}, 'metric': 'run_time', 'official': True, 'owner': 'jenkins', 'product_name': 'PerfKitBenchmarker', 'run_uri': '19099ac3-435f4d64-39ba-4cff-b91d-f8bc06eb0d84', 'sample_uri': '902d5f09-cc58-4108-8684-28725569eddd', 'test': 'beam_integration_benchmark', 'timestamp': 1525846091.918777, 'unit': 'seconds', 'value': 397.585004} {'metadata': {'perfkitbenchmarker_version': 'v1.12.0-580-g36cfc82'}, 'metric': 'End to End Runtime', 'official': True, 'owner': 'jenkins', 'product_name': 'PerfKitBenchmarker', 'run_uri': '19099ac3-435f4d64-39ba-4cff-b91d-f8bc06eb0d84', 'sample_uri': '7b63adc8-da55-4656-b461-dab773cb6bd2', 'test': 'beam_integration_benchmark', 'timestamp': 1525846091.920672, 'unit': 'seconds', 'value': 452.4898889064789} -------------------------PerfKitBenchmarker Results Summary------------------------- BEAM_INTEGRATION_BENCHMARK: dpb_cluster_shape="n1-standard-1" dpb_cluster_size="2" dpb_dataflow_runner="DataflowRunner" dpb_service="dataflow" run_number="0" run_time 397.585004 seconds End to End Runtime 452.489889 seconds ------------------------- For all tests: dpb_cluster_id="None" dpb_dataflow_sdk="None" perfkitbenchmarker_version="v1.12.0-580-g36cfc82" 2018-05-09 06:08:15,928 19099ac3 MainThread INFO Publishing 2 samples to <https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT/ws/runs/19099ac3/perfkitbenchmarker_results.json> 2018-05-09 06:08:15,928 19099ac3 MainThread INFO Benchmark run statuses: ------------------------------------------------------------------------------------ Name UID Status Failed Substatus ------------------------------------------------------------------------------------ beam_integration_benchmark beam_integration_benchmark0 SUCCEEDED ------------------------------------------------------------------------------------ Success rate: 100.00% (1/1) 2018-05-09 06:08:15,928 19099ac3 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT/ws/runs/19099ac3/pkb.log> 2018-05-09 06:08:15,928 19099ac3 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_Compressed_TextIOIT/ws/runs/19099ac3/completion_statuses.json> [Set GitHub commit status (universal)] SUCCESS on repos [GHRepository@2b180ef3[description=Apache Beam,homepage=,name=beam,fork=false,size=58754,milestones={},language=Java,commits={},source=<null>,parent=<null>,responseHeaderFields={null=[HTTP/1.1 200 OK], Access-Control-Allow-Origin=[*], Access-Control-Expose-Headers=[ETag, Link, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval], Cache-Control=[private, max-age=60, s-maxage=60], Content-Encoding=[gzip], Content-Security-Policy=[default-src 'none'], Content-Type=[application/json; charset=utf-8], Date=[Wed, 09 May 2018 06:08:16 GMT], ETag=[W/"b325fcee1bcdcb6673ffb3a642ac3060"], Last-Modified=[Wed, 09 May 2018 00:26:13 GMT], OkHttp-Received-Millis=[1525846096704], OkHttp-Response-Source=[NETWORK 200], OkHttp-Selected-Protocol=[http/1.1], OkHttp-Sent-Millis=[1525846096569], Referrer-Policy=[origin-when-cross-origin, strict-origin-when-cross-origin], Server=[GitHub.com], Status=[200 OK], Strict-Transport-Security=[max-age=31536000; includeSubdomains; preload], Transfer-Encoding=[chunked], Vary=[Accept, Authorization, Cookie, X-GitHub-OTP], X-Accepted-OAuth-Scopes=[repo], X-Content-Type-Options=[nosniff], X-Frame-Options=[deny], X-GitHub-Media-Type=[github.v3; format=json], X-GitHub-Request-Id=[DE20:4F40:98C70E:14FC582:5AF29043], X-OAuth-Scopes=[admin:repo_hook, repo, repo:status], X-RateLimit-Limit=[5000], X-RateLimit-Remaining=[4955], X-RateLimit-Reset=[1525848688], X-Runtime-rack=[0.051890], X-XSS-Protection=[1; mode=block]},url=https://api.github.com/repos/apache/beam,id=50904245]] (sha:60f90c8) with context:beam_PerformanceTests_Compressed_TextIOIT Setting commit status on GitHub for https://github.com/apache/beam/commit/60f90c8dcb229c35a82c7be15e64a89678bae058 ERROR: Build step failed with exception java.io.FileNotFoundException: https://api.github.com/repos/apache/beam/statuses/60f90c8dcb229c35a82c7be15e64a89678bae058 at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243) at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210) at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25) at org.kohsuke.github.Requester.parse(Requester.java:612) at org.kohsuke.github.Requester.parse(Requester.java:594) at org.kohsuke.github.Requester._to(Requester.java:272) Caused: org.kohsuke.github.GHFileNotFoundException: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/repos/statuses/#create-a-status"} at org.kohsuke.github.Requester.handleApiError(Requester.java:686) at org.kohsuke.github.Requester._to(Requester.java:293) at org.kohsuke.github.Requester.to(Requester.java:234) at org.kohsuke.github.GHRepository.createCommitStatus(GHRepository.java:1075) at org.jenkinsci.plugins.github.status.GitHubCommitStatusSetter.perform(GitHubCommitStatusSetter.java:160) Caused: org.jenkinsci.plugins.github.common.CombineErrorHandler$ErrorHandlingException at org.jenkinsci.plugins.github.common.CombineErrorHandler.handle(CombineErrorHandler.java:74) at org.jenkinsci.plugins.github.status.GitHubCommitStatusSetter.perform(GitHubCommitStatusSetter.java:164) at com.cloudbees.jenkins.GitHubCommitNotifier.perform(GitHubCommitNotifier.java:151) at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744) at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690) at hudson.model.Build$BuildExecution.post2(Build.java:186) at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635) at hudson.model.Run.execute(Run.java:1749) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Build step 'Set build status on GitHub commit [deprecated]' marked build as failure
