See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/475/display/redirect?page=changes>
Changes: [Pablo] Make experiments as set attr of RuntimeValueProvider ------------------------------------------ [...truncated 50.49 KB...] > Task :beam-model-pipeline:jar > Task :beam-model-job-management:generateProto > Task :beam-model-pipeline:extractIncludeTestProto > Task :beam-model-pipeline:extractTestProto > Task :beam-model-pipeline:generateTestProto NO-SOURCE > Task :beam-model-pipeline:compileTestJava NO-SOURCE > Task :beam-model-pipeline:processTestResources NO-SOURCE > Task :beam-model-pipeline:testClasses UP-TO-DATE > Task :beam-model-pipeline:packageTests > Task :beam-model-fn-execution:generateProto > Task :beam-sdks-java-extensions-sql:compileJavacc Java Compiler Compiler Version 4.0 (Parser Generator) (type "javacc" with no arguments for help) Warning: Bad option "-grammar_encoding=UTF-8" will be ignored. Reading from file <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/runs/dd1c6c58/beam/sdks/java/extensions/sql/build/generated/fmpp/javacc/Parser.jj> . . . Note: UNICODE_INPUT option is specified. Please make sure you create the parser/lexer using a Reader with the correct character encoding. Warning: Lookahead adequacy checking not being performed since option LOOKAHEAD is more than 1. Set option FORCE_LA_CHECK to true to force checking. File "TokenMgrError.java" does not exist. Will create one. File "ParseException.java" does not exist. Will create one. File "Token.java" does not exist. Will create one. File "SimpleCharStream.java" does not exist. Will create one. Parser generated with 0 errors and 1 warnings. > Task :beam-model-pipeline:install > Task :beam-sdks-java-extensions-sql:processResources > Task :beam-sdks-java-extensions-sql:processTestResources NO-SOURCE > Task :beam-model-job-management:compileJava FROM-CACHE > Task :beam-model-job-management:classes > Task :beam-model-fn-execution:compileJava FROM-CACHE > Task :beam-model-fn-execution:classes > Task :beam-sdks-python:setupVirtualenv done. Running virtualenv with interpreter /usr/bin/python2 Requirement already up-to-date: tox==3.0.0 in /home/jenkins/.local/lib/python2.7/site-packages (3.0.0) Collecting grpcio-tools==1.3.5 Using cached https://files.pythonhosted.org/packages/05/f6/0296e29b1bac6f85d2a8556d48adf825307f73109a3c2c17fb734292db0a/grpcio_tools-1.3.5-cp27-cp27mu-manylinux1_x86_64.whl Requirement not upgraded as not directly required: pluggy<1.0,>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from tox==3.0.0) (0.6.0) Requirement not upgraded as not directly required: six in /usr/local/lib/python2.7/dist-packages (from tox==3.0.0) (1.11.0) Requirement not upgraded as not directly required: virtualenv>=1.11.2 in /home/jenkins/.local/lib/python2.7/site-packages (from tox==3.0.0) (15.2.0) Requirement not upgraded as not directly required: py>=1.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from tox==3.0.0) (1.5.3) Collecting grpcio>=1.3.5 (from grpcio-tools==1.3.5) Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl Collecting protobuf>=3.2.0 (from grpcio-tools==1.3.5) Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl Requirement not upgraded as not directly required: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio>=1.3.5->grpcio-tools==1.3.5) (1.1.6) Collecting futures>=2.2.0 (from grpcio>=1.3.5->grpcio-tools==1.3.5) Using cached https://files.pythonhosted.org/packages/2d/99/b2c4e9d5a30f6471e410a146232b4118e697fa3ffc06d6a65efde84debd0/futures-3.2.0-py2-none-any.whl Requirement not upgraded as not directly required: setuptools in /usr/local/lib/python2.7/dist-packages (from protobuf>=3.2.0->grpcio-tools==1.3.5) (39.0.1) > Task :beam-model-job-management:shadowJar > Task :beam-model-job-management:jar > Task :beam-model-job-management:extractIncludeTestProto > Task :beam-model-job-management:generateTestProto NO-SOURCE > Task :beam-model-job-management:compileTestJava NO-SOURCE > Task :beam-model-job-management:testClasses UP-TO-DATE > Task :beam-model-job-management:packageTests > Task :beam-model-fn-execution:shadowJar > Task :beam-model-job-management:install > Task :beam-model-fn-execution:jar > Task :beam-model-fn-execution:extractIncludeTestProto > Task :beam-model-fn-execution:generateTestProto NO-SOURCE > Task :beam-model-fn-execution:compileTestJava NO-SOURCE > Task :beam-model-fn-execution:testClasses UP-TO-DATE > Task :beam-model-fn-execution:packageTests > Task :beam-model-fn-execution:install > Task :beam-sdks-python:setupVirtualenv FAILED Installing collected packages: protobuf, futures, grpcio, grpcio-tools Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/protobuf-3.5.2.post1-py2.7-nspkg.pth' Consider using the `--user` option or check the permissions. > Task :beam-model-fn-execution:shadowTestJar > Task :beam-sdks-java-core:shadowJar Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0. See https://docs.gradle.org/4.7/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 34s 141 actionable tasks: 134 executed, 5 from cache, 2 up-to-date Publishing build scan... https://gradle.com/s/tsgsh4hkk54du STDERR: FAILURE: Build failed with an exception. * Where: Build file '<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/runs/dd1c6c58/beam/sdks/python/build.gradle'> line: 36 * What went wrong: Execution failed for task ':beam-sdks-python:setupVirtualenv'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Exception is: org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':beam-sdks-python:setupVirtualenv'. at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:103) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:73) at org.gradle.api.internal.tasks.execution.OutputDirectoryCreatingTaskExecuter.execute(OutputDirectoryCreatingTaskExecuter.java:51) at org.gradle.api.internal.tasks.execution.SkipCachedTaskExecuter.execute(SkipCachedTaskExecuter.java:105) at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:59) at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54) at org.gradle.api.internal.tasks.execution.ResolveBuildCacheKeyExecuter.execute(ResolveBuildCacheKeyExecuter.java:66) at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:59) at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101) at org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44) at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91) at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62) at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:59) at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54) at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43) at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34) at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker$1.run(DefaultTaskGraphExecuter.java:256) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309) at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185) at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97) at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31) at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:249) at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:238) at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:104) at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker$1.execute(DefaultTaskPlanExecutor.java:98) at org.gradle.execution.taskgraph.DefaultTaskExecutionPlan.execute(DefaultTaskExecutionPlan.java:663) at org.gradle.execution.taskgraph.DefaultTaskExecutionPlan.executeWithTask(DefaultTaskExecutionPlan.java:596) at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor$TaskExecutorWorker.run(DefaultTaskPlanExecutor.java:98) at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63) at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46) at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55) Caused by: org.gradle.process.internal.ExecException: Process 'command 'sh'' finished with non-zero exit value 1 at org.gradle.process.internal.DefaultExecHandle$ExecResultImpl.assertNormalExitValue(DefaultExecHandle.java:389) at org.gradle.process.internal.DefaultExecAction.execute(DefaultExecAction.java:36) at org.gradle.api.internal.file.DefaultFileOperations.exec(DefaultFileOperations.java:192) at org.gradle.api.internal.project.DefaultProject.exec(DefaultProject.java:1087) at org.gradle.groovy.scripts.DefaultScript.exec(DefaultScript.java:253) at org.gradle.internal.metaobject.BeanDynamicObject$MetaClassAdapter.invokeMethod(BeanDynamicObject.java:479) at org.gradle.internal.metaobject.BeanDynamicObject.tryInvokeMethod(BeanDynamicObject.java:191) at org.gradle.groovy.scripts.BasicScript$ScriptDynamicObject.tryInvokeMethod(BasicScript.java:130) at org.gradle.internal.metaobject.ConfigureDelegate.invokeMethod(ConfigureDelegate.java:78) at build_n921ab6m8g3ths8gqnluth85$_run_closure2$_closure16.doCall(<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/runs/dd1c6c58/beam/sdks/python/build.gradle>:36) at org.gradle.api.internal.AbstractTask$ClosureTaskAction.execute(AbstractTask.java:732) at org.gradle.api.internal.AbstractTask$ClosureTaskAction.execute(AbstractTask.java:705) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:124) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:317) at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:309) at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:185) at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:97) at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:113) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:95) ... 31 more * Get more help at https://help.gradle.org 2018-05-09 06:01:13,443 dd1c6c58 MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark 2018-05-09 06:01:13,450 dd1c6c58 MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/runs/dd1c6c58/beam/gradlew> integrationTest --tests=org.apache.beam.sdk.io.avro.AvroIOIT -p sdks/java/io/file-based-io-tests -DintegrationTestRunner=dataflow -Dfilesystem=gcs -DintegrationTestPipelineOptions=["--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--filenamePrefix=gs://temp-storage-for-perf-tests/beam_PerformanceTests_AvroIOIT/475/","--numberOfRecords=1000000","--runner=TestDataflowRunner"] --stacktrace --info 2018-05-09 06:07:39,139 dd1c6c58 MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark 2018-05-09 06:07:39,139 dd1c6c58 MainThread beam_integration_benchmark(1/1) INFO Tearing down resources for benchmark beam_integration_benchmark 2018-05-09 06:07:39,173 dd1c6c58 MainThread INFO Publishing 2 samples to <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/runs/dd1c6c58/perfkit-bq-pubmUo6uL.json> 2018-05-09 06:07:39,174 dd1c6c58 MainThread INFO Publishing 2 samples to beam_performance.avroioit_pkb_results 2018-05-09 06:07:39,174 dd1c6c58 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.avroioit_pkb_results <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/runs/dd1c6c58/perfkit-bq-pubmUo6uL.json> 2018-05-09 06:07:43,876 dd1c6c58 MainThread INFO -------------------------PerfKitBenchmarker Complete Results------------------------- {'metadata': {'dpb_cluster_id': None, 'dpb_cluster_shape': 'n1-standard-1', 'dpb_cluster_size': 2, 'dpb_dataflow_runner': 'DataflowRunner', 'dpb_dataflow_sdk': None, 'dpb_service': 'dataflow', 'perfkitbenchmarker_version': 'v1.12.0-580-g36cfc82', 'run_number': 0}, 'metric': 'run_time', 'official': True, 'owner': 'jenkins', 'product_name': 'PerfKitBenchmarker', 'run_uri': 'dd1c6c58-7111b772-b1cd-4e20-8dbc-48c788bd59d9', 'sample_uri': '440fe7ae-f9ff-4108-bbbc-cdc02e98842b', 'test': 'beam_integration_benchmark', 'timestamp': 1525846059.138243, 'unit': 'seconds', 'value': 385.694287} {'metadata': {'perfkitbenchmarker_version': 'v1.12.0-580-g36cfc82'}, 'metric': 'End to End Runtime', 'official': True, 'owner': 'jenkins', 'product_name': 'PerfKitBenchmarker', 'run_uri': 'dd1c6c58-7111b772-b1cd-4e20-8dbc-48c788bd59d9', 'sample_uri': '9b017f43-506f-4c4b-8788-7991f7713c27', 'test': 'beam_integration_benchmark', 'timestamp': 1525846059.139666, 'unit': 'seconds', 'value': 426.1720519065857} -------------------------PerfKitBenchmarker Results Summary------------------------- BEAM_INTEGRATION_BENCHMARK: dpb_cluster_shape="n1-standard-1" dpb_cluster_size="2" dpb_dataflow_runner="DataflowRunner" dpb_service="dataflow" run_number="0" run_time 385.694287 seconds End to End Runtime 426.172052 seconds ------------------------- For all tests: dpb_cluster_id="None" dpb_dataflow_sdk="None" perfkitbenchmarker_version="v1.12.0-580-g36cfc82" 2018-05-09 06:07:43,876 dd1c6c58 MainThread INFO Publishing 2 samples to <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/runs/dd1c6c58/perfkitbenchmarker_results.json> 2018-05-09 06:07:43,877 dd1c6c58 MainThread INFO Benchmark run statuses: ------------------------------------------------------------------------------------ Name UID Status Failed Substatus ------------------------------------------------------------------------------------ beam_integration_benchmark beam_integration_benchmark0 SUCCEEDED ------------------------------------------------------------------------------------ Success rate: 100.00% (1/1) 2018-05-09 06:07:43,877 dd1c6c58 MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/runs/dd1c6c58/pkb.log> 2018-05-09 06:07:43,877 dd1c6c58 MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT/ws/runs/dd1c6c58/completion_statuses.json> [Set GitHub commit status (universal)] SUCCESS on repos [GHRepository@48091139[description=Apache Beam,homepage=,name=beam,fork=false,size=58754,milestones={},language=Java,commits={},source=<null>,parent=<null>,responseHeaderFields={null=[HTTP/1.1 200 OK], Access-Control-Allow-Origin=[*], Access-Control-Expose-Headers=[ETag, Link, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval], Cache-Control=[private, max-age=60, s-maxage=60], Content-Encoding=[gzip], Content-Security-Policy=[default-src 'none'], Content-Type=[application/json; charset=utf-8], Date=[Wed, 09 May 2018 06:07:44 GMT], ETag=[W/"b325fcee1bcdcb6673ffb3a642ac3060"], Last-Modified=[Wed, 09 May 2018 00:26:13 GMT], OkHttp-Received-Millis=[1525846064605], OkHttp-Response-Source=[NETWORK 200], OkHttp-Selected-Protocol=[http/1.1], OkHttp-Sent-Millis=[1525846064465], Referrer-Policy=[origin-when-cross-origin, strict-origin-when-cross-origin], Server=[GitHub.com], Status=[200 OK], Strict-Transport-Security=[max-age=31536000; includeSubdomains; preload], Transfer-Encoding=[chunked], Vary=[Accept, Authorization, Cookie, X-GitHub-OTP], X-Accepted-OAuth-Scopes=[repo], X-Content-Type-Options=[nosniff], X-Frame-Options=[deny], X-GitHub-Media-Type=[github.v3; format=json], X-GitHub-Request-Id=[DE20:4F40:98BF34:14FA3B9:5AF2900A], X-OAuth-Scopes=[admin:repo_hook, repo, repo:status], X-RateLimit-Limit=[5000], X-RateLimit-Remaining=[4961], X-RateLimit-Reset=[1525848688], X-Runtime-rack=[0.058224], X-XSS-Protection=[1; mode=block]},url=https://api.github.com/repos/apache/beam,id=50904245]] (sha:60f90c8) with context:beam_PerformanceTests_AvroIOIT Setting commit status on GitHub for https://github.com/apache/beam/commit/60f90c8dcb229c35a82c7be15e64a89678bae058 ERROR: Build step failed with exception java.io.FileNotFoundException: https://api.github.com/repos/apache/beam/statuses/60f90c8dcb229c35a82c7be15e64a89678bae058 at com.squareup.okhttp.internal.huc.HttpURLConnectionImpl.getInputStream(HttpURLConnectionImpl.java:243) at com.squareup.okhttp.internal.huc.DelegatingHttpsURLConnection.getInputStream(DelegatingHttpsURLConnection.java:210) at com.squareup.okhttp.internal.huc.HttpsURLConnectionImpl.getInputStream(HttpsURLConnectionImpl.java:25) at org.kohsuke.github.Requester.parse(Requester.java:612) at org.kohsuke.github.Requester.parse(Requester.java:594) at org.kohsuke.github.Requester._to(Requester.java:272) Caused: org.kohsuke.github.GHFileNotFoundException: {"message":"Not Found","documentation_url":"https://developer.github.com/v3/repos/statuses/#create-a-status"} at org.kohsuke.github.Requester.handleApiError(Requester.java:686) at org.kohsuke.github.Requester._to(Requester.java:293) at org.kohsuke.github.Requester.to(Requester.java:234) at org.kohsuke.github.GHRepository.createCommitStatus(GHRepository.java:1075) at org.jenkinsci.plugins.github.status.GitHubCommitStatusSetter.perform(GitHubCommitStatusSetter.java:160) Caused: org.jenkinsci.plugins.github.common.CombineErrorHandler$ErrorHandlingException at org.jenkinsci.plugins.github.common.CombineErrorHandler.handle(CombineErrorHandler.java:74) at org.jenkinsci.plugins.github.status.GitHubCommitStatusSetter.perform(GitHubCommitStatusSetter.java:164) at com.cloudbees.jenkins.GitHubCommitNotifier.perform(GitHubCommitNotifier.java:151) at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20) at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744) at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:690) at hudson.model.Build$BuildExecution.post2(Build.java:186) at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:635) at hudson.model.Run.execute(Run.java:1749) at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43) at hudson.model.ResourceController.execute(ResourceController.java:97) at hudson.model.Executor.run(Executor.java:429) Build step 'Set build status on GitHub commit [deprecated]' marked build as failure
