See <https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/42/display/redirect>
------------------------------------------ [...truncated 509.95 KB...] copying apache_beam/utils/retry_test.py -> apache-beam-2.3.0/apache_beam/utils copying apache_beam/utils/timestamp.py -> apache-beam-2.3.0/apache_beam/utils copying apache_beam/utils/timestamp_test.py -> apache-beam-2.3.0/apache_beam/utils copying apache_beam/utils/urns.py -> apache-beam-2.3.0/apache_beam/utils copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.3.0/apache_beam/utils copying apache_beam/utils/windowed_value.py -> apache-beam-2.3.0/apache_beam/utils copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.3.0/apache_beam/utils Writing apache-beam-2.3.0/setup.cfg creating dist Creating tar archive removing 'apache-beam-2.3.0' (and everything under it) Processing ./dist/apache-beam-2.3.0.tar.gz Collecting avro<2.0.0,>=1.8.1 (from apache-beam==2.3.0) /tmp/groovy-generated-7662706818481476792-tmpdir/apache-beam-2.3.0/sdks/python/temp_virtualenv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning. SNIMissingWarning /tmp/groovy-generated-7662706818481476792-tmpdir/apache-beam-2.3.0/sdks/python/temp_virtualenv/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning Collecting crcmod<2.0,>=1.7 (from apache-beam==2.3.0) Collecting dill==0.2.6 (from apache-beam==2.3.0) Collecting grpcio<2,>=1.0 (from apache-beam==2.3.0) Using cached grpcio-1.9.1-cp27-cp27mu-manylinux1_x86_64.whl Collecting httplib2<0.10,>=0.8 (from apache-beam==2.3.0) Collecting mock<3.0.0,>=1.0.1 (from apache-beam==2.3.0) Using cached mock-2.0.0-py2.py3-none-any.whl Collecting oauth2client<5,>=2.0.1 (from apache-beam==2.3.0) Using cached oauth2client-4.1.2-py2.py3-none-any.whl Collecting protobuf<4,>=3.5.0.post1 (from apache-beam==2.3.0) Using cached protobuf-3.5.1-cp27-cp27mu-manylinux1_x86_64.whl Collecting pyyaml<4.0.0,>=3.12 (from apache-beam==2.3.0) Collecting pyvcf<0.7.0,>=0.6.8 (from apache-beam==2.3.0) Collecting six<1.12,>=1.9 (from apache-beam==2.3.0) Using cached six-1.11.0-py2.py3-none-any.whl Collecting typing<3.7.0,>=3.6.0 (from apache-beam==2.3.0) Using cached typing-3.6.4-py2-none-any.whl Collecting futures<4.0.0,>=3.1.1 (from apache-beam==2.3.0) Using cached futures-3.2.0-py2-none-any.whl Collecting hdfs3<0.4.0,>=0.3.0 (from apache-beam==2.3.0) Using cached hdfs3-0.3.0-py2.py3-none-any.whl Collecting google-apitools<=0.5.20,>=0.5.18 (from apache-beam==2.3.0) Using cached google_apitools-0.5.20-py2-none-any.whl Collecting proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0 (from apache-beam==2.3.0) Collecting googledatastore==7.0.1 (from apache-beam==2.3.0) Collecting google-cloud-pubsub==0.26.0 (from apache-beam==2.3.0) Using cached google_cloud_pubsub-0.26.0-py2.py3-none-any.whl Collecting google-cloud-bigquery==0.25.0 (from apache-beam==2.3.0) Using cached google_cloud_bigquery-0.25.0-py2.py3-none-any.whl Collecting enum34>=1.0.4 (from grpcio<2,>=1.0->apache-beam==2.3.0) Using cached enum34-1.1.6-py2-none-any.whl Collecting funcsigs>=1; python_version < "3.3" (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0) Using cached funcsigs-1.0.2-py2.py3-none-any.whl Collecting pbr>=0.11 (from mock<3.0.0,>=1.0.1->apache-beam==2.3.0) Using cached pbr-3.1.1-py2.py3-none-any.whl Collecting rsa>=3.1.4 (from oauth2client<5,>=2.0.1->apache-beam==2.3.0) Using cached rsa-3.4.2-py2.py3-none-any.whl Collecting pyasn1-modules>=0.0.5 (from oauth2client<5,>=2.0.1->apache-beam==2.3.0) Using cached pyasn1_modules-0.2.1-py2.py3-none-any.whl Collecting pyasn1>=0.1.7 (from oauth2client<5,>=2.0.1->apache-beam==2.3.0) Using cached pyasn1-0.4.2-py2.py3-none-any.whl Requirement already satisfied: setuptools in ./temp_virtualenv/lib/python2.7/site-packages (from protobuf<4,>=3.5.0.post1->apache-beam==2.3.0) Collecting fasteners>=0.14 (from google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0) Using cached fasteners-0.14.1-py2.py3-none-any.whl Collecting googleapis-common-protos<2.0dev,>=1.5.2 (from proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.3.0) Collecting google-cloud-core<0.26dev,>=0.25.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0) Using cached google_cloud_core-0.25.0-py2.py3-none-any.whl Collecting gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 (from google-cloud-pubsub==0.26.0->apache-beam==2.3.0) Collecting monotonic>=0.1 (from fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.3.0) Using cached monotonic-1.4-py2.py3-none-any.whl Collecting google-auth<2.0.0dev,>=0.4.0 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0) Using cached google_auth-1.4.1-py2.py3-none-any.whl Collecting google-auth-httplib2 (from google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0) Using cached google_auth_httplib2-0.0.3-py2.py3-none-any.whl Collecting google-gax<0.16dev,>=0.15.7 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0) Using cached google_gax-0.15.16-py2.py3-none-any.whl Collecting grpc-google-iam-v1<0.12dev,>=0.11.1 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0) Collecting proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 (from gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0) Collecting cachetools>=2.0.0 (from google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0) Using cached cachetools-2.0.1-py2.py3-none-any.whl Collecting future<0.17dev,>=0.16.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0) Collecting ply==3.8 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0) Collecting requests<3.0dev,>=2.13.0 (from google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0) Using cached requests-2.18.4-py2.py3-none-any.whl Collecting urllib3<1.23,>=1.21.1 (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0) Using cached urllib3-1.22-py2.py3-none-any.whl Collecting idna<2.7,>=2.5 (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0) Using cached idna-2.6-py2.py3-none-any.whl Collecting chardet<3.1.0,>=3.0.2 (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0) Using cached chardet-3.0.4-py2.py3-none-any.whl Collecting certifi>=2017.4.17 (from requests<3.0dev,>=2.13.0->google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.3.0) Using cached certifi-2018.1.18-py2.py3-none-any.whl Building wheels for collected packages: apache-beam Running setup.py bdist_wheel for apache-beam: started Running setup.py bdist_wheel for apache-beam: finished with status 'done' Stored in directory: /home/jenkins/.cache/pip/wheels/20/3b/4d/43d3bcfd7cf9a35f7a9acce994b3b5aa9a3d86d615291ae7f4 Successfully built apache-beam Installing collected packages: avro, crcmod, dill, futures, six, protobuf, enum34, grpcio, httplib2, funcsigs, pbr, mock, pyasn1, rsa, pyasn1-modules, oauth2client, pyyaml, pyvcf, typing, hdfs3, monotonic, fasteners, google-apitools, googleapis-common-protos, proto-google-cloud-datastore-v1, googledatastore, cachetools, google-auth, google-auth-httplib2, google-cloud-core, future, ply, urllib3, idna, chardet, certifi, requests, google-gax, grpc-google-iam-v1, proto-google-cloud-pubsub-v1, gapic-google-cloud-pubsub-v1, google-cloud-pubsub, google-cloud-bigquery, apache-beam Successfully installed apache-beam-2.3.0 avro-1.8.2 cachetools-2.0.1 certifi-2018.1.18 chardet-3.0.4 crcmod-1.7 dill-0.2.6 enum34-1.1.6 fasteners-0.14.1 funcsigs-1.0.2 future-0.16.0 futures-3.2.0 gapic-google-cloud-pubsub-v1-0.15.4 google-apitools-0.5.20 google-auth-1.4.1 google-auth-httplib2-0.0.3 google-cloud-bigquery-0.25.0 google-cloud-core-0.25.0 google-cloud-pubsub-0.26.0 google-gax-0.15.16 googleapis-common-protos-1.5.3 googledatastore-7.0.1 grpc-google-iam-v1-0.11.4 grpcio-1.9.1 hdfs3-0.3.0 httplib2-0.9.2 idna-2.6 mock-2.0.0 monotonic-1.4 oauth2client-4.1.2 pbr-3.1.1 ply-3.8 proto-google-cloud-datastore-v1-0.90.4 proto-google-cloud-pubsub-v1-0.15.4 protobuf-3.5.1 pyasn1-0.4.2 pyasn1-modules-0.2.1 pyvcf-0.6.8 pyyaml-3.12 requests-2.18.4 rsa-3.4.2 six-1.11.0 typing-3.6.4 urllib3-1.22 ---------------------------------------------------------------- Running wordcount example with DirectRunner and verify results ---------------------------------------------------------------- python -m apache_beam.examples.wordcount --output wordcount_direct.txt INFO:root:Missing pipeline option (runner). Executing pipeline using the default runner: DirectRunner. apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters super(GcsIO, cls).__new__(cls, storage_client)) INFO:root:Starting the size estimation of the input INFO:oauth2client.transport:Attempting refresh to obtain initial access_token INFO:root:Finished the size estimation of the input at 1 files. Estimation took 0.0828478336334 seconds apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>. warnings.warn('Using fallback coder for typehint: %r.' % typehint) apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any. warnings.warn('Using fallback coder for typehint: %r.' % typehint) INFO:root:Running pipeline with DirectRunner. INFO:root:Starting the size estimation of the input INFO:oauth2client.transport:Attempting refresh to obtain initial access_token INFO:root:Finished the size estimation of the input at 1 files. Estimation took 0.0766088962555 seconds INFO:oauth2client.transport:Attempting refresh to obtain initial access_token INFO:root:Starting finalize_write threads with num_shards: 1, batches: 1, num_threads: 1 INFO:root:Renamed 1 shards in 0.12 seconds. INFO:root:number of empty lines: 1663 INFO:root:average word length: 4 ls apache_beam apache_beam.egg-info build.gradle container dist findSupportedPython.groovy generate_pydoc.sh gen_protos.py gen_protos.pyc MANIFEST.in pom.xml run_postcommit.sh run_pylint.sh run_validatesrunner.sh setup.cfg setup.py temp_virtualenv test_config.py tox.ini wordcount_direct.txt-00000-of-00001 Verified wordcount_direct.txt-00000-of-00001 ---------------------------------------------------------------- Running wordcount example with DataflowRunner with command: python -m apache_beam.examples.wordcount --output gs://yifan_auto_verification_test_bucket/wordcount_direct.txt --staging_location gs://yifan_auto_verification_test_bucket/temp --temp_location gs://yifan_auto_verification_test_bucket/temp --runner DataflowRunner --job_name wordcount --project my-first-project-190318 --num_workers 1 --sdk_location dist/apache-beam-2.3.0.tar.gz ---------------------------------------------------------------- python -m apache_beam.examples.wordcount --output gs://yifan_auto_verification_test_bucket/wordcount_direct.txt --staging_location gs://yifan_auto_verification_test_bucket/temp --temp_location gs://yifan_auto_verification_test_bucket/temp --runner DataflowRunner --job_name wordcount --project my-first-project-190318 --num_workers 1 --sdk_location dist/apache-beam-2.3.0.tar.gz apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters super(GcsIO, cls).__new__(cls, storage_client)) INFO:root:Starting the size estimation of the input INFO:oauth2client.transport:Attempting refresh to obtain initial access_token INFO:root:Finished the size estimation of the input at 1 files. Estimation took 0.0796480178833 seconds apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>. warnings.warn('Using fallback coder for typehint: %r.' % typehint) apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any. warnings.warn('Using fallback coder for typehint: %r.' % typehint) INFO:root:Starting the size estimation of the input INFO:oauth2client.transport:Attempting refresh to obtain initial access_token INFO:root:Finished the size estimation of the input at 1 files. Estimation took 0.0779528617859 seconds INFO:root:Starting GCS upload to gs://yifan_auto_verification_test_bucket/temp/wordcount.1518480439.014223/pipeline.pb... INFO:oauth2client.transport:Attempting refresh to obtain initial access_token Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/tmp/groovy-generated-7662706818481476792-tmpdir/apache-beam-2.3.0/sdks/python/apache_beam/examples/wordcount.py", line 134, in <module> run() File "/tmp/groovy-generated-7662706818481476792-tmpdir/apache-beam-2.3.0/sdks/python/apache_beam/examples/wordcount.py", line 113, in run result = p.run() File "apache_beam/pipeline.py", line 333, in run self.to_runner_api(), self.runner, self._options).run(False) File "apache_beam/pipeline.py", line 342, in run return self.runner.run_pipeline(self) File "apache_beam/runners/dataflow/dataflow_runner.py", line 315, in run_pipeline self.dataflow_client.create_job(self.job), self) File "apache_beam/utils/retry.py", line 175, in wrapper return fun(*args, **kwargs) File "apache_beam/runners/dataflow/internal/apiclient.py", line 461, in create_job self.create_job_description(job) File "apache_beam/runners/dataflow/internal/apiclient.py", line 487, in create_job_description StringIO(job.proto_pipeline.SerializeToString())) File "apache_beam/runners/dataflow/internal/apiclient.py", line 453, in stage_file (gcs_or_local_path, reportable_errors[e.status_code])) IOError: Could not upload to GCS path gs://yifan_auto_verification_test_bucket/temp/wordcount.1518480439.014223: access denied. Please verify that credentials are valid and that you have write access to the specified path. apache_beam/io/gcp/gcsio.py:160: DeprecationWarning: object() takes no parameters super(GcsIO, cls).__new__(cls, storage_client)) INFO:root:Starting the size estimation of the input INFO:oauth2client.transport:Attempting refresh to obtain initial access_token INFO:root:Finished the size estimation of the input at 1 files. Estimation took 0.0796480178833 seconds apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: <type 'NoneType'>. warnings.warn('Using fallback coder for typehint: %r.' % typehint) apache_beam/coders/typecoders.py:133: UserWarning: Using fallback coder for typehint: Any. warnings.warn('Using fallback coder for typehint: %r.' % typehint) INFO:root:Starting the size estimation of the input INFO:oauth2client.transport:Attempting refresh to obtain initial access_token INFO:root:Finished the size estimation of the input at 1 files. Estimation took 0.0779528617859 seconds INFO:root:Starting GCS upload to gs://yifan_auto_verification_test_bucket/temp/wordcount.1518480439.014223/pipeline.pb... INFO:oauth2client.transport:Attempting refresh to obtain initial access_token Traceback (most recent call last): File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/tmp/groovy-generated-7662706818481476792-tmpdir/apache-beam-2.3.0/sdks/python/apache_beam/examples/wordcount.py", line 134, in <module> run() File "/tmp/groovy-generated-7662706818481476792-tmpdir/apache-beam-2.3.0/sdks/python/apache_beam/examples/wordcount.py", line 113, in run result = p.run() File "apache_beam/pipeline.py", line 333, in run self.to_runner_api(), self.runner, self._options).run(False) File "apache_beam/pipeline.py", line 342, in run return self.runner.run_pipeline(self) File "apache_beam/runners/dataflow/dataflow_runner.py", line 315, in run_pipeline self.dataflow_client.create_job(self.job), self) File "apache_beam/utils/retry.py", line 175, in wrapper return fun(*args, **kwargs) File "apache_beam/runners/dataflow/internal/apiclient.py", line 461, in create_job self.create_job_description(job) File "apache_beam/runners/dataflow/internal/apiclient.py", line 487, in create_job_description StringIO(job.proto_pipeline.SerializeToString())) File "apache_beam/runners/dataflow/internal/apiclient.py", line 453, in stage_file (gcs_or_local_path, reportable_errors[e.status_code])) IOError: Could not upload to GCS path gs://yifan_auto_verification_test_bucket/temp/wordcount.1518480439.014223: access denied. Please verify that credentials are valid and that you have write access to the specified path. [ERROR] Failed command: :release:runQuickstartsPython FAILED FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':release:runQuickstartsPython'. > Process 'command '/usr/local/asfpackages/java/jdk1.8.0_152/bin/java'' > finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org BUILD FAILED in 1m 32s 2 actionable tasks: 2 executed Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure Not sending mail to unregistered user c...@google.com Not sending mail to unregistered user daniel.o.program...@gmail.com Not sending mail to unregistered user j...@nanthrax.net Not sending mail to unregistered user apill...@google.com Not sending mail to unregistered user fjetum...@gmail.com Not sending mail to unregistered user git...@alasdairhodge.co.uk
