See <https://ci-beam.apache.org/job/beam_python_mongoio_load_test/2373/display/redirect?page=changes>
Changes: [dpires] [BEAM-12715] Use shard number specified by user in SnowflakeIO batch [relax] fix GroupIntoBatches [noreply] [BEAM-12703] Fix universal metrics. (#15260) [noreply] [BEAM-12702] Pull step unique names from pipeline for metrics. (#15261) [noreply] [BEAM-12678] Add dependency of java jars when running go VR on portable [noreply] [BEAM-12671] Mark known composite transforms native (#15236) ------------------------------------------ [...truncated 32.37 KB...] Using cached hdfs-2.6.0-py3-none-any.whl (33 kB) Collecting httplib2<0.20.0,>=0.8 Using cached httplib2-0.19.1-py3-none-any.whl (95 kB) Collecting numpy<1.21.0,>=1.14.3 Using cached numpy-1.19.5-cp36-cp36m-manylinux2010_x86_64.whl (14.8 MB) Collecting pymongo<4.0.0,>=3.8.0 Using cached pymongo-3.12.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (523 kB) Collecting oauth2client<5,>=2.0.1 Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB) Requirement already satisfied: protobuf<4,>=3.12.2 in <https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages> (from apache-beam==2.33.0.dev0) (3.17.3) Collecting pyarrow<5.0.0,>=0.15.1 Using cached pyarrow-4.0.1-cp36-cp36m-manylinux2014_x86_64.whl (21.9 MB) Collecting pydot<2,>=1.2.0 Using cached pydot-1.4.2-py2.py3-none-any.whl (21 kB) Collecting python-dateutil<3,>=2.8.0 Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB) Collecting pytz>=2018.3 Using cached pytz-2021.1-py2.py3-none-any.whl (510 kB) Collecting requests<3.0.0,>=2.24.0 Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB) Collecting typing-extensions<3.8.0,>=3.7.0 Using cached typing_extensions-3.7.4.3-py3-none-any.whl (22 kB) Collecting dataclasses Using cached dataclasses-0.8-py3-none-any.whl (19 kB) Collecting orjson<4.0 Using cached orjson-3.6.1.tar.gz (746 kB) Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing wheel metadata: started Preparing wheel metadata: finished with status 'error' ERROR: Command errored out with exit status 1: command: <https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/build/gradleenv/-1734967053/bin/python3.6> <https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/pip/_vendor/pep517/in_process/_in_process.py> prepare_metadata_for_build_wheel /tmp/tmpqkb1ca3k cwd: /tmp/pip-install-50pht8d7/orjson_9202cf309e3c423f8d4462ea42599560 Complete output (6 lines): Cargo, the Rust package manager, is not installed or is not on PATH. This package requires Rust and Cargo to compile extensions. Install it through the system's package manager or via https://rustup.rs/ Checking for Rust toolchain.... ---------------------------------------- WARNING: Discarding https://files.pythonhosted.org/packages/92/97/895dfe0c2e7820fd5453d0efab6a9036de7f97b4edfd157a6a414dd3b0ee/orjson-3.6.1.tar.gz#sha256=5ee598ce6e943afeb84d5706dc604bf90f74e67dc972af12d08af22249bd62d6 (from https://pypi.org/simple/orjson/) (requires-python:>=3.6). Command errored out with exit status 1: <https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/build/gradleenv/-1734967053/bin/python3.6> <https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/pip/_vendor/pep517/in_process/_in_process.py> prepare_metadata_for_build_wheel /tmp/tmpqkb1ca3k Check the logs for full command output. Using cached orjson-3.6.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (236 kB) Collecting cachetools<5,>=3.1.0 Using cached cachetools-4.2.2-py3-none-any.whl (11 kB) Collecting google-apitools<0.5.32,>=0.5.31 Using cached google_apitools-0.5.31-py3-none-any.whl Collecting google-auth<2,>=1.18.0 Using cached google_auth-1.34.0-py2.py3-none-any.whl (152 kB) Collecting google-cloud-datastore<2,>=1.8.0 Using cached google_cloud_datastore-1.15.3-py2.py3-none-any.whl (134 kB) Collecting google-cloud-pubsub<2,>=0.39.0 Using cached google_cloud_pubsub-1.7.0-py2.py3-none-any.whl (144 kB) Collecting google-cloud-bigquery<3,>=1.6.0 Using cached google_cloud_bigquery-2.23.2-py2.py3-none-any.whl (196 kB) Collecting google-cloud-core<2,>=0.28.1 Using cached google_cloud_core-1.7.2-py2.py3-none-any.whl (28 kB) Collecting google-cloud-bigtable<2,>=0.31.1 Using cached google_cloud_bigtable-1.7.0-py2.py3-none-any.whl (267 kB) Collecting google-cloud-spanner<2,>=1.13.0 Using cached google_cloud_spanner-1.19.1-py2.py3-none-any.whl (255 kB) Collecting grpcio-gcp<1,>=0.2.2 Using cached grpcio_gcp-0.2.2-py2.py3-none-any.whl (9.4 kB) Collecting google-cloud-dlp<2,>=0.12.0 Using cached google_cloud_dlp-1.0.0-py2.py3-none-any.whl (169 kB) Collecting google-cloud-language<2,>=1.3.0 Using cached google_cloud_language-1.3.0-py2.py3-none-any.whl (83 kB) Collecting google-cloud-videointelligence<2,>=1.8.0 Using cached google_cloud_videointelligence-1.16.1-py2.py3-none-any.whl (183 kB) Collecting google-cloud-vision<2,>=0.38.0 Using cached google_cloud_vision-1.0.0-py2.py3-none-any.whl (435 kB) Collecting google-cloud-recommendations-ai<=0.2.0,>=0.1.0 Using cached google_cloud_recommendations_ai-0.2.0-py2.py3-none-any.whl (180 kB) Collecting freezegun>=0.3.12 Using cached freezegun-1.1.0-py2.py3-none-any.whl (16 kB) Collecting mock<3.0.0,>=1.0.1 Using cached mock-2.0.0-py2.py3-none-any.whl (56 kB) Collecting nose>=1.3.7 Using cached nose-1.3.7-py3-none-any.whl (154 kB) Collecting nose_xunitmp>=0.4.1 Using cached nose_xunitmp-0.4.1-py3-none-any.whl Collecting pandas<1.3.0,>=1.0 Using cached pandas-1.1.5-cp36-cp36m-manylinux1_x86_64.whl (9.5 MB) Collecting parameterized<0.8.0,>=0.7.1 Using cached parameterized-0.7.5-py2.py3-none-any.whl (17 kB) Collecting pyhamcrest!=1.10.0,<2.0.0,>=1.9 Using cached PyHamcrest-1.10.1-py3-none-any.whl (48 kB) Collecting pyyaml<6.0.0,>=3.12 Using cached PyYAML-5.4.1-cp36-cp36m-manylinux1_x86_64.whl (640 kB) Collecting requests_mock<2.0,>=1.7 Using cached requests_mock-1.9.3-py2.py3-none-any.whl (27 kB) Collecting tenacity<6.0,>=5.0.2 Using cached tenacity-5.1.5-py2.py3-none-any.whl (34 kB) Collecting pytest<5.0,>=4.4.0 Using cached pytest-4.6.11-py2.py3-none-any.whl (231 kB) Collecting pytest-xdist<2,>=1.29.0 Using cached pytest_xdist-1.34.0-py2.py3-none-any.whl (36 kB) Collecting pytest-timeout<2,>=1.3.3 Using cached pytest_timeout-1.4.2-py2.py3-none-any.whl (10 kB) Collecting sqlalchemy<2.0,>=1.3 Using cached SQLAlchemy-1.4.22-cp36-cp36m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.6 MB) Collecting psycopg2-binary<3.0.0,>=2.8.5 Using cached psycopg2_binary-2.9.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.4 MB) Collecting testcontainers<4.0.0,>=3.0.3 Using cached testcontainers-3.4.1-py2.py3-none-any.whl (31 kB) Collecting azure-storage-blob>=12.3.2 Using cached azure_storage_blob-12.8.1-py2.py3-none-any.whl (345 kB) Collecting azure-core>=1.7.0 Using cached azure_core-1.16.0-py2.py3-none-any.whl (163 kB) Collecting boto3>=1.9 Using cached boto3-1.18.13-py3-none-any.whl (131 kB) Requirement already satisfied: six>=1.11.0 in <https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages> (from azure-core>=1.7.0->apache-beam==2.33.0.dev0) (1.16.0) Collecting cryptography>=2.1.4 Using cached cryptography-3.4.7-cp36-abi3-manylinux2014_x86_64.whl (3.2 MB) Collecting msrest>=0.6.18 Using cached msrest-0.6.21-py2.py3-none-any.whl (85 kB) Collecting s3transfer<0.6.0,>=0.5.0 Using cached s3transfer-0.5.0-py3-none-any.whl (79 kB) Collecting jmespath<1.0.0,>=0.7.1 Using cached jmespath-0.10.0-py2.py3-none-any.whl (24 kB) Collecting botocore<1.22.0,>=1.21.13 Using cached botocore-1.21.13-py3-none-any.whl (7.8 MB) Collecting urllib3<1.27,>=1.25.4 Using cached urllib3-1.26.6-py2.py3-none-any.whl (138 kB) Collecting cffi>=1.12 Using cached cffi-1.14.6-cp36-cp36m-manylinux1_x86_64.whl (401 kB) Collecting pycparser Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB) Collecting fasteners>=0.14 Using cached fasteners-0.16.3-py2.py3-none-any.whl (28 kB) Collecting rsa<5,>=3.1.4 Using cached rsa-4.7.2-py3-none-any.whl (34 kB) Requirement already satisfied: setuptools>=40.3.0 in <https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages> (from google-auth<2,>=1.18.0->apache-beam==2.33.0.dev0) (57.4.0) Collecting pyasn1-modules>=0.2.1 Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB) Requirement already satisfied: packaging>=14.3 in <https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages> (from google-cloud-bigquery<3,>=1.6.0->apache-beam==2.33.0.dev0) (21.0) Collecting google-resumable-media<3.0dev,>=0.6.0 Using cached google_resumable_media-1.3.3-py2.py3-none-any.whl (75 kB) Collecting google-api-core[grpc]<3.0.0dev,>=1.29.0 Using cached google_api_core-1.31.1-py2.py3-none-any.whl (93 kB) Collecting proto-plus>=1.10.0 Using cached proto_plus-1.19.0-py3-none-any.whl (42 kB) Collecting googleapis-common-protos<2.0dev,>=1.6.0 Using cached googleapis_common_protos-1.53.0-py2.py3-none-any.whl (198 kB) Collecting grpc-google-iam-v1<0.13dev,>=0.12.3 Using cached grpc_google_iam_v1-0.12.3-py3-none-any.whl Collecting google-crc32c<2.0dev,>=1.0 Using cached google_crc32c-1.1.2-cp36-cp36m-manylinux2014_x86_64.whl (38 kB) Collecting docopt Using cached docopt-0.6.2-py2.py3-none-any.whl Requirement already satisfied: pyparsing<3,>=2.4.2 in <https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages> (from httplib2<0.20.0,>=0.8->apache-beam==2.33.0.dev0) (2.4.7) Collecting pbr>=0.11 Using cached pbr-5.6.0-py2.py3-none-any.whl (111 kB) Collecting isodate>=0.6.0 Using cached isodate-0.6.0-py2.py3-none-any.whl (45 kB) Collecting requests-oauthlib>=0.5.0 Using cached requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB) Collecting certifi>=2017.4.17 Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB) Collecting pyasn1>=0.1.7 Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB) Collecting atomicwrites>=1.0 Using cached atomicwrites-1.4.0-py2.py3-none-any.whl (6.8 kB) Requirement already satisfied: importlib-metadata>=0.12 in <https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.33.0.dev0) (2.1.1) Collecting more-itertools>=4.0.0 Using cached more_itertools-8.8.0-py3-none-any.whl (48 kB) Requirement already satisfied: pluggy<1.0,>=0.12 in <https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.33.0.dev0) (0.13.1) Requirement already satisfied: py>=1.5.0 in <https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages> (from pytest<5.0,>=4.4.0->apache-beam==2.33.0.dev0) (1.10.0) Collecting attrs>=17.4.0 Using cached attrs-21.2.0-py2.py3-none-any.whl (53 kB) Collecting wcwidth Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB) Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages> (from importlib-metadata>=0.12->pytest<5.0,>=4.4.0->apache-beam==2.33.0.dev0) (3.5.0) Collecting pytest-forked Using cached pytest_forked-1.3.0-py2.py3-none-any.whl (4.7 kB) Collecting execnet>=1.1 Using cached execnet-1.9.0-py2.py3-none-any.whl (39 kB) Collecting idna<4,>=2.5 Using cached idna-3.2-py3-none-any.whl (59 kB) Collecting charset-normalizer~=2.0.0 Using cached charset_normalizer-2.0.4-py3-none-any.whl (36 kB) Collecting oauthlib>=3.0.0 Using cached oauthlib-3.1.1-py2.py3-none-any.whl (146 kB) Collecting greenlet!=0.4.17 Using cached greenlet-1.1.0-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (155 kB) Collecting deprecation Using cached deprecation-2.1.0-py2.py3-none-any.whl (11 kB) Collecting docker Using cached docker-5.0.0-py2.py3-none-any.whl (146 kB) Collecting wrapt Using cached wrapt-1.12.1-cp36-cp36m-linux_x86_64.whl Collecting websocket-client>=0.32.0 Using cached websocket_client-1.1.0-py2.py3-none-any.whl (68 kB) Building wheels for collected packages: apache-beam Building wheel for apache-beam (setup.py): started Building wheel for apache-beam (setup.py): finished with status 'done' Created wheel for apache-beam: filename=apache_beam-2.33.0.dev0-py3-none-any.whl size=2523549 sha256=56c0fb54d0f4a066bdd7b9d61dfe4b0a1a583f9afaee1414425fd8c540e4fa66 Stored in directory: /home/jenkins/.cache/pip/wheels/84/1e/e9/ee5e7224f16ba01ef34d3345aac43abb3e3f731295e5c52cb7 Successfully built apache-beam Installing collected packages: pyasn1, urllib3, rsa, pycparser, pyasn1-modules, idna, charset-normalizer, certifi, cachetools, wcwidth, requests, pytz, python-dateutil, oauthlib, more-itertools, jmespath, googleapis-common-protos, google-auth, cffi, attrs, atomicwrites, websocket-client, requests-oauthlib, pytest, numpy, isodate, httplib2, grpcio-gcp, google-crc32c, google-api-core, docopt, botocore, wrapt, typing-extensions, s3transfer, pytest-forked, pymongo, pydot, pyarrow, proto-plus, pbr, orjson, oauth2client, nose, msrest, hdfs, grpc-google-iam-v1, greenlet, google-resumable-media, google-cloud-core, fasteners, fastavro, execnet, docker, dill, deprecation, dataclasses, cryptography, crcmod, azure-core, avro-python3, testcontainers, tenacity, sqlalchemy, requests-mock, pyyaml, pytest-xdist, pytest-timeout, pyhamcrest, psycopg2-binary, parameterized, pandas, nose-xunitmp, mock, google-cloud-vision, google-cloud-videointelligence, google-cloud-spanner, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery, google-apitools, freezegun, boto3, azure-storage-blob, apache-beam Successfully installed apache-beam-2.33.0.dev0 atomicwrites-1.4.0 attrs-21.2.0 avro-python3-1.9.2.1 azure-core-1.16.0 azure-storage-blob-12.8.1 boto3-1.18.13 botocore-1.21.13 cachetools-4.2.2 certifi-2021.5.30 cffi-1.14.6 charset-normalizer-2.0.4 crcmod-1.7 cryptography-3.4.7 dataclasses-0.8 deprecation-2.1.0 dill-0.3.1.1 docker-5.0.0 docopt-0.6.2 execnet-1.9.0 fastavro-1.4.4 fasteners-0.16.3 freezegun-1.1.0 google-api-core-1.31.1 google-apitools-0.5.31 google-auth-1.34.0 google-cloud-bigquery-2.23.2 google-cloud-bigtable-1.7.0 google-cloud-core-1.7.2 google-cloud-datastore-1.15.3 google-cloud-dlp-1.0.0 google-cloud-language-1.3.0 google-cloud-pubsub-1.7.0 google-cloud-recommendations-ai-0.2.0 google-cloud-spanner-1.19.1 google-cloud-videointelligence-1.16.1 google-cloud-vision-1.0.0 google-crc32c-1.1.2 google-resumable-media-1.3.3 googleapis-common-protos-1.53.0 greenlet-1.1.0 grpc-google-iam-v1-0.12.3 grpcio-gcp-0.2.2 hdfs-2.6.0 httplib2-0.19.1 idna-3.2 isodate-0.6.0 jmespath-0.10.0 mock-2.0.0 more-itertools-8.8.0 msrest-0.6.21 nose-1.3.7 nose-xunitmp-0.4.1 numpy-1.19.5 oauth2client-4.1.3 oauthlib-3.1.1 orjson-3.6.0 pandas-1.1.5 parameterized-0.7.5 pbr-5.6.0 proto-plus-1.19.0 psycopg2-binary-2.9.1 pyarrow-4.0.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.20 pydot-1.4.2 pyhamcrest-1.10.1 pymongo-3.12.0 pytest-4.6.11 pytest-forked-1.3.0 pytest-timeout-1.4.2 pytest-xdist-1.34.0 python-dateutil-2.8.2 pytz-2021.1 pyyaml-5.4.1 requests-2.26.0 requests-mock-1.9.3 requests-oauthlib-1.3.0 rsa-4.7.2 s3transfer-0.5.0 sqlalchemy-1.4.22 tenacity-5.1.5 testcontainers-3.4.1 typing-extensions-3.7.4.3 urllib3-1.26.6 wcwidth-0.2.5 websocket-client-1.1.0 wrapt-1.12.1 > Task :sdks:python:test-suites:dataflow:py36:mongodbioIT Traceback (most recent call last): File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/usr/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "<https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/apache_beam/io/mongodbio_it_test.py";,> line 170, in <module> run() File "<https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/apache_beam/io/mongodbio_it_test.py";,> line 69, in run with TestPipeline(options=PipelineOptions(pipeline_args)) as p: File "<https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/apache_beam/testing/test_pipeline.py";,> line 109, in __init__ super(TestPipeline, self).__init__(runner, options) File "<https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/build/gradleenv/-1734967053/lib/python3.6/site-packages/apache_beam/pipeline.py";,> line 200, in __init__ 'Pipeline has validations errors: \n' + '\n'.join(errors)) ValueError: Pipeline has validations errors: Missing required option: region. > Task :sdks:python:test-suites:dataflow:py36:mongodbioIT FAILED FAILURE: Build failed with an exception. * Where: Script '<https://ci-beam.apache.org/job/beam_python_mongoio_load_test/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 238 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py36:mongodbioIT'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 1m 15s 5 actionable tasks: 5 executed Publishing build scan... https://gradle.com/s/bgmssfgybvdcy Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
