See <https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Dataflow_Streaming/728/display/redirect?page=changes>
Changes: [noreply] [Playground] [Backend] Datastore queries and mappers to get precompiled [noreply] Open Allow and test pyarrow 8.x and 9.x (#22997) [noreply] (BQ Python) Pass project field from options or parameter when writing [noreply] Update python-machine-learning.md (#23209) [noreply] Pin the version of cloudpickle to 2.1.x (#23120) [noreply] Add streaming test for Write API sink (#21903) [noreply] [Go SDK] Proto changes for timer param (#23216) [noreply] Bump github.com/testcontainers/testcontainers-go in /sdks (#23201) [noreply] Update to objsize to 0.5.2 which is under BSD-3 license (fixes #23096) [noreply] Exclude insignificant whitespace from cloud object (#23217) [noreply] Trying out property-based tests for Beam python coders (#22233) [noreply] Publish results of JMH benchmark runs (Java SDK) to InfluxDB (part of ------------------------------------------ [...truncated 527 B...] No credentials specified Wiping out workspace first. Cloning the remote Git repository Cloning repository https://github.com/apache/beam.git > git init > <https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Dataflow_Streaming/ws/src> > # timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git --version # timeout=10 > git --version # 'git version 2.25.1' > git fetch --tags --force --progress -- https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # > timeout=10 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git fetch --tags --force --progress -- https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* # > timeout=10 > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 9d9db56ca63b0f2dc5969d519000eb75dca223b3 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 9d9db56ca63b0f2dc5969d519000eb75dca223b3 # timeout=10 Commit message: "Publish results of JMH benchmark runs (Java SDK) to InfluxDB (part of #22238). (#23041)" > git rev-list --no-walk ebacef90771d3fbc83f2c17e517e3f2195f746d0 # timeout=10 No emails were triggered. [EnvInject] - Executing scripts and injecting environment variables after the SCM step. [EnvInject] - Injecting as environment variables the properties content SETUPTOOLS_USE_DISTUTILS=stdlib SPARK_LOCAL_IP=127.0.0.1 [EnvInject] - Variables injected successfully. [beam_LoadTests_Python_ParDo_Dataflow_Streaming] $ /bin/bash -xe /tmp/jenkins9013182454559176771.sh + echo '*** ParDo Python Load test: 2GB 100 byte records 10 times ***' *** ParDo Python Load test: 2GB 100 byte records 10 times *** [Gradle] - Launching build. [src] $ <https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Dataflow_Streaming/ws/src/gradlew> -PloadTest.mainClass=apache_beam.testing.load_tests.pardo_test -Prunner=DataflowRunner '-PloadTest.args=--job_name=load-tests-python-dataflow-streaming-pardo-1-0914125515 --project=apache-beam-testing --region=us-central1 --temp_location=gs://temp-storage-for-perf-tests/loadtests --publish_to_big_query=true --metrics_dataset=load_test --metrics_table=python_dataflow_streaming_pardo_1 --influx_measurement=python_streaming_pardo_1 --input_options='{"num_records": 20000000,"key_size": 10,"value_size": 90}' --iterations=10 --number_of_counter_operations=0 --number_of_counters=0 --num_****s=5 --autoscaling_algorithm=NONE --influx_db_name=beam_test_metrics --influx_hostname=http://10.128.0.96:8086 --streaming --experiments=use_runner_v2,shuffle_mode=appliance --runner=DataflowRunner' -PpythonVersion=3.7 --continue --max-****s=8 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx6g -Dorg.gradle.vfs.watch=false -Pdocker-pull-licenses :sdks:python:apache_beam:testing:load_tests:run Starting a Gradle Daemon, 2 busy and 1 incompatible Daemons could not be reused, use --status for details Configuration on demand is an incubating feature. > Task :buildSrc:compileJava NO-SOURCE > Task :buildSrc:compileGroovy FROM-CACHE > Task :buildSrc:pluginDescriptors > Task :buildSrc:processResources > Task :buildSrc:classes > Task :buildSrc:jar > Task :buildSrc:assemble > Task :buildSrc:spotlessGroovy FROM-CACHE > Task :buildSrc:spotlessGroovyCheck UP-TO-DATE > Task :buildSrc:spotlessGroovyGradle FROM-CACHE > Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE > Task :buildSrc:spotlessCheck UP-TO-DATE > Task :buildSrc:pluginUnderTestMetadata > Task :buildSrc:compileTestJava NO-SOURCE > Task :buildSrc:compileTestGroovy NO-SOURCE > Task :buildSrc:processTestResources NO-SOURCE > Task :buildSrc:testClasses UP-TO-DATE > Task :buildSrc:test NO-SOURCE > Task :buildSrc:validatePlugins FROM-CACHE > Task :buildSrc:check UP-TO-DATE > Task :buildSrc:build > Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv Collecting pip Using cached pip-22.2.2-py3-none-any.whl (2.0 MB) Installing collected packages: pip Attempting uninstall: pip Found existing installation: pip 20.1.1 Uninstalling pip-20.1.1: Successfully uninstalled pip-20.1.1 Successfully installed pip-22.2.2 Collecting tox==3.20.1 Using cached tox-3.20.1-py2.py3-none-any.whl (83 kB) Requirement already satisfied: setuptools in <https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Dataflow_Streaming/ws/src/build/gradleenv/1329484227/lib/python3.7/site-packages> (from -r <https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Dataflow_Streaming/ws/src/sdks/python/build-requirements.txt> (line 20)) (47.1.0) Collecting setuptools Using cached setuptools-65.3.0-py3-none-any.whl (1.2 MB) Collecting wheel>=0.36.0 Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB) Collecting grpcio-tools==1.37.0 Using cached grpcio_tools-1.37.0-cp37-cp37m-manylinux2014_x86_64.whl (2.5 MB) Collecting mypy-protobuf==1.18 Using cached mypy_protobuf-1.18-py3-none-any.whl (7.3 kB) Collecting distlib==0.3.1 Using cached distlib-0.3.1-py2.py3-none-any.whl (335 kB) Collecting filelock>=3.0.0 Using cached filelock-3.8.0-py3-none-any.whl (10 kB) Collecting six>=1.14.0 Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) Collecting pluggy>=0.12.0 Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB) Collecting py>=1.4.17 Using cached py-1.11.0-py2.py3-none-any.whl (98 kB) Collecting packaging>=14 Using cached packaging-21.3-py3-none-any.whl (40 kB) Collecting virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0 Using cached virtualenv-20.16.5-py3-none-any.whl (8.8 MB) Collecting toml>=0.9.4 Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB) Collecting importlib-metadata<3,>=0.12 Using cached importlib_metadata-2.1.3-py2.py3-none-any.whl (10 kB) Collecting protobuf<4.0dev,>=3.5.0.post1 Using cached protobuf-3.20.2-py2.py3-none-any.whl (162 kB) Collecting grpcio>=1.37.0 Using cached grpcio-1.48.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.6 MB) Collecting zipp>=0.5 Using cached zipp-3.8.1-py3-none-any.whl (5.6 kB) Collecting pyparsing!=3.0.5,>=2.0.2 Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB) Collecting platformdirs<3,>=2.4 Using cached platformdirs-2.5.2-py3-none-any.whl (14 kB) Collecting virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0 Using cached virtualenv-20.16.4-py3-none-any.whl (8.8 MB) Using cached virtualenv-20.16.3-py2.py3-none-any.whl (8.8 MB) Using cached virtualenv-20.16.2-py2.py3-none-any.whl (8.8 MB) Installing collected packages: distlib, zipp, wheel, toml, six, setuptools, pyparsing, py, protobuf, platformdirs, filelock, packaging, mypy-protobuf, importlib-metadata, grpcio, virtualenv, pluggy, grpcio-tools, tox Attempting uninstall: setuptools Found existing installation: setuptools 47.1.0 Uninstalling setuptools-47.1.0: Successfully uninstalled setuptools-47.1.0 Successfully installed distlib-0.3.1 filelock-3.8.0 grpcio-1.48.1 grpcio-tools-1.37.0 importlib-metadata-2.1.3 mypy-protobuf-1.18 packaging-21.3 platformdirs-2.5.2 pluggy-1.0.0 protobuf-3.20.2 py-1.11.0 pyparsing-3.0.9 setuptools-65.3.0 six-1.16.0 toml-0.10.2 tox-3.20.1 virtualenv-20.16.2 wheel-0.37.1 zipp-3.8.1 > Task :sdks:python:setupVirtualenv Collecting pip Using cached pip-22.2.2-py3-none-any.whl (2.0 MB) Installing collected packages: pip Attempting uninstall: pip Found existing installation: pip 20.1.1 Uninstalling pip-20.1.1: Successfully uninstalled pip-20.1.1 Successfully installed pip-22.2.2 Collecting tox==3.20.1 Using cached tox-3.20.1-py2.py3-none-any.whl (83 kB) Requirement already satisfied: setuptools in <https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Dataflow_Streaming/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages> (from -r <https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Dataflow_Streaming/ws/src/sdks/python/build-requirements.txt> (line 20)) (47.1.0) Collecting setuptools Using cached setuptools-65.3.0-py3-none-any.whl (1.2 MB) Collecting wheel>=0.36.0 Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB) Collecting grpcio-tools==1.37.0 Using cached grpcio_tools-1.37.0-cp37-cp37m-manylinux2014_x86_64.whl (2.5 MB) Collecting mypy-protobuf==1.18 Using cached mypy_protobuf-1.18-py3-none-any.whl (7.3 kB) Collecting distlib==0.3.1 Using cached distlib-0.3.1-py2.py3-none-any.whl (335 kB) Collecting virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0 Using cached virtualenv-20.16.5-py3-none-any.whl (8.8 MB) Collecting filelock>=3.0.0 Using cached filelock-3.8.0-py3-none-any.whl (10 kB) Collecting importlib-metadata<3,>=0.12 Using cached importlib_metadata-2.1.3-py2.py3-none-any.whl (10 kB) Collecting py>=1.4.17 Using cached py-1.11.0-py2.py3-none-any.whl (98 kB) Collecting six>=1.14.0 Using cached six-1.16.0-py2.py3-none-any.whl (11 kB) Collecting toml>=0.9.4 Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB) Collecting packaging>=14 Using cached packaging-21.3-py3-none-any.whl (40 kB) Collecting pluggy>=0.12.0 Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB) Collecting grpcio>=1.37.0 Using cached grpcio-1.48.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.6 MB) Collecting protobuf<4.0dev,>=3.5.0.post1 Using cached protobuf-3.20.2-py2.py3-none-any.whl (162 kB) Collecting zipp>=0.5 Using cached zipp-3.8.1-py3-none-any.whl (5.6 kB) Collecting pyparsing!=3.0.5,>=2.0.2 Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB) Collecting virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0 Using cached virtualenv-20.16.4-py3-none-any.whl (8.8 MB) Using cached virtualenv-20.16.3-py2.py3-none-any.whl (8.8 MB) Using cached virtualenv-20.16.2-py2.py3-none-any.whl (8.8 MB) Collecting platformdirs<3,>=2 Using cached platformdirs-2.5.2-py3-none-any.whl (14 kB) Installing collected packages: distlib, zipp, wheel, toml, six, setuptools, pyparsing, py, protobuf, platformdirs, filelock, packaging, mypy-protobuf, importlib-metadata, grpcio, virtualenv, pluggy, grpcio-tools, tox Attempting uninstall: setuptools Found existing installation: setuptools 47.1.0 Uninstalling setuptools-47.1.0: Successfully uninstalled setuptools-47.1.0 Successfully installed distlib-0.3.1 filelock-3.8.0 grpcio-1.48.1 grpcio-tools-1.37.0 importlib-metadata-2.1.3 mypy-protobuf-1.18 packaging-21.3 platformdirs-2.5.2 pluggy-1.0.0 protobuf-3.20.2 py-1.11.0 pyparsing-3.0.9 setuptools-65.3.0 six-1.16.0 toml-0.10.2 tox-3.20.1 virtualenv-20.16.2 wheel-0.37.1 zipp-3.8.1 > Task :sdks:python:sdist org/apache/beam/model/interactive/v1/beam_interactive_api.proto:36:1: warning: Import google/protobuf/timestamp.proto is unused. Writing mypy to org/apache/beam/model/pipeline/v1/external_transforms_pb2.pyi Writing mypy to org/apache/beam/model/pipeline/v1/beam_runner_api_pb2.pyi Writing mypy to org/apache/beam/model/pipeline/v1/standard_window_fns_pb2.pyi Writing mypy to org/apache/beam/model/pipeline/v1/metrics_pb2.pyi Writing mypy to org/apache/beam/model/pipeline/v1/endpoints_pb2.pyi Writing mypy to org/apache/beam/model/pipeline/v1/schema_pb2.pyi Writing mypy to org/apache/beam/model/job_management/v1/beam_artifact_api_pb2.pyi Writing mypy to org/apache/beam/model/job_management/v1/beam_expansion_api_pb2.pyi Writing mypy to org/apache/beam/model/job_management/v1/beam_job_api_pb2.pyi Writing mypy to org/apache/beam/model/fn_execution/v1/beam_provision_api_pb2.pyi Writing mypy to org/apache/beam/model/fn_execution/v1/beam_fn_api_pb2.pyi Writing mypy to org/apache/beam/model/interactive/v1/beam_interactive_api_pb2.pyi setup.py:142: UserWarning: Could not import gen_protos, skipping proto generation. warnings.warn("Could not import gen_protos, skipping proto generation.") <https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Dataflow_Streaming/ws/src/build/gradleenv/1922375555/lib/python3.7/site-packages/setuptools/dist.py>:530: UserWarning: Normalizing '2.43.0.dev' to '2.43.0.dev0' warnings.warn(tmpl.format(**locals())) warning: no files found matching 'README.md' warning: no files found matching 'NOTICE' warning: no files found matching 'LICENSE' warning: no files found matching 'LICENSE.python' warning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md > Task :sdks:python:apache_beam:testing:load_tests:installGcpTest FAILED Processing <https://ci-beam.apache.org/job/beam_LoadTests_Python_ParDo_Dataflow_Streaming/ws/src/sdks/python/build/apache-beam.tar.gz> Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'error' error: subprocess-exited-with-error × python setup.py egg_info did not run successfully. │ exit code: 1 ╰─> [10 lines of output] Traceback (most recent call last): File "<string>", line 36, in <module> File "<pip-setuptools-caller>", line 34, in <module> File "/tmp/pip-req-build-rsfnymd8/setup.py", line 169, in <module> generate_protos_first() File "/tmp/pip-req-build-rsfnymd8/setup.py", line 139, in generate_protos_first gen_protos.generate_proto_files() File "/tmp/pip-req-build-rsfnymd8/gen_protos.py", line 476, in generate_proto_files raise RuntimeError(error_msg) RuntimeError: Not in apache git tree, unable to find proto definitions. [end of output] note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip. hint: See above for details. FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':sdks:python:apache_beam:testing:load_tests:installGcpTest'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. * Get more help at https://help.gradle.org BUILD FAILED in 30s 14 actionable tasks: 8 executed, 4 from cache, 2 up-to-date Publishing build scan... https://gradle.com/s/lvg7v6vjfpjxg Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
