See <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/18/display/redirect?page=changes>
Changes: [Kenneth Knowles] [BEAM-10080] Fix PipelineOptions tests for Java 11 compat [Kenneth Knowles] [BEAM-10080] Fix string match in ReflectHelpersTest for Java 11 [Udi Meiri] [BEAM-7014] filesystemio.py PipeStream rewind fix [Kenneth Knowles] Only add test task in applyPythonNature if it does not already exist [Kenneth Knowles] Add UsesTimersInParDo to test that was missing it [Kenneth Knowles] Reduce Java portable ArtifactRetrieval chunk size to avoid breaking [Kenneth Knowles] Add ability to specify stdout and stderr files for local_job_service; [Kenneth Knowles] Create TestUniversalRunner for testing against Python ULR [Kenneth Knowles] Add validatesRunner suite for local job service [kcweaver] [BEAM-10460] Limit SparkPortableExecutionTest to a single thread. [Kenneth Knowles] Exclude beam.* from license download since these are invalid artifact [noreply] [BEAM-10420] Fix minor race condition related to splitting that will [noreply] Merge pull request #12236 from [BEAM-10419] Ignore missing temporary [noreply] Make tracking/cleanup of cache and in-environment states per pipeline ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on apache-beam-jenkins-10 (beam) in workspace <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/> No credentials specified Wiping out workspace first. Cloning the remote Git repository Cloning repository https://github.com/apache/beam.git > git init > <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/src> > # timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # > timeout=10 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git fetch --tags --progress https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* # > timeout=10 > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 613edd392773f16c3411c75c821be8e78d867532 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 613edd392773f16c3411c75c821be8e78d867532 # timeout=10 Commit message: "Make tracking/cleanup of cache and in-environment states per pipeline (#12249)" > git rev-list --no-walk 7a77912fa4cbdc3b25a806322c0461a2c054aaa3 # timeout=10 [EnvInject] - Executing scripts and injecting environment variables after the SCM step. [EnvInject] - Injecting as environment variables the properties content SPARK_LOCAL_IP=127.0.0.1 [EnvInject] - Variables injected successfully. [beam_PerformanceTests_WordCountIT_Py36] $ /bin/bash -xe /tmp/jenkins3433649267655929762.sh + rm -rf <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/PerfKitBenchmarker> [beam_PerformanceTests_WordCountIT_Py36] $ /bin/bash -xe /tmp/jenkins7495991981260030802.sh + rm -rf <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/env/.perfkit_env> [beam_PerformanceTests_WordCountIT_Py36] $ /bin/bash -xe /tmp/jenkins3262123177911168218.sh + virtualenv <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/env/.perfkit_env> --python=python2.7 New python executable in <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/env/.perfkit_env/bin/python2.7> Also creating executable in <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/env/.perfkit_env/bin/python> Installing setuptools, pip, wheel... done. Running virtualenv with interpreter /usr/bin/python2.7 [beam_PerformanceTests_WordCountIT_Py36] $ /bin/bash -xe /tmp/jenkins7110509753400458736.sh + <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (44.1.1) Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (20.1.1) [beam_PerformanceTests_WordCountIT_Py36] $ /bin/bash -xe /tmp/jenkins5624144621380491166.sh + git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/PerfKitBenchmarker> Cloning into '<https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/PerfKitBenchmarker'...> [beam_PerformanceTests_WordCountIT_Py36] $ /bin/bash -xe /tmp/jenkins6296037084690632027.sh + <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/env/.perfkit_env/bin/pip> install -r <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/PerfKitBenchmarker/requirements.txt> DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support Processing /home/jenkins/.cache/pip/wheels/b9/0a/0a/a99d0c1fe13d01efdf7aa29be87a4b1ac608072c150c669ba6/rsa-3.1.4-py2-none-any.whl Processing /home/jenkins/.cache/pip/wheels/37/83/b0/40d9e9f3d5a7021dfda2f3ea1f0088235679cd7747761c7b93/absl_py-0.9.0-py2-none-any.whl Collecting jinja2>=2.7 Using cached Jinja2-2.11.2-py2.py3-none-any.whl (125 kB) Requirement already satisfied: setuptools>=40.3.0 in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/PerfKitBenchmarker/requirements.txt> (line 17)) (44.1.1) Collecting colorlog[windows]==2.6.0 Using cached colorlog-2.6.0-py2.py3-none-any.whl (11 kB) Processing /home/jenkins/.cache/pip/wheels/7b/1f/a0/7ce496c69b9be7cef75645749e256d5ef2dc9d08336f92248b/blinker-1.4-py2-none-any.whl Collecting futures>=3.0.3 Using cached futures-3.3.0-py2-none-any.whl (16 kB) Processing /home/jenkins/.cache/pip/wheels/87/9b/a7/9bfdaa1487acce958269a6b5f86db2e4d38204dff4e256e23a/PyYAML-5.1.2-cp27-cp27mu-linux_x86_64.whl Collecting pint>=0.7 Using cached Pint-0.9-py2.py3-none-any.whl (138 kB) Collecting numpy==1.16.4 Using cached numpy-1.16.4-cp27-cp27mu-manylinux1_x86_64.whl (17.0 MB) Collecting contextlib2>=0.5.1 Using cached contextlib2-0.6.0.post1-py2.py3-none-any.whl (9.8 kB) Processing /home/jenkins/.cache/pip/wheels/2d/f7/d6/a43d1d088536280fca881232b9c352fa8c6be425d53e09d595/pywinrm-0.4.1-py2.py3-none-any.whl Processing /home/jenkins/.cache/pip/wheels/dc/f9/36/5ff60c0209ec9f77bce9aa535b7e570a87da8731ba6416db9d/timeout_decorator-0.4.1-py2-none-any.whl Collecting google-cloud-datastore Using cached google_cloud_datastore-1.13.1-py2.py3-none-any.whl (131 kB) Collecting google-cloud-monitoring Using cached google_cloud_monitoring-1.0.0-py2.py3-none-any.whl (235 kB) Collecting pyasn1>=0.1.3 Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB) Collecting six Using cached six-1.15.0-py2.py3-none-any.whl (10 kB) Collecting enum34; python_version < "3.4" Using cached enum34-1.1.10-py2-none-any.whl (11 kB) Collecting MarkupSafe>=0.23 Using cached MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl (24 kB) Collecting colorama; extra == "windows" Using cached colorama-0.4.3-py2.py3-none-any.whl (15 kB) Collecting funcsigs; python_version == "2.7" Using cached funcsigs-1.0.2-py2.py3-none-any.whl (17 kB) Collecting requests-ntlm>=0.3.0 Using cached requests_ntlm-1.1.0-py2.py3-none-any.whl (5.7 kB) Collecting requests>=2.9.1 Using cached requests-2.24.0-py2.py3-none-any.whl (61 kB) Collecting xmltodict Using cached xmltodict-0.12.0-py2.py3-none-any.whl (9.2 kB) Collecting google-api-core[grpc]<2.0.0dev,>=1.14.0 Using cached google_api_core-1.21.0-py2.py3-none-any.whl (90 kB) Collecting google-cloud-core<2.0dev,>=1.0.3 Using cached google_cloud_core-1.3.0-py2.py3-none-any.whl (26 kB) Collecting cryptography>=1.3 Using cached cryptography-2.9.2-cp27-cp27mu-manylinux2010_x86_64.whl (2.7 MB) Collecting ntlm-auth>=1.0.2 Using cached ntlm_auth-1.5.0-py2.py3-none-any.whl (29 kB) Collecting certifi>=2017.4.17 Using cached certifi-2020.6.20-py2.py3-none-any.whl (156 kB) Collecting idna<3,>=2.5 Using cached idna-2.10-py2.py3-none-any.whl (58 kB) Collecting chardet<4,>=3.0.2 Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB) Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 Using cached urllib3-1.25.9-py2.py3-none-any.whl (126 kB) Collecting google-auth<2.0dev,>=1.18.0 Using cached google_auth-1.19.0-py2.py3-none-any.whl (91 kB) Collecting protobuf>=3.12.0 Using cached protobuf-3.12.2-cp27-cp27mu-manylinux1_x86_64.whl (1.3 MB) Collecting googleapis-common-protos<2.0dev,>=1.6.0 Using cached googleapis_common_protos-1.52.0-py2.py3-none-any.whl (100 kB) Collecting pytz Using cached pytz-2020.1-py2.py3-none-any.whl (510 kB) Collecting grpcio<2.0dev,>=1.29.0; extra == "grpc" Using cached grpcio-1.30.0-cp27-cp27mu-manylinux2010_x86_64.whl (3.0 MB) Collecting cffi!=1.11.3,>=1.8 Using cached cffi-1.14.0-cp27-cp27mu-manylinux1_x86_64.whl (387 kB) Collecting ipaddress; python_version < "3" Using cached ipaddress-1.0.23-py2.py3-none-any.whl (18 kB) Collecting pyasn1-modules>=0.2.1 Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB) Collecting cachetools<5.0,>=2.0.0 Using cached cachetools-3.1.1-py2.py3-none-any.whl (11 kB) Collecting pycparser Using cached pycparser-2.20-py2.py3-none-any.whl (112 kB) Installing collected packages: pyasn1, rsa, six, enum34, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, contextlib2, certifi, idna, chardet, urllib3, requests, pycparser, cffi, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator, pyasn1-modules, cachetools, google-auth, protobuf, googleapis-common-protos, pytz, grpcio, google-api-core, google-cloud-core, google-cloud-datastore, google-cloud-monitoring Successfully installed MarkupSafe-1.1.1 PyYAML-5.1.2 absl-py-0.9.0 blinker-1.4 cachetools-3.1.1 certifi-2020.6.20 cffi-1.14.0 chardet-3.0.4 colorama-0.4.3 colorlog-2.6.0 contextlib2-0.6.0.post1 cryptography-2.9.2 enum34-1.1.10 funcsigs-1.0.2 futures-3.3.0 google-api-core-1.21.0 google-auth-1.19.0 google-cloud-core-1.3.0 google-cloud-datastore-1.13.1 google-cloud-monitoring-1.0.0 googleapis-common-protos-1.52.0 grpcio-1.30.0 idna-2.10 ipaddress-1.0.23 jinja2-2.11.2 ntlm-auth-1.5.0 numpy-1.16.4 pint-0.9 protobuf-3.12.2 pyasn1-0.4.8 pyasn1-modules-0.2.8 pycparser-2.20 pytz-2020.1 pywinrm-0.4.1 requests-2.24.0 requests-ntlm-1.1.0 rsa-3.1.4 six-1.15.0 timeout-decorator-0.4.1 urllib3-1.25.9 xmltodict-0.12.0 [beam_PerformanceTests_WordCountIT_Py36] $ /bin/bash -xe /tmp/jenkins1818693949098847534.sh + <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/env/.perfkit_env/bin/python> <https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.wordcount_py36_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/> --beam_location=<https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/src> --official=true --dpb_service_zone=fake_zone --beam_sdk=python --benchmarks=beam_integration_benchmark --beam_it_class=apache_beam.examples.wordcount_it_test:WordCountIT.test_wordcount_it --beam_it_module=:sdks:python:test-suites:dataflow:py36 --beam_prebuilt=true --beam_python_sdk_location=build/apache-beam.tar.gz --beam_runner=TestDataflowRunner --beam_it_timeout=1200 '--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--input=gs://apache-beam-samples/input_small_files/ascii_sort_1MB_input.0000*,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output,--expect_checksum=ea0ca2e5ee4ea5f218790f28d0b9fe7d09d8d710,--num_workers=10,--autoscaling_algorithm=NONE' Traceback (most recent call last): File "<https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/PerfKitBenchmarker/pkb.py";,> line 19, in <module> from perfkitbenchmarker.pkb import Main File "<https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,> line 76, in <module> from perfkitbenchmarker import archive File "<https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/PerfKitBenchmarker/perfkitbenchmarker/archive.py";,> line 24, in <module> from perfkitbenchmarker.providers.aws.util import AWS_PATH File "<https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/__init__.py";,> line 19, in <module> from perfkitbenchmarker import events File "<https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/PerfKitBenchmarker/perfkitbenchmarker/events.py";,> line 26, in <module> from perfkitbenchmarker import data File "<https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/PerfKitBenchmarker/perfkitbenchmarker/data/__init__.py";,> line 34, in <module> from perfkitbenchmarker import temp_dir File "<https://ci-beam.apache.org/job/beam_PerformanceTests_WordCountIT_Py36/ws/PerfKitBenchmarker/perfkitbenchmarker/temp_dir.py";,> line 29, in <module> import functools32 as functools ImportError: No module named functools32 Build step 'Execute shell' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
