See
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/292/display/redirect?page=changes>
Changes:
[zyichi] [BEAM-9263] Bump up python sdk dataflow environment major versions
[jozsi] Update Jet version to 4.0
[kyoungha] [BEAM-9325] Added Proper Write Method in UnownedOutputStream
[kyoungha] fix format warning
[kcweaver] [BEAM-9638] Strengthen worker region & zone options tests.
[samuelw] [BEAM-9399] Change the redirection of System.err to be a custom
[kyoungha] [BEAM-9325] reflect comment : inline testing methods
[boyuanz] [BEAM-9454] Add Deduplication PTransform
[robertwb] [BEAM-9577] Rename the Artifact{Staging,Retrieval}Service.
[robertwb] [BEAM-9577] Define the new Artifact{Staging,Retrieval}Service.
[robertwb] [BEAM-9577] Regenerate protos.
[jozsi] Update Jet Runner web page with info about 4.0
[jozsi] Add Beam-Jet compatibility table
[robertwb] [BEAM-9577] Implement the new Artifact{Staging,Retrieval}Services in
[samuelw] Fix missing test import
[kyoungha] [BEAM-9325] reflect comment : Fix JAXBCoder + change test
[chamikara] Refactors X-Lang test pipelines.
[pabloem] [BEAM-9608] Increase reliance on Context Managers for FnApiRunner
------------------------------------------
[...truncated 113.54 KB...]
Collecting chardet<4,>=3.0.2
Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Using cached urllib3-1.25.8-py2.py3-none-any.whl (125 kB)
Collecting docutils<0.16,>=0.10
Using cached docutils-0.15.2-py2-none-any.whl (548 kB)
Requirement already satisfied: contextlib2; python_version < "3" in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from importlib-metadata>=0.12; python_version <
"3.8"->pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (0.6.0.post1)
Requirement already satisfied: zipp>=0.5 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from importlib-metadata>=0.12; python_version <
"3.8"->pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (1.2.0)
Requirement already satisfied: configparser>=3.5; python_version < "3" in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from importlib-metadata>=0.12; python_version <
"3.8"->pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (4.0.2)
Requirement already satisfied: scandir; python_version < "3.5" in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from pathlib2>=2.2.0; python_version <
"3.6"->pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (1.10.0)
Collecting apipkg>=1.4
Using cached apipkg-1.5-py2.py3-none-any.whl (4.9 kB)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam:
filename=apache_beam-2.21.0.dev0-py2-none-any.whl size=2035245
sha256=1f322aaf66efcbb373527b4c0233aab75988f08001ccd0253abda6c7e9e332b7
Stored in directory:
/home/jenkins/.cache/pip/wheels/8f/2c/77/dfc39e2134b1dcf200c3972c98ea93a5f510bd86e811dc94bc
Successfully built apache-beam
Installing collected packages: crcmod, dill, fastavro, docopt, certifi, idna,
chardet, urllib3, requests, hdfs, httplib2, pbr, funcsigs, mock, numpy,
pymongo, pyasn1, rsa, pyasn1-modules, oauth2client, pyparsing, pydot,
python-dateutil, pytz, typing-extensions, avro, pyvcf, pyarrow, docutils,
jmespath, botocore, s3transfer, boto3, cachetools, monotonic, fasteners,
google-apitools, google-auth, googleapis-common-protos, google-api-core,
google-cloud-core, google-cloud-datastore, grpc-google-iam-v1,
google-cloud-pubsub, google-resumable-media, google-cloud-bigquery,
google-cloud-bigtable, google-cloud-spanner, grpcio-gcp, google-cloud-dlp,
google-cloud-language, google-cloud-videointelligence, google-cloud-vision,
proto-google-cloud-datastore-v1, googledatastore, freezegun, nose,
nose-xunitmp, pandas, parameterized, pyhamcrest, pyyaml, requests-mock,
tenacity, atomicwrites, packaging, attrs, wcwidth, more-itertools, pytest,
pytest-forked, apipkg, execnet, pytest-xdist, pytest-timeout, apache-beam
Successfully installed apache-beam-2.21.0.dev0 apipkg-1.5 atomicwrites-1.3.0
attrs-19.3.0 avro-1.9.2 boto3-1.12.33 botocore-1.15.33 cachetools-3.1.1
certifi-2019.11.28 chardet-3.0.4 crcmod-1.7 dill-0.3.1.1 docopt-0.6.2
docutils-0.15.2 execnet-1.7.1 fastavro-0.21.24 fasteners-0.15 freezegun-0.3.15
funcsigs-1.0.2 google-api-core-1.16.0 google-apitools-0.5.28 google-auth-1.12.0
google-cloud-bigquery-1.24.0 google-cloud-bigtable-1.0.0
google-cloud-core-1.3.0 google-cloud-datastore-1.7.4 google-cloud-dlp-0.13.0
google-cloud-language-1.3.0 google-cloud-pubsub-1.0.2
google-cloud-spanner-1.13.0 google-cloud-videointelligence-1.13.0
google-cloud-vision-0.42.0 google-resumable-media-0.5.0
googleapis-common-protos-1.51.0 googledatastore-7.0.2 grpc-google-iam-v1-0.12.3
grpcio-gcp-0.2.2 hdfs-2.5.8 httplib2-0.12.0 idna-2.9 jmespath-0.9.5 mock-2.0.0
monotonic-1.5 more-itertools-5.0.0 nose-1.3.7 nose-xunitmp-0.4.1 numpy-1.16.6
oauth2client-3.0.0 packaging-20.3 pandas-0.24.2 parameterized-0.7.1 pbr-5.4.4
proto-google-cloud-datastore-v1-0.90.4 pyarrow-0.16.0 pyasn1-0.4.8
pyasn1-modules-0.2.8 pydot-1.4.1 pyhamcrest-1.10.1 pymongo-3.10.1
pyparsing-2.4.6 pytest-4.6.9 pytest-forked-1.1.3 pytest-timeout-1.3.4
pytest-xdist-1.31.0 python-dateutil-2.8.1 pytz-2019.3 pyvcf-0.6.8 pyyaml-5.3.1
requests-2.23.0 requests-mock-1.7.0 rsa-4.0 s3transfer-0.3.3 tenacity-5.1.5
typing-extensions-3.7.4.1 urllib3-1.25.8 wcwidth-0.1.9
> Task :sdks:python:apache_beam:testing:load_tests:run
apache_beam/__init__.py:82: UserWarning: You are using Apache Beam with Python
2. New releases of Apache Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
WARNING:root:Make sure that locally built Python SDK docker image has Python
2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev.
If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:====================
<function lift_combiners at 0x7fa584827a28> ====================
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args:
['--iterations=1', '--fanout=4']
INFO:apache_beam.runners.portability.portable_runner:Job state changed to
STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to
STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to
RUNNING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to DONE
INFO:apache_beam.testing.load_tests.load_test_metrics_utils:Load test results
for test: d6f09b3cba11404781bdca0a5fd33364 and timestamp: 1585747151.84:
INFO:apache_beam.testing.load_tests.load_test_metrics_utils:Metric:
python_flink_batch_GBK_4_runtime Value: 214
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 4m 43s
5 actionable tasks: 4 executed, 1 up-to-date
Publishing build scan...
https://gradle.com/s/n2ymb4em3deoa
[beam_LoadTests_Python_GBK_Flink_Batch] $ /bin/bash -xe
/tmp/jenkins3101944233103736732.sh
+ echo src Load test: fanout 8 times with 2GB 10-byte records total src
src Load test: fanout 8 times with 2GB 10-byte records total src
[Gradle] - Launching build.
[src] $
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/gradlew>
-PloadTest.mainClass=apache_beam.testing.load_tests.group_by_key_test
-Prunner=PortableRunner
'-PloadTest.args=--job_name=load_tests_Python_Flink_Batch_GBK_5_0401100353
--publish_to_big_query=true --project=apache-beam-testing
--metrics_dataset=load_test --metrics_table=python_flink_batch_GBK_5
--input_options='{"num_records": 2500000,"key_size": 10,"value_size":90}'
--iterations=1 --fanout=8 --parallelism=16 --job_endpoint=localhost:8099
--environment_config=gcr.io/apache-beam-testing/beam_portability/beam_python2.7_sdk:latest
--environment_type=DOCKER --runner=PortableRunner' --continue --max-workers=12
-Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g
:sdks:python:apache_beam:testing:load_tests:run
> Task :buildSrc:compileJava NO-SOURCE
> Task :buildSrc:compileGroovy UP-TO-DATE
> Task :buildSrc:pluginDescriptors UP-TO-DATE
> Task :buildSrc:processResources UP-TO-DATE
> Task :buildSrc:classes UP-TO-DATE
> Task :buildSrc:jar UP-TO-DATE
> Task :buildSrc:assemble UP-TO-DATE
> Task :buildSrc:spotlessGroovy UP-TO-DATE
> Task :buildSrc:spotlessGroovyCheck UP-TO-DATE
> Task :buildSrc:spotlessGroovyGradle UP-TO-DATE
> Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE
> Task :buildSrc:spotlessCheck UP-TO-DATE
> Task :buildSrc:pluginUnderTestMetadata UP-TO-DATE
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validateTaskProperties UP-TO-DATE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:python:setupVirtualenv UP-TO-DATE
> Task :sdks:python:apache_beam:testing:load_tests:setupVirtualenv UP-TO-DATE
> Task :sdks:python:sdist
setup.py:251: UserWarning: You are using Apache Beam with Python 2. New
releases of Apache Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/setuptools/dist.py>:476:
UserWarning: Normalizing '2.21.0.dev' to '2.21.0.dev0'
normalized_version,
INFO:gen_protos:Skipping proto regeneration: all files up to date
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
warning: sdist: standard file not found: should have one of README, README.rst,
README.txt, README.md
> Task :sdks:python:apache_beam:testing:load_tests:installGcpTest
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020.
Please upgrade your Python as Python 2.7 is no longer maintained. A future
version of pip will drop support for Python 2.7. More details about Python 2
support in pip, can be found at
https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Processing
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/sdks/python/build/apache-beam.tar.gz>
Requirement already satisfied: crcmod<2.0,>=1.7 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.7)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (0.3.1.1)
Requirement already satisfied: fastavro<0.22,>=0.21.4 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (0.21.24)
Requirement already satisfied: future<1.0.0,>=0.16.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (0.16.0)
Requirement already satisfied: grpcio<2,>=1.12.1 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.27.2)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (2.5.8)
Requirement already satisfied: httplib2<=0.12.0,>=0.8 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (0.12.0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (2.0.0)
Requirement already satisfied: numpy<2,>=1.14.3 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.16.6)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (3.10.1)
Requirement already satisfied: oauth2client<4,>=2.0.1 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (3.0.0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (3.11.3)
Requirement already satisfied: pydot<2,>=1.2.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.4.1)
Requirement already satisfied: python-dateutil<3,>=2.8.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (2.8.1)
Requirement already satisfied: pytz>=2018.3 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (2019.3)
Requirement already satisfied: typing-extensions<3.8.0,>=3.7.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (3.7.4.1)
Requirement already satisfied: avro<1.10.0,>=1.8.1 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.9.2)
Requirement already satisfied: funcsigs<2,>=1.0.2 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.0.2)
Requirement already satisfied: futures<4.0.0,>=3.2.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (3.3.0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (0.6.8)
Requirement already satisfied: pyarrow<0.17.0,>=0.15.1 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (0.16.0)
Requirement already satisfied: typing<3.8.0,>=3.7.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (3.7.4.1)
Requirement already satisfied: boto3>=1.9 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.12.33)
Requirement already satisfied: cachetools<4,>=3.1.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (3.1.1)
Requirement already satisfied: google-apitools<0.5.29,>=0.5.28 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (0.5.28)
Requirement already satisfied: google-cloud-datastore<1.8.0,>=1.7.1 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.7.4)
Requirement already satisfied: google-cloud-pubsub<1.1.0,>=0.39.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.0.2)
Requirement already satisfied: google-cloud-bigquery<=1.24.0,>=1.6.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.24.0)
Requirement already satisfied: google-cloud-core<2,>=0.28.1 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-bigtable<1.1.0,>=0.31.1 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.0.0)
Requirement already satisfied: google-cloud-spanner<1.14.0,>=1.13.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.13.0)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (0.2.2)
Requirement already satisfied: google-cloud-dlp<=0.13.0,>=0.12.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (0.13.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.3.0)
Requirement already satisfied: google-cloud-videointelligence<1.14.0,>=1.8.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.13.0)
Requirement already satisfied: google-cloud-vision<0.43.0,>=0.38.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (0.42.0)
Requirement already satisfied: googledatastore<7.1,>=7.0.1 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (7.0.2)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0
in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (0.90.4)
Requirement already satisfied: freezegun>=0.3.12 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (0.3.15)
Requirement already satisfied: nose>=1.3.7 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.3.7)
Requirement already satisfied: nose_xunitmp>=0.4.1 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (0.4.1)
Requirement already satisfied: pandas<0.25,>=0.23.4 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (0.24.2)
Requirement already satisfied: parameterized<0.8.0,>=0.7.1 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (0.7.1)
Requirement already satisfied: pyhamcrest!=1.10.0,<2.0.0,>=1.9 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.10.1)
Requirement already satisfied: pyyaml<6.0.0,>=3.12 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (5.3.1)
Requirement already satisfied: requests_mock<2.0,>=1.7 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.7.0)
Requirement already satisfied: tenacity<6.0,>=5.0.2 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (5.1.5)
Requirement already satisfied: pytest<5.0,>=4.4.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (4.6.9)
Requirement already satisfied: pytest-xdist<2,>=1.29.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.31.0)
Requirement already satisfied: pytest-timeout<2,>=1.3.3 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from apache-beam==2.21.0.dev0) (1.3.4)
Requirement already satisfied: six>=1.5.2 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from grpcio<2,>=1.12.1->apache-beam==2.21.0.dev0) (1.14.0)
Requirement already satisfied: enum34>=1.0.4; python_version < "3.4" in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from grpcio<2,>=1.12.1->apache-beam==2.21.0.dev0) (1.1.10)
Requirement already satisfied: docopt in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from hdfs<3.0.0,>=2.1.0->apache-beam==2.21.0.dev0) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from hdfs<3.0.0,>=2.1.0->apache-beam==2.21.0.dev0) (2.23.0)
Requirement already satisfied: pbr>=0.11 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from mock<3.0.0,>=1.0.1->apache-beam==2.21.0.dev0) (5.4.4)
Requirement already satisfied: rsa>=3.1.4 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from oauth2client<4,>=2.0.1->apache-beam==2.21.0.dev0) (4.0)
Requirement already satisfied: pyasn1>=0.1.7 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from oauth2client<4,>=2.0.1->apache-beam==2.21.0.dev0) (0.4.8)
Requirement already satisfied: pyasn1-modules>=0.0.5 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from oauth2client<4,>=2.0.1->apache-beam==2.21.0.dev0) (0.2.8)
Requirement already satisfied: setuptools in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from protobuf<4,>=3.5.0.post1->apache-beam==2.21.0.dev0) (44.1.0)
Requirement already satisfied: pyparsing>=2.1.4 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from pydot<2,>=1.2.0->apache-beam==2.21.0.dev0) (2.4.6)
Requirement already satisfied: s3transfer<0.4.0,>=0.3.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from boto3>=1.9->apache-beam==2.21.0.dev0) (0.3.3)
Requirement already satisfied: botocore<1.16.0,>=1.15.33 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from boto3>=1.9->apache-beam==2.21.0.dev0) (1.15.33)
Requirement already satisfied: jmespath<1.0.0,>=0.7.1 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from boto3>=1.9->apache-beam==2.21.0.dev0) (0.9.5)
Requirement already satisfied: fasteners>=0.14 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from google-apitools<0.5.29,>=0.5.28->apache-beam==2.21.0.dev0) (0.15)
Requirement already satisfied: google-api-core[grpc]<2.0.0dev,>=1.6.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from google-cloud-datastore<1.8.0,>=1.7.1->apache-beam==2.21.0.dev0) (1.16.0)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from google-cloud-pubsub<1.1.0,>=0.39.0->apache-beam==2.21.0.dev0) (0.12.3)
Requirement already satisfied: google-auth<2.0dev,>=1.9.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from google-cloud-bigquery<=1.24.0,>=1.6.0->apache-beam==2.21.0.dev0) (1.12.0)
Requirement already satisfied: google-resumable-media<0.6dev,>=0.5.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from google-cloud-bigquery<=1.24.0,>=1.6.0->apache-beam==2.21.0.dev0) (0.5.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from
proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.21.0.dev0)
(1.51.0)
Requirement already satisfied: monotonic>=0.6; python_version == "2.7" in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from tenacity<6.0,>=5.0.2->apache-beam==2.21.0.dev0) (1.5)
Requirement already satisfied: atomicwrites>=1.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (1.3.0)
Requirement already satisfied: pluggy<1.0,>=0.12 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (0.13.1)
Requirement already satisfied: packaging in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (20.3)
Requirement already satisfied: attrs>=17.4.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (19.3.0)
Requirement already satisfied: importlib-metadata>=0.12; python_version < "3.8"
in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (1.6.0)
Requirement already satisfied: wcwidth in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (0.1.9)
Requirement already satisfied: more-itertools<6.0.0,>=4.0.0; python_version <=
"2.7" in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (5.0.0)
Requirement already satisfied: pathlib2>=2.2.0; python_version < "3.6" in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (2.3.5)
Requirement already satisfied: py>=1.5.0 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (1.8.1)
Requirement already satisfied: pytest-forked in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from pytest-xdist<2,>=1.29.0->apache-beam==2.21.0.dev0) (1.1.3)
Requirement already satisfied: execnet>=1.1 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from pytest-xdist<2,>=1.29.0->apache-beam==2.21.0.dev0) (1.7.1)
Requirement already satisfied: certifi>=2017.4.17 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.21.0.dev0)
(2019.11.28)
Requirement already satisfied: idna<3,>=2.5 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.21.0.dev0) (2.9)
Requirement already satisfied: chardet<4,>=3.0.2 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.21.0.dev0) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.21.0.dev0) (1.25.8)
Requirement already satisfied: docutils<0.16,>=0.10 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from botocore<1.16.0,>=1.15.33->boto3>=1.9->apache-beam==2.21.0.dev0) (0.15.2)
Requirement already satisfied: contextlib2; python_version < "3" in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from importlib-metadata>=0.12; python_version <
"3.8"->pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (0.6.0.post1)
Requirement already satisfied: zipp>=0.5 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from importlib-metadata>=0.12; python_version <
"3.8"->pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (1.2.0)
Requirement already satisfied: configparser>=3.5; python_version < "3" in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from importlib-metadata>=0.12; python_version <
"3.8"->pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (4.0.2)
Requirement already satisfied: scandir; python_version < "3.5" in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from pathlib2>=2.2.0; python_version <
"3.6"->pytest<5.0,>=4.4.0->apache-beam==2.21.0.dev0) (1.10.0)
Requirement already satisfied: apipkg>=1.4 in
<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/build/gradleenv/1329484227/lib/python2.7/site-packages>
(from execnet>=1.1->pytest-xdist<2,>=1.29.0->apache-beam==2.21.0.dev0) (1.5)
Building wheels for collected packages: apache-beam
Building wheel for apache-beam (setup.py): started
Building wheel for apache-beam (setup.py): finished with status 'done'
Created wheel for apache-beam:
filename=apache_beam-2.21.0.dev0-py2-none-any.whl size=2035245
sha256=f49cf28985c187d97522d4cf0bafabc111bd6a20bd45f1e48b00c601b80af823
Stored in directory:
/home/jenkins/.cache/pip/wheels/8f/2c/77/dfc39e2134b1dcf200c3972c98ea93a5f510bd86e811dc94bc
Successfully built apache-beam
Installing collected packages: apache-beam
Attempting uninstall: apache-beam
Found existing installation: apache-beam 2.21.0.dev0
Uninstalling apache-beam-2.21.0.dev0:
Successfully uninstalled apache-beam-2.21.0.dev0
Successfully installed apache-beam-2.21.0.dev0
> Task :sdks:python:apache_beam:testing:load_tests:run
apache_beam/__init__.py:82: UserWarning: You are using Apache Beam with Python
2. New releases of Apache Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
WARNING:root:Make sure that locally built Python SDK docker image has Python
2.7 interpreter.
INFO:root:Using Python SDK docker image: apache/beam_python2.7_sdk:2.21.0.dev.
If the image is not available at local, we will try to pull from hub.docker.com
INFO:apache_beam.runners.portability.fn_api_runner.translations:====================
<function lift_combiners at 0x7f99e3801a28> ====================
WARNING:apache_beam.options.pipeline_options:Discarding unparseable args:
['--iterations=1', '--fanout=8']
INFO:apache_beam.runners.portability.portable_runner:Job state changed to
STOPPED
INFO:apache_beam.runners.portability.portable_runner:Job state changed to
STARTING
INFO:apache_beam.runners.portability.portable_runner:Job state changed to
RUNNING
ERROR:root:java.lang.OutOfMemoryError: Metaspace
INFO:apache_beam.runners.portability.portable_runner:Job state changed to FAILED
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File
"<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/group_by_key_test.py";,>
line 113, in <module>
GroupByKeyTest().run()
File "apache_beam/testing/load_tests/load_test.py", line 104, in run
self.result = self.pipeline.run()
File "apache_beam/testing/test_pipeline.py", line 114, in run
state = result.wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 550, in
wait_until_finish
(self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
load_tests_Python_Flink_Batch_GBK_5_0401100353_27e8be8c-9d0b-4ff6-833e-e1194ce889c4
failed in state FAILED: java.lang.OutOfMemoryError: Metaspace
> Task :sdks:python:apache_beam:testing:load_tests:run FAILED
FAILURE: Build failed with an exception.
* Where:
Build file
'<https://builds.apache.org/job/beam_LoadTests_Python_GBK_Flink_Batch/ws/src/sdks/python/apache_beam/testing/load_tests/build.gradle'>
line: 51
* What went wrong:
Execution failed for task ':sdks:python:apache_beam:testing:load_tests:run'.
> error occurred
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 30s
5 actionable tasks: 3 executed, 2 up-to-date
Publishing build scan...
https://gradle.com/s/x2zp3t2sejqcm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
