See
<https://builds.apache.org/job/beam_PerformanceTests_Python/942/display/redirect?page=changes>
Changes:
[luke_zhu] Revert invalid use of io.StringIO in utils/profiler.py
[kirpichov] A relative directory should be applied (if specified) even when
using a
------------------------------------------
[...truncated 726 B...]
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 3e5ce5cfd67efd4fcb4c9cb5db1cc9e677caeff8 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 3e5ce5cfd67efd4fcb4c9cb5db1cc9e677caeff8
Commit message: "Merge pull request #4713 from luke-zhu/python3"
> git rev-list --no-walk 14e4cb30105c4aeb43483155e1447ffaae2fbcb5 # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the
SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_Python] $ /bin/bash -xe
/tmp/jenkins2430456843286655857.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_Python] $ /bin/bash -xe
/tmp/jenkins6259997144783648603.sh
+ rm -rf .env
[beam_PerformanceTests_Python] $ /bin/bash -xe
/tmp/jenkins6780985243548626273.sh
+ virtualenv .env --system-site-packages
New python executable in .env/bin/python
Installing setuptools, pip...done.
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins704804726630153349.sh
+ .env/bin/pip install --upgrade setuptools pip
Downloading/unpacking setuptools from
https://pypi.python.org/packages/43/41/033a273f9a25cb63050a390ee8397acbc7eae2159195d85f06f17e7be45a/setuptools-38.5.1-py2.py3-none-any.whl#md5=908b8b5e50bf429e520b2b5fa1b350e5
Downloading/unpacking pip from
https://pypi.python.org/packages/b6/ac/7015eb97dc749283ffdec1c3a88ddb8ae03b8fad0f0e611408f196358da3/pip-9.0.1-py2.py3-none-any.whl#md5=297dbd16ef53bcef0447d245815f5144
Installing collected packages: setuptools, pip
Found existing installation: setuptools 2.2
Uninstalling setuptools:
Successfully uninstalled setuptools
Found existing installation: pip 1.5.4
Uninstalling pip:
Successfully uninstalled pip
Successfully installed setuptools pip
Cleaning up...
[beam_PerformanceTests_Python] $ /bin/bash -xe /tmp/jenkins455838049296166197.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_Python] $ /bin/bash -xe
/tmp/jenkins3167293236549596172.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in
/home/jenkins/.local/lib/python2.7/site-packages (from -r
PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in
/usr/local/lib/python2.7/dist-packages (from -r
PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from -r
PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in
/home/jenkins/.local/lib/python2.7/site-packages (from -r
PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in
/home/jenkins/.local/lib/python2.7/site-packages (from -r
PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in
/usr/local/lib/python2.7/dist-packages (from -r
PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in
/home/jenkins/.local/lib/python2.7/site-packages (from -r
PerfKitBenchmarker/requirements.txt (line 21))
Requirement already satisfied: numpy==1.13.3 in
/home/jenkins/.local/lib/python2.7/site-packages (from -r
PerfKitBenchmarker/requirements.txt (line 22))
Requirement already satisfied: functools32 in
/usr/local/lib/python2.7/dist-packages (from -r
PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from -r
PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in
/home/jenkins/.local/lib/python2.7/site-packages (from -r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages
(from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe>=0.23 in
/usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r
PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in
/usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r
PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in
/home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in
/usr/local/lib/python2.7/dist-packages (from pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in
/home/jenkins/.local/lib/python2.7/site-packages (from
requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in
/home/jenkins/.local/lib/python2.7/site-packages (from
requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.23,>=1.21.1 in
/usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.7,>=2.5 in
/usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in
/usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in
/usr/local/lib/python2.7/dist-packages (from requests>=2.9.1->pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation !=
"PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from
cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in
/usr/local/lib/python2.7/dist-packages (from
cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from
cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in
/usr/local/lib/python2.7/dist-packages (from
cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in
/home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7;
platform_python_implementation !=
"PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318:
SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name
Indication) extension to TLS is not available on this platform. This may cause
the server to present an incorrect TLS certificate, which can cause validation
failures. You can upgrade to a newer version of Python to solve this. For more
information, see
https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122:
InsecurePlatformWarning: A true SSLContext object is not available. This
prevents urllib3 from configuring SSL appropriately and may cause certain SSL
connections to fail. You can upgrade to a newer version of Python to solve
this. For more information, see
https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
[beam_PerformanceTests_Python] $ /bin/bash -xe
/tmp/jenkins4843874162345279877.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining
file://<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in
/usr/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: dill==0.2.6 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: grpcio<2,>=1.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.4.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:318:
SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name
Indication) extension to TLS is not available on this platform. This may cause
the server to present an incorrect TLS certificate, which can cause validation
failures. You can upgrade to a newer version of Python to solve this. For more
information, see
https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py>:122:
InsecurePlatformWarning: A true SSLContext object is not available. This
prevents urllib3 from configuring SSL appropriately and may cause certain SSL
connections to fail. You can upgrade to a newer version of Python to solve
this. For more information, see
https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Requirement already satisfied: httplib2<0.10,>=0.8 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in
/usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in
/usr/local/lib/python2.7/dist-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0
in /home/jenkins/.local/lib/python2.7/site-packages (from
apache-beam==2.4.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.4.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in
/usr/local/lib/python2.7/dist-packages (from
grpcio<2,>=1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: requests>=2.7.0 in
/usr/local/lib/python2.7/dist-packages (from
hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages
(from hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in
/home/jenkins/.local/lib/python2.7/site-packages (from
mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pbr>=0.11 in
/home/jenkins/.local/lib/python2.7/site-packages (from
mock<3.0.0,>=1.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in
/home/jenkins/.local/lib/python2.7/site-packages (from
oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in
/home/jenkins/.local/lib/python2.7/site-packages (from
oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in
/usr/local/lib/python2.7/dist-packages (from
oauth2client<5,>=2.0.1->apache-beam==2.4.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages
(from protobuf<4,>=3.5.0.post1->apache-beam==2.4.0.dev0)
Requirement already satisfied: fasteners>=0.14 in
/home/jenkins/.local/lib/python2.7/site-packages (from
google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in
/home/jenkins/.local/lib/python2.7/site-packages (from
proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from
google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from
google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: urllib3<1.23,>=1.21.1 in
/usr/local/lib/python2.7/dist-packages (from
requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: idna<2.7,>=2.5 in
/usr/local/lib/python2.7/dist-packages (from
requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in
/usr/local/lib/python2.7/dist-packages (from
requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in
/usr/local/lib/python2.7/dist-packages (from
requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: monotonic>=0.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from
fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from
google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-auth-httplib2 in
/home/jenkins/.local/lib/python2.7/site-packages (from
google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in
/home/jenkins/.local/lib/python2.7/site-packages (from
gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from
gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied:
proto-google-cloud-pubsub-v1[grpc]<0.16dev,>=0.15.4 in
/home/jenkins/.local/lib/python2.7/site-packages (from
gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from
google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from
google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Requirement already satisfied: ply==3.8 in
/home/jenkins/.local/lib/python2.7/site-packages (from
google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.4.0.dev0)
Installing collected packages: hdfs, apache-beam
Running setup.py develop for apache-beam
Successfully installed apache-beam hdfs-2.1.0
[beam_PerformanceTests_Python] $ /bin/bash -xe
/tmp/jenkins8181098787144289999.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing
--dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--bigquery_table=beam_performance.pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--official=true --beam_sdk=python --benchmarks=beam_integration_benchmark
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
2018-02-22 06:00:48,348 338f55b8 MainThread INFO Verbose logging to:
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/338f55b8/pkb.log>
2018-02-22 06:00:48,348 338f55b8 MainThread INFO PerfKitBenchmarker
version: v1.12.0-362-g15db1b8
2018-02-22 06:00:48,349 338f55b8 MainThread INFO Flag values:
--beam_sdk=python
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/>
--beam_it_args=--project=apache-beam-testing,--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it,--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it,--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--project=apache-beam-testing
--bigquery_table=beam_performance.pkb_results
--official
--dpb_log_level=INFO
--benchmarks=beam_integration_benchmark
2018-02-22 06:00:48,921 338f55b8 MainThread WARNING The key "flags" was not in
the default config, but was in user overrides. This may indicate a typo.
2018-02-22 06:00:48,922 338f55b8 MainThread INFO Initializing the edw
service decoder
2018-02-22 06:00:49,026 338f55b8 MainThread beam_integration_benchmark(1/1)
INFO Provisioning resources for benchmark beam_integration_benchmark
2018-02-22 06:00:49,028 338f55b8 MainThread beam_integration_benchmark(1/1)
INFO Preparing benchmark beam_integration_benchmark
2018-02-22 06:00:49,028 338f55b8 MainThread beam_integration_benchmark(1/1)
INFO Running: git clone https://github.com/apache/beam.git
2018-02-22 06:00:56,488 338f55b8 MainThread beam_integration_benchmark(1/1)
INFO Running: /home/jenkins/tools/maven/latest/bin/mvn clean install
-DskipTests -Dcheckstyle.skip=true -Pdataflow-runner
2018-02-22 06:10:01,915 338f55b8 MainThread beam_integration_benchmark(1/1)
INFO Running benchmark beam_integration_benchmark
2018-02-22 06:10:02,013 338f55b8 MainThread beam_integration_benchmark(1/1)
INFO Running: python setup.py nosetests
--tests=apache_beam.examples.wordcount_it_test --attr=IT
--test-pipeline-options="--project=apache-beam-testing"
"--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it"
"--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it"
"--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output"
--runner=TestDataflowRunner
--sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/338f55b8/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz>
2018-02-22 06:10:03,597 338f55b8 MainThread beam_integration_benchmark(1/1)
INFO Ran: {python setup.py nosetests
--tests=apache_beam.examples.wordcount_it_test --attr=IT
--test-pipeline-options="--project=apache-beam-testing"
"--staging_location=gs://temp-storage-for-end-to-end-tests/staging-it"
"--temp_location=gs://temp-storage-for-end-to-end-tests/temp-it"
"--output=gs://temp-storage-for-end-to-end-tests/py-it-cloud/output"
--runner=TestDataflowRunner
--sdk_location=<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/338f55b8/beam/sdks/python/target/apache-beam-2.4.0.dev0.tar.gz}>
ReturnCode:1, WallTime:0:01.53s, CPU:1.00s, MaxMemory:36160kb
STDOUT:
STDERR: /usr/local/lib/python2.7/dist-packages/setuptools/dist.py:355:
UserWarning: Normalizing '2.4.0.dev' to '2.4.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/338f55b8/beam/sdks/python/.eggs/nose-1.3.7-py2.7.egg/nose/plugins/manager.py>:395:
RuntimeWarning: Unable to load plugin beam_test_plugin =
test_config:BeamTestPlugin: The 'hdfs<3.0.0,>=2.1.0' distribution was not found
and is required by the application
RuntimeWarning)
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: setup.py --help [cmd1 cmd2 ...]
or: setup.py --help-commands
or: setup.py cmd --help
error: option --test-pipeline-options not recognized
2018-02-22 06:10:03,598 338f55b8 MainThread beam_integration_benchmark(1/1)
ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 622, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File
"<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 525, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File
"<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
line 159, in Run
job_type=job_type)
File
"<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,>
line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-22 06:10:03,600 338f55b8 MainThread beam_integration_benchmark(1/1)
INFO Cleaning up benchmark beam_integration_benchmark
2018-02-22 06:10:03,602 338f55b8 MainThread beam_integration_benchmark(1/1)
ERROR Exception running benchmark
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 732, in RunBenchmarkTask
RunBenchmark(spec, collector)
File
"<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 622, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File
"<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 525, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File
"<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
line 159, in Run
job_type=job_type)
File
"<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/PerfKitBenchmarker/perfkitbenchmarker/providers/gcp/gcp_dpb_dataflow.py";,>
line 90, in SubmitJob
assert retcode == 0, "Integration Test Failed."
AssertionError: Integration Test Failed.
2018-02-22 06:10:03,603 338f55b8 MainThread beam_integration_benchmark(1/1)
ERROR Benchmark 1/1 beam_integration_benchmark (UID:
beam_integration_benchmark0) failed. Execution will continue.
2018-02-22 06:10:03,603 338f55b8 MainThread beam_integration_benchmark(1/1)
INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed
Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-02-22 06:10:03,603 338f55b8 MainThread beam_integration_benchmark(1/1)
INFO Complete logs can be found at:
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/338f55b8/pkb.log>
2018-02-22 06:10:03,603 338f55b8 MainThread beam_integration_benchmark(1/1)
INFO Completion statuses can be found at:
<https://builds.apache.org/job/beam_PerformanceTests_Python/ws/runs/338f55b8/completion_statuses.json>
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user
[email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
Not sending mail to unregistered user [email protected]
