See
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/41/display/redirect>
------------------------------------------
[...truncated 732 B...]
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision b8cf07dc03643ded1d4e9d45caf4151316cf2c3e (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f b8cf07dc03643ded1d4e9d45caf4151316cf2c3e
Commit message: "Merge pull request #5105: [BEAM-3991] Updates gcsio to use a
API specific batch endpoint"
> git rev-list --no-walk b8cf07dc03643ded1d4e9d45caf4151316cf2c3e # timeout=10
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the
SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[beam_PerformanceTests_MongoDBIO_IT] $ /bin/bash -xe
/tmp/jenkins2652988010595885638.sh
+ gcloud container clusters get-credentials io-datastores --zone=us-central1-a
--verbosity=debug
DEBUG: Running gcloud.container.clusters.get-credentials with
Namespace(__calliope_internal_deepest_parser=ArgumentParser(prog='gcloud.container.clusters.get-credentials',
usage=None, description='See
https://cloud.google.com/container-engine/docs/kubectl for\nkubectl
documentation.', version=None, formatter_class=<class
'argparse.HelpFormatter'>, conflict_handler='error', add_help=False),
account=None, api_version=None, authority_selector=None,
authorization_token_file=None, cmd_func=<bound method Command.Run of
<googlecloudsdk.calliope.backend.Command object at 0x7f3bc0efd510>>,
command_path=['gcloud', 'container', 'clusters', 'get-credentials'],
configuration=None, credential_file_override=None, document=None, format=None,
h=None, help=None, http_timeout=None, log_http=None, name='io-datastores',
project=None, quiet=None, trace_email=None, trace_log=None, trace_token=None,
user_output_enabled=None, verbosity='debug', version=None,
zone='us-central1-a').
WARNING: Accessing a Container Engine cluster requires the kubernetes
commandline
client [kubectl]. To install, run
$ gcloud components install kubectl
Fetching cluster endpoint and auth data.
DEBUG: Saved kubeconfig to /home/jenkins/.kube/config
kubeconfig entry generated for io-datastores.
INFO: Display format "default".
[beam_PerformanceTests_MongoDBIO_IT] $ /bin/bash -xe
/tmp/jenkins4258390343795895899.sh
+ cp /home/jenkins/.kube/config
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
[beam_PerformanceTests_MongoDBIO_IT] $ /bin/bash -xe
/tmp/jenkins5822718769865421515.sh
+ kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
create namespace mongodbioit-1523527322360
namespace "mongodbioit-1523527322360" created
[beam_PerformanceTests_MongoDBIO_IT] $ /bin/bash -xe
/tmp/jenkins5010043646183418103.sh
++ kubectl config current-context
+ kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
config set-context gke_apache-beam-testing_us-central1-a_io-datastores
--namespace=mongodbioit-1523527322360
Context "gke_apache-beam-testing_us-central1-a_io-datastores" modified.
[beam_PerformanceTests_MongoDBIO_IT] $ /bin/bash -xe
/tmp/jenkins6806629079233356699.sh
+ rm -rf PerfKitBenchmarker
[beam_PerformanceTests_MongoDBIO_IT] $ /bin/bash -xe
/tmp/jenkins4246352469306246169.sh
+ rm -rf .env
[beam_PerformanceTests_MongoDBIO_IT] $ /bin/bash -xe
/tmp/jenkins4015817620624415186.sh
+ virtualenv .env --system-site-packages
New python executable in
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/.env/bin/python>
Installing setuptools, pip, wheel...done.
[beam_PerformanceTests_MongoDBIO_IT] $ /bin/bash -xe
/tmp/jenkins3814270304076204423.sh
+ .env/bin/pip install --upgrade setuptools pip
Requirement already up-to-date: setuptools in ./.env/lib/python2.7/site-packages
Requirement already up-to-date: pip in ./.env/lib/python2.7/site-packages
[beam_PerformanceTests_MongoDBIO_IT] $ /bin/bash -xe
/tmp/jenkins6863220779979149566.sh
+ git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git
Cloning into 'PerfKitBenchmarker'...
[beam_PerformanceTests_MongoDBIO_IT] $ /bin/bash -xe
/tmp/jenkins8987329480272096066.sh
+ .env/bin/pip install -r PerfKitBenchmarker/requirements.txt
Requirement already satisfied: absl-py in
/home/jenkins/.local/lib/python2.7/site-packages (from -r
PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: jinja2>=2.7 in
/usr/local/lib/python2.7/dist-packages (from -r
PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages
(from -r PerfKitBenchmarker/requirements.txt (line 16))
Requirement already satisfied: colorlog[windows]==2.6.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from -r
PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: blinker>=1.3 in
/home/jenkins/.local/lib/python2.7/site-packages (from -r
PerfKitBenchmarker/requirements.txt (line 18))
Requirement already satisfied: futures>=3.0.3 in
/home/jenkins/.local/lib/python2.7/site-packages (from -r
PerfKitBenchmarker/requirements.txt (line 19))
Requirement already satisfied: PyYAML==3.12 in
/home/jenkins/.local/lib/python2.7/site-packages (from -r
PerfKitBenchmarker/requirements.txt (line 20))
Requirement already satisfied: pint>=0.7 in
/home/jenkins/.local/lib/python2.7/site-packages (from -r
PerfKitBenchmarker/requirements.txt (line 21))
Collecting numpy==1.13.3 (from -r PerfKitBenchmarker/requirements.txt (line 22))
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:339:
SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name
Indication) extension to TLS is not available on this platform. This may cause
the server to present an incorrect TLS certificate, which can cause validation
failures. You can upgrade to a newer version of Python to solve this. For more
information, see
https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137:
InsecurePlatformWarning: A true SSLContext object is not available. This
prevents urllib3 from configuring SSL appropriately and may cause certain SSL
connections to fail. You can upgrade to a newer version of Python to solve
this. For more information, see
https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied: functools32 in
/home/jenkins/.local/lib/python2.7/site-packages (from -r
PerfKitBenchmarker/requirements.txt (line 23))
Requirement already satisfied: contextlib2>=0.5.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from -r
PerfKitBenchmarker/requirements.txt (line 24))
Requirement already satisfied: pywinrm in
/home/jenkins/.local/lib/python2.7/site-packages (from -r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: six in
/home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r
PerfKitBenchmarker/requirements.txt (line 14))
Requirement already satisfied: MarkupSafe in
/usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r
PerfKitBenchmarker/requirements.txt (line 15))
Requirement already satisfied: colorama; extra == "windows" in
/usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r
PerfKitBenchmarker/requirements.txt (line 17))
Requirement already satisfied: xmltodict in
/home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests-ntlm>=0.3.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: requests>=2.9.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ntlm-auth>=1.0.2 in
/home/jenkins/.local/lib/python2.7/site-packages (from
requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cryptography>=1.3 in
/home/jenkins/.local/lib/python2.7/site-packages (from
requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: idna<2.6,>=2.5 in
/home/jenkins/.local/lib/python2.7/site-packages (from
requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: urllib3<1.22,>=1.21.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from
requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in
/home/jenkins/.local/lib/python2.7/site-packages (from
requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: certifi>=2017.4.17 in
/home/jenkins/.local/lib/python2.7/site-packages (from
requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: cffi>=1.7; platform_python_implementation !=
"PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from
cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: enum34; python_version < "3" in
/home/jenkins/.local/lib/python2.7/site-packages (from
cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: asn1crypto>=0.21.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from
cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: ipaddress; python_version < "3" in
/usr/local/lib/python2.7/dist-packages (from
cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Requirement already satisfied: pycparser in
/home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7;
platform_python_implementation !=
"PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r
PerfKitBenchmarker/requirements.txt (line 25))
Installing collected packages: numpy
Found existing installation: numpy 1.11.3
Not uninstalling numpy at /home/jenkins/.local/lib/python2.7/site-packages,
outside environment
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/.env>
Successfully installed numpy-1.13.3
[beam_PerformanceTests_MongoDBIO_IT] $ /bin/bash -xe
/tmp/jenkins7992666220579952034.sh
+ .env/bin/pip install -e 'src/sdks/python/[gcp,test]'
Obtaining
file://<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/sdks/python>
Requirement already satisfied: avro<2.0.0,>=1.8.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: crcmod<2.0,>=1.7 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: dill==0.2.6 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Collecting grpcio<2,>=1.8 (from apache-beam==2.5.0.dev0)
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:339:
SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name
Indication) extension to TLS is not available on this platform. This may cause
the server to present an incorrect TLS certificate, which can cause validation
failures. You can upgrade to a newer version of Python to solve this. For more
information, see
https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
SNIMissingWarning
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/.env/local/lib/python2.7/site-packages/pip/_vendor/urllib3/util/ssl_.py>:137:
InsecurePlatformWarning: A true SSLContext object is not available. This
prevents urllib3 from configuring SSL appropriately and may cause certain SSL
connections to fail. You can upgrade to a newer version of Python to solve
this. For more information, see
https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecurePlatformWarning
Using cached grpcio-1.10.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam==2.5.0.dev0)
Requirement already satisfied: httplib2<0.10,>=0.8 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: mock<3.0.0,>=1.0.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: oauth2client<5,>=2.0.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: protobuf<4,>=3.5.0.post1 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Collecting pytz>=2018.3 (from apache-beam==2.5.0.dev0)
Using cached pytz-2018.4-py2.py3-none-any.whl
Requirement already satisfied: pyyaml<4.0.0,>=3.12 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: six<1.12,>=1.9 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: typing<3.7.0,>=3.6.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: futures<4.0.0,>=3.1.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: google-apitools<=0.5.20,>=0.5.18 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0
in /home/jenkins/.local/lib/python2.7/site-packages (from
apache-beam==2.5.0.dev0)
Requirement already satisfied: googledatastore==7.0.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: google-cloud-pubsub==0.26.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: proto-google-cloud-pubsub-v1==0.15.4 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: google-cloud-bigquery==0.25.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Collecting nose>=1.3.7 (from apache-beam==2.5.0.dev0)
Using cached nose-1.3.7-py2-none-any.whl
Requirement already satisfied: pyhamcrest<2.0,>=1.9 in
/home/jenkins/.local/lib/python2.7/site-packages (from apache-beam==2.5.0.dev0)
Requirement already satisfied: enum34>=1.0.4 in
/home/jenkins/.local/lib/python2.7/site-packages (from
grpcio<2,>=1.8->apache-beam==2.5.0.dev0)
Requirement already satisfied: requests>=2.7.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from
hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: docopt in /usr/local/lib/python2.7/dist-packages
(from hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in
/home/jenkins/.local/lib/python2.7/site-packages (from
mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Requirement already satisfied: pbr>=0.11 in
/home/jenkins/.local/lib/python2.7/site-packages (from
mock<3.0.0,>=1.0.1->apache-beam==2.5.0.dev0)
Requirement already satisfied: pyasn1>=0.1.7 in
/home/jenkins/.local/lib/python2.7/site-packages (from
oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Requirement already satisfied: pyasn1-modules>=0.0.5 in
/home/jenkins/.local/lib/python2.7/site-packages (from
oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Requirement already satisfied: rsa>=3.1.4 in
/home/jenkins/.local/lib/python2.7/site-packages (from
oauth2client<5,>=2.0.1->apache-beam==2.5.0.dev0)
Requirement already satisfied: setuptools in ./.env/lib/python2.7/site-packages
(from protobuf<4,>=3.5.0.post1->apache-beam==2.5.0.dev0)
Requirement already satisfied: fasteners>=0.14 in
/home/jenkins/.local/lib/python2.7/site-packages (from
google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.5.2 in
/home/jenkins/.local/lib/python2.7/site-packages (from
proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: google-cloud-core<0.26dev,>=0.25.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from
google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from
google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: idna<2.6,>=2.5 in
/home/jenkins/.local/lib/python2.7/site-packages (from
requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: urllib3<1.22,>=1.21.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from
requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in
/home/jenkins/.local/lib/python2.7/site-packages (from
requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: certifi>=2017.4.17 in
/home/jenkins/.local/lib/python2.7/site-packages (from
requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: monotonic>=0.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from
fasteners>=0.14->google-apitools<=0.5.20,>=0.5.18->apache-beam==2.5.0.dev0)
Requirement already satisfied: google-auth<2.0.0dev,>=0.4.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from
google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: google-auth-httplib2 in
/home/jenkins/.local/lib/python2.7/site-packages (from
google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: google-gax<0.16dev,>=0.15.7 in
/home/jenkins/.local/lib/python2.7/site-packages (from
gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: grpc-google-iam-v1<0.12dev,>=0.11.1 in
/home/jenkins/.local/lib/python2.7/site-packages (from
gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: cachetools>=2.0.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from
google-auth<2.0.0dev,>=0.4.0->google-cloud-core<0.26dev,>=0.25.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: future<0.17dev,>=0.16.0 in
/home/jenkins/.local/lib/python2.7/site-packages (from
google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Requirement already satisfied: ply==3.8 in
/home/jenkins/.local/lib/python2.7/site-packages (from
google-gax<0.16dev,>=0.15.7->gapic-google-cloud-pubsub-v1<0.16dev,>=0.15.0->google-cloud-pubsub==0.26.0->apache-beam==2.5.0.dev0)
Installing collected packages: grpcio, hdfs, pytz, nose, apache-beam
Found existing installation: grpcio 1.3.5
Not uninstalling grpcio at
/home/jenkins/.local/lib/python2.7/site-packages, outside environment
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/.env>
Found existing installation: apache-beam 2.5.0.dev0
Not uninstalling apache-beam at
/home/jenkins/jenkins-slave/workspace/beam_PerformanceTests_TextIOIT/src/sdks/python,
outside environment
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/.env>
Running setup.py develop for apache-beam
Successfully installed apache-beam grpcio-1.10.1 hdfs-2.1.0 nose-1.3.7
pytz-2018.4
[beam_PerformanceTests_MongoDBIO_IT] $ /bin/bash -xe
/tmp/jenkins6387691400289956956.sh
+ .env/bin/python PerfKitBenchmarker/pkb.py --project=apache-beam-testing
--dpb_log_level=INFO --maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--bigquery_table=beam_performance.mongodbioit_pkb_results
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/>
--official=true
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
--beam_it_timeout=1800 --benchmarks=beam_integration_benchmark
--beam_it_profile=io-it --beam_prebuilt=true --beam_sdk=java
--beam_it_module=sdks/java/io/mongodb
--beam_it_class=org.apache.beam.sdk.io.mongodb.MongoDBIOIT
'--beam_it_options=[--tempRoot=gs://temp-storage-for-perf-tests,--project=apache-beam-testing,--numberOfRecords=10000000]'
--beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/.test-infra/kubernetes/mongodb/load-balancer/mongo.yml>
--beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/.test-infra/kubernetes/mongodb/load-balancer/pkb-config.yml>
2018-04-12 12:00:51,320 805b0cab MainThread INFO Verbose logging to:
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/runs/805b0cab/pkb.log>
2018-04-12 12:00:51,320 805b0cab MainThread INFO PerfKitBenchmarker
version: v1.12.0-485-g4e039b9
2018-04-12 12:00:51,321 805b0cab MainThread INFO Flag values:
--beam_it_class=org.apache.beam.sdk.io.mongodb.MongoDBIOIT
--beam_it_timeout=1800
--beam_sdk=java
--temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/>
--maven_binary=/home/jenkins/tools/maven/latest/bin/mvn
--beam_it_options=[--tempRoot=gs://temp-storage-for-perf-tests,--project=apache-beam-testing,--numberOfRecords=10000000]
--beam_prebuilt
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
--project=apache-beam-testing
--beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/.test-infra/kubernetes/mongodb/load-balancer/mongo.yml>
--bigquery_table=beam_performance.mongodbioit_pkb_results
--official
--beam_it_module=sdks/java/io/mongodb
--dpb_log_level=INFO
--beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/.test-infra/kubernetes/mongodb/load-balancer/pkb-config.yml>
--beam_it_profile=io-it
--benchmarks=beam_integration_benchmark
2018-04-12 12:00:51,625 805b0cab MainThread WARNING The key "flags" was not in
the default config, but was in user overrides. This may indicate a typo.
2018-04-12 12:00:51,626 805b0cab MainThread INFO Initializing the edw
service decoder
2018-04-12 12:00:51,749 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Provisioning resources for benchmark beam_integration_benchmark
2018-04-12 12:00:51,752 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Preparing benchmark beam_integration_benchmark
2018-04-12 12:00:51,752 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: git clone https://github.com/apache/beam.git
2018-04-12 12:01:04,251 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
create -f
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/.test-infra/kubernetes/mongodb/load-balancer/mongo.yml>
2018-04-12 12:01:04,515 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running benchmark beam_integration_benchmark
2018-04-12 12:01:04,520 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:01:14,681 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:01:24,819 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:01:34,956 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:01:45,103 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:01:55,246 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:02:05,408 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:02:15,557 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:02:25,710 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:02:35,840 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:02:45,982 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:02:56,128 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:03:06,254 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:03:16,381 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:03:26,520 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:03:36,645 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:03:46,778 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:03:56,906 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:04:07,065 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
get svc mongo-load-balancer-service
-ojsonpath={.status.loadBalancer.ingress[0].ip}
2018-04-12 12:04:07,185 805b0cab MainThread beam_integration_benchmark(1/1)
ERROR Error during benchmark beam_integration_benchmark
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 623, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
line 148, in Run
dynamic_pipeline_options)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py";,>
line 100, in GenerateAllPipelineOptions
EvaluateDynamicPipelineOptions(dynamic_pipeline_options))
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py";,>
line 67, in EvaluateDynamicPipelineOptions
argValue = RetrieveLoadBalancerIp(optionDescriptor)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py";,>
line 153, in RetrieveLoadBalancerIp
raise "Could not retrieve LoadBalancer IP address"
TypeError: exceptions must be old-style classes or derived from BaseException,
not str
2018-04-12 12:04:07,186 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Cleaning up benchmark beam_integration_benchmark
2018-04-12 12:04:07,187 805b0cab MainThread beam_integration_benchmark(1/1)
INFO Running: kubectl
--kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/config-mongodbioit-1523527322360>
delete -f
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/src/.test-infra/kubernetes/mongodb/load-balancer/mongo.yml>
2018-04-12 12:04:07,490 805b0cab MainThread beam_integration_benchmark(1/1)
ERROR Exception running benchmark
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 757, in RunBenchmarkTask
RunBenchmark(spec, collector)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 623, in RunBenchmark
DoRunPhase(spec, collector, detailed_timer)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,>
line 526, in DoRunPhase
samples = spec.BenchmarkRun(spec)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/linux_benchmarks/beam_integration_benchmark.py";,>
line 148, in Run
dynamic_pipeline_options)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py";,>
line 100, in GenerateAllPipelineOptions
EvaluateDynamicPipelineOptions(dynamic_pipeline_options))
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py";,>
line 67, in EvaluateDynamicPipelineOptions
argValue = RetrieveLoadBalancerIp(optionDescriptor)
File
"<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/PerfKitBenchmarker/perfkitbenchmarker/beam_pipeline_options.py";,>
line 153, in RetrieveLoadBalancerIp
raise "Could not retrieve LoadBalancer IP address"
TypeError: exceptions must be old-style classes or derived from BaseException,
not str
2018-04-12 12:04:07,491 805b0cab MainThread beam_integration_benchmark(1/1)
ERROR Benchmark 1/1 beam_integration_benchmark (UID:
beam_integration_benchmark0) failed. Execution will continue.
2018-04-12 12:04:07,497 805b0cab MainThread INFO Benchmark run statuses:
---------------------------------------------------------------------------------
Name UID Status Failed
Substatus
---------------------------------------------------------------------------------
beam_integration_benchmark beam_integration_benchmark0 FAILED
---------------------------------------------------------------------------------
Success rate: 0.00% (0/1)
2018-04-12 12:04:07,497 805b0cab MainThread INFO Complete logs can be found
at:
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/runs/805b0cab/pkb.log>
2018-04-12 12:04:07,498 805b0cab MainThread INFO Completion statuses can be
found at:
<https://builds.apache.org/job/beam_PerformanceTests_MongoDBIO_IT/ws/runs/805b0cab/completion_statuses.json>
Build step 'Execute shell' marked build as failure
