See <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/1630/display/redirect>
------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on apache-beam-jenkins-14 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/> No credentials specified > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision c02af609c49f268366cdecf680621e907700e59b (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f c02af609c49f268366cdecf680621e907700e59b Commit message: "[BEAM-4858] Increase tolerance on linear regression tests (#8556)" > git rev-list --no-walk c02af609c49f268366cdecf680621e907700e59b # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 [EnvInject] - Executing scripts and injecting environment variables after the SCM step. [EnvInject] - Injecting as environment variables the properties content SPARK_LOCAL_IP=127.0.0.1 [EnvInject] - Variables injected successfully. [beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins4678668997963266423.sh + gcloud container clusters get-credentials io-datastores --zone=us-central1-a --verbosity=debug DEBUG: Running [gcloud.container.clusters.get-credentials] with arguments: [--verbosity: "debug", --zone: "us-central1-a", NAME: "io-datastores"] Fetching cluster endpoint and auth data. DEBUG: Saved kubeconfig to /home/jenkins/.kube/config kubeconfig entry generated for io-datastores. INFO: Display format: "default" DEBUG: SDK update checks are disabled. [beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins284794299757244558.sh + cp /home/jenkins/.kube/config <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-beam-performancetests-textioit-hdfs-1630> [beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins325524360985204577.sh + kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-beam-performancetests-textioit-hdfs-1630> create namespace beam-performancetests-textioit-hdfs-1630 namespace/beam-performancetests-textioit-hdfs-1630 created [beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins1035327021764350781.sh ++ kubectl config current-context + kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-beam-performancetests-textioit-hdfs-1630> config set-context gke_apache-beam-testing_us-central1-a_io-datastores --namespace=beam-performancetests-textioit-hdfs-1630 Context "gke_apache-beam-testing_us-central1-a_io-datastores" modified. [beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins3666302684327441381.sh + rm -rf <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker> [beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins4546963357716989368.sh + rm -rf <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/env/.perfkit_env> [beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins4215805513353733504.sh + virtualenv <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/env/.perfkit_env> --python=python2.7 New python executable in <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/env/.perfkit_env/bin/python2.7> Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/env/.perfkit_env/bin/python> Installing setuptools, pip, wheel... done. Running virtualenv with interpreter /usr/bin/python2.7 [beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins325259543312093932.sh + <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/env/.perfkit_env/bin/pip> install --upgrade setuptools pip DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. Requirement already up-to-date: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (41.0.1) Requirement already up-to-date: pip in ./env/.perfkit_env/lib/python2.7/site-packages (19.1.1) [beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins3653487145858907887.sh + git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker> Cloning into '<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker'...> [beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins7929067564543884591.sh + <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/env/.perfkit_env/bin/pip> install -r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. Collecting absl-py (from -r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 14)) Collecting jinja2>=2.7 (from -r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 15)) Using cached https://files.pythonhosted.org/packages/1d/e7/fd8b501e7a6dfe492a433deb7b9d833d39ca74916fa8bc63dd1a4947a671/Jinja2-2.10.1-py2.py3-none-any.whl Requirement already satisfied: setuptools in ./env/.perfkit_env/lib/python2.7/site-packages (from -r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 16)) (41.0.1) Collecting colorlog[windows]==2.6.0 (from -r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 17)) Using cached https://files.pythonhosted.org/packages/59/1a/46a1bf2044ad8b30b52fed0f389338c85747e093fe7f51a567f4cb525892/colorlog-2.6.0-py2.py3-none-any.whl Collecting blinker>=1.3 (from -r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 18)) Collecting futures>=3.0.3 (from -r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 19)) Using cached https://files.pythonhosted.org/packages/2d/99/b2c4e9d5a30f6471e410a146232b4118e697fa3ffc06d6a65efde84debd0/futures-3.2.0-py2-none-any.whl Collecting PyYAML==3.12 (from -r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 20)) Collecting pint>=0.7 (from -r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 21)) Using cached https://files.pythonhosted.org/packages/15/9d/bf177ebbc57d25e9e296addc14a1303d1e34d7964af5df428a8332349c42/Pint-0.9-py2.py3-none-any.whl Collecting numpy==1.13.3 (from -r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 22)) Using cached https://files.pythonhosted.org/packages/eb/be/737f3df5806192ac4096e549e48c8c76cfaa2fb880a1c62a7bb085adaa9b/numpy-1.13.3-cp27-cp27mu-manylinux1_x86_64.whl Collecting functools32 (from -r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 23)) Collecting contextlib2>=0.5.1 (from -r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 24)) Using cached https://files.pythonhosted.org/packages/a2/71/8273a7eeed0aff6a854237ab5453bc9aa67deb49df4832801c21f0ff3782/contextlib2-0.5.5-py2.py3-none-any.whl Collecting pywinrm (from -r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 25)) Using cached https://files.pythonhosted.org/packages/0d/12/13a3117bbd2230043aa32dcfa2198c33269665eaa1a8fa26174ce49b338f/pywinrm-0.3.0-py2.py3-none-any.whl Collecting timeout-decorator (from -r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 26)) Collecting enum34; python_version < "3.4" (from absl-py->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 14)) Using cached https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl Collecting six (from absl-py->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 14)) Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl Collecting MarkupSafe>=0.23 (from jinja2>=2.7->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 15)) Using cached https://files.pythonhosted.org/packages/fb/40/f3adb7cf24a8012813c5edb20329eb22d5d8e2a0ecf73d21d6b85865da11/MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl Collecting colorama; extra == "windows" (from colorlog[windows]==2.6.0->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 17)) Using cached https://files.pythonhosted.org/packages/4f/a6/728666f39bfff1719fc94c481890b2106837da9318031f71a8424b662e12/colorama-0.4.1-py2.py3-none-any.whl Collecting funcsigs; python_version == "2.7" (from pint>=0.7->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 21)) Using cached https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl Collecting requests-ntlm>=0.3.0 (from pywinrm->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 25)) Using cached https://files.pythonhosted.org/packages/03/4b/8b9a1afde8072c4d5710d9fa91433d504325821b038e00237dc8d6d833dc/requests_ntlm-1.1.0-py2.py3-none-any.whl Collecting requests>=2.9.1 (from pywinrm->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 25)) Using cached https://files.pythonhosted.org/packages/7d/e3/20f3d364d6c8e5d2353c72a67778eb189176f08e873c9900e10c0287b84b/requests-2.21.0-py2.py3-none-any.whl Collecting xmltodict (from pywinrm->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 25)) Using cached https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl Collecting cryptography>=1.3 (from requests-ntlm>=0.3.0->pywinrm->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 25)) Using cached https://files.pythonhosted.org/packages/c3/c1/cf8665c955c9393e9ff0872ba6cd3dc6f46ef915e94afcf6e0410508ca69/cryptography-2.6.1-cp27-cp27mu-manylinux1_x86_64.whl Collecting ntlm-auth>=1.0.2 (from requests-ntlm>=0.3.0->pywinrm->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 25)) Using cached https://files.pythonhosted.org/packages/d5/3d/1c54e92f62bbc747a638da94adb439f99dc2d2f3041fe41a06b0da4f2808/ntlm_auth-1.3.0-py2.py3-none-any.whl Collecting certifi>=2017.4.17 (from requests>=2.9.1->pywinrm->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 25)) Using cached https://files.pythonhosted.org/packages/60/75/f692a584e85b7eaba0e03827b3d51f45f571c2e793dd731e598828d380aa/certifi-2019.3.9-py2.py3-none-any.whl Collecting chardet<3.1.0,>=3.0.2 (from requests>=2.9.1->pywinrm->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 25)) Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl Collecting idna<2.9,>=2.5 (from requests>=2.9.1->pywinrm->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 25)) Using cached https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl Collecting urllib3<1.25,>=1.21.1 (from requests>=2.9.1->pywinrm->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 25)) Using cached https://files.pythonhosted.org/packages/01/11/525b02e4acc0c747de8b6ccdab376331597c569c42ea66ab0a1dbd36eca2/urllib3-1.24.3-py2.py3-none-any.whl Collecting cffi!=1.11.3,>=1.8 (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 25)) Using cached https://files.pythonhosted.org/packages/8d/e9/0c8afd1579e5cf7bc0f06fbcd7cdb954cbc0baadd505973949a99337da1c/cffi-1.12.3-cp27-cp27mu-manylinux1_x86_64.whl Collecting asn1crypto>=0.21.0 (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 25)) Using cached https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl Collecting ipaddress; python_version < "3" (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 25)) Using cached https://files.pythonhosted.org/packages/fc/d0/7fc3a811e011d4b388be48a0e381db8d990042df54aa4ef4599a31d39853/ipaddress-1.0.22-py2.py3-none-any.whl Collecting pycparser (from cffi!=1.11.3,>=1.8->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/requirements.txt> (line 25)) Installing collected packages: enum34, six, absl-py, MarkupSafe, jinja2, colorama, colorlog, blinker, futures, PyYAML, funcsigs, pint, numpy, functools32, contextlib2, certifi, chardet, idna, urllib3, requests, pycparser, cffi, asn1crypto, ipaddress, cryptography, ntlm-auth, requests-ntlm, xmltodict, pywinrm, timeout-decorator Successfully installed MarkupSafe-1.1.1 PyYAML-3.12 absl-py-0.7.1 asn1crypto-0.24.0 blinker-1.4 certifi-2019.3.9 cffi-1.12.3 chardet-3.0.4 colorama-0.4.1 colorlog-2.6.0 contextlib2-0.5.5 cryptography-2.6.1 enum34-1.1.6 funcsigs-1.0.2 functools32-3.2.3.post2 futures-3.2.0 idna-2.8 ipaddress-1.0.22 jinja2-2.10.1 ntlm-auth-1.3.0 numpy-1.13.3 pint-0.9 pycparser-2.19 pywinrm-0.3.0 requests-2.21.0 requests-ntlm-1.1.0 six-1.12.0 timeout-decorator-0.4.1 urllib3-1.24.3 xmltodict-0.12.0 [beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins3656871650855071797.sh + <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/env/.perfkit_env/bin/python> <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/PerfKitBenchmarker/pkb.py> --project=apache-beam-testing --dpb_log_level=INFO --bigquery_table=beam_performance.textioit_hdfs_pkb_results --k8s_get_retry_count=36 --k8s_get_wait_interval=10 --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src> --official=true --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-beam-performancetests-textioit-hdfs-1630> --benchmarks=beam_integration_benchmark --beam_it_timeout=1200 --beam_prebuilt=false --beam_sdk=java --beam_it_module=sdks/java/io/file-based-io-tests --beam_it_class=org.apache.beam.sdk.io.text.TextIOIT '--beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--bigQueryDataset=beam_performance,--bigQueryTable=textioit_hdfs_results,--numberOfRecords=1000000]' '--beam_extra_properties=[filesystem=hdfs]' --beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/LargeITCluster/pkb-config.yml> --beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/LargeITCluster/hdfs-multi-datanode-cluster.yml> 2019-05-12 18:18:27,969 00b2679d MainThread INFO Verbose logging to: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/00b2679d/pkb.log> 2019-05-12 18:18:27,969 00b2679d MainThread INFO PerfKitBenchmarker version: v1.12.0-1171-gd5f36c7 2019-05-12 18:18:27,971 00b2679d MainThread INFO Flag values: --beam_it_class=org.apache.beam.sdk.io.text.TextIOIT --beam_it_timeout=1200 --beam_it_module=sdks/java/io/file-based-io-tests --beam_sdk=java --k8s_get_wait_interval=10 --bigquery_table=beam_performance.textioit_hdfs_pkb_results --temp_dir=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/> --beam_it_options=[--project=apache-beam-testing,--tempRoot=gs://temp-storage-for-perf-tests,--bigQueryDataset=beam_performance,--bigQueryTable=textioit_hdfs_results,--numberOfRecords=1000000] --nobeam_prebuilt --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-beam-performancetests-textioit-hdfs-1630> --beam_extra_properties=[filesystem=hdfs] --project=apache-beam-testing --beam_kubernetes_scripts=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/LargeITCluster/hdfs-multi-datanode-cluster.yml> --official --dpb_log_level=INFO --beam_options_config_file=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/LargeITCluster/pkb-config.yml> --beam_location=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src> --k8s_get_retry_count=36 --benchmarks=beam_integration_benchmark 2019-05-12 18:18:28,150 00b2679d MainThread INFO Setting --max_concurrent_threads=200. 2019-05-12 18:18:28,172 00b2679d MainThread WARNING The key "flags" was not in the default config, but was in user overrides. This may indicate a typo. 2019-05-12 18:18:28,197 00b2679d MainThread beam_integration_benchmark(1/1) INFO Provisioning resources for benchmark beam_integration_benchmark 2019-05-12 18:18:28,199 00b2679d MainThread beam_integration_benchmark(1/1) INFO Preparing benchmark beam_integration_benchmark 2019-05-12 18:18:28,200 00b2679d MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/gradlew> clean assemble --stacktrace --info -p sdks/java/io/file-based-io-tests -DintegrationTestRunner=dataflow -Dfilesystem=hdfs 2019-05-12 18:19:48,556 00b2679d MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-beam-performancetests-textioit-hdfs-1630> create -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/LargeITCluster/hdfs-multi-datanode-cluster.yml> 2019-05-12 18:19:48,940 00b2679d MainThread beam_integration_benchmark(1/1) INFO Running benchmark beam_integration_benchmark 2019-05-12 18:19:48,945 00b2679d MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-beam-performancetests-textioit-hdfs-1630> get svc hadoop -ojsonpath={.status.loadBalancer.ingress[0].ip} 2019-05-12 18:19:59,084 00b2679d MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-beam-performancetests-textioit-hdfs-1630> get svc hadoop -ojsonpath={.status.loadBalancer.ingress[0].ip} 2019-05-12 18:20:09,189 00b2679d MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-beam-performancetests-textioit-hdfs-1630> get svc hadoop -ojsonpath={.status.loadBalancer.ingress[0].ip} 2019-05-12 18:20:19,365 00b2679d MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-beam-performancetests-textioit-hdfs-1630> get svc hadoop -ojsonpath={.status.loadBalancer.ingress[0].ip} 2019-05-12 18:20:29,467 00b2679d MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-beam-performancetests-textioit-hdfs-1630> get svc hadoop -ojsonpath={.status.loadBalancer.ingress[0].ip} 2019-05-12 18:20:39,576 00b2679d MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-beam-performancetests-textioit-hdfs-1630> get svc hadoop -ojsonpath={.status.loadBalancer.ingress[0].ip} 2019-05-12 18:20:39,665 00b2679d MainThread beam_integration_benchmark(1/1) INFO Using LoadBalancer IP Address: 35.184.248.207 2019-05-12 18:20:39,665 00b2679d MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-beam-performancetests-textioit-hdfs-1630> get svc hadoop -ojsonpath={.status.loadBalancer.ingress[0].ip} 2019-05-12 18:20:39,753 00b2679d MainThread beam_integration_benchmark(1/1) INFO Using LoadBalancer IP Address: 35.184.248.207 2019-05-12 18:20:39,765 00b2679d MainThread beam_integration_benchmark(1/1) INFO Running: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/gradlew> integrationTest --tests=org.apache.beam.sdk.io.text.TextIOIT -p sdks/java/io/file-based-io-tests -DintegrationTestRunner=dataflow -Dfilesystem=hdfs -DintegrationTestPipelineOptions=["--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--bigQueryDataset=beam_performance","--bigQueryTable=textioit_hdfs_results","--numberOfRecords=1000000","--hdfsConfiguration=[{\"fs.defaultFS\":\"hdfs://35.184.248.207:9000\",\"dfs.replication\":1}]","--filenamePrefix=hdfs://35.184.248.207:9000/TEXTIO_IT_","--runner=TestDataflowRunner"] --stacktrace --info --scan 2019-05-12 18:25:37,103 00b2679d MainThread beam_integration_benchmark(1/1) INFO Cleaning up benchmark beam_integration_benchmark 2019-05-12 18:25:37,104 00b2679d MainThread beam_integration_benchmark(1/1) INFO Running: kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-beam-performancetests-textioit-hdfs-1630> delete -f <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/LargeITCluster/hdfs-multi-datanode-cluster.yml> --ignore-not-found 2019-05-12 18:26:12,143 00b2679d MainThread beam_integration_benchmark(1/1) INFO Tearing down resources for benchmark beam_integration_benchmark 2019-05-12 18:26:12,146 00b2679d MainThread INFO Publishing 2 samples to <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/00b2679d/perfkit-bq-pubwwhcpf.json> 2019-05-12 18:26:12,146 00b2679d MainThread INFO Publishing 2 samples to beam_performance.textioit_hdfs_pkb_results 2019-05-12 18:26:12,147 00b2679d MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.textioit_hdfs_pkb_results <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/00b2679d/perfkit-bq-pubwwhcpf.json> 2019-05-12 18:26:17,393 00b2679d MainThread INFO -------------------------PerfKitBenchmarker Complete Results------------------------- {'metadata': {'dpb_cluster_id': None, 'dpb_cluster_shape': 'n1-standard-1', 'dpb_cluster_size': 2, 'dpb_dataflow_runner': 'DataflowRunner', 'dpb_dataflow_sdk': None, 'dpb_service': 'dataflow', 'perfkitbenchmarker_version': 'v1.12.0-1171-gd5f36c7', 'run_number': 0}, 'metric': 'run_time', 'official': True, 'owner': 'jenkins', 'product_name': 'PerfKitBenchmarker', 'run_uri': '00b2679d-5ce08d5d-4d36-4c20-b075-4f6fc9a77ba9', 'sample_uri': '160fa72e-ecc6-4abc-b4f0-d5f31405bdec', 'test': 'beam_integration_benchmark', 'timestamp': 1557685537.1027, 'unit': 'seconds', 'value': 297.348522} {'metadata': {'perfkitbenchmarker_version': 'v1.12.0-1171-gd5f36c7'}, 'metric': 'End to End Runtime', 'official': True, 'owner': 'jenkins', 'product_name': 'PerfKitBenchmarker', 'run_uri': '00b2679d-5ce08d5d-4d36-4c20-b075-4f6fc9a77ba9', 'sample_uri': '89e1af32-b751-4d71-8fe9-e1b051523246', 'test': 'beam_integration_benchmark', 'timestamp': 1557685572.143943, 'unit': 'seconds', 'value': 463.9466857910156} -------------------------PerfKitBenchmarker Results Summary------------------------- BEAM_INTEGRATION_BENCHMARK: dpb_cluster_shape="n1-standard-1" dpb_cluster_size="2" dpb_dataflow_runner="DataflowRunner" dpb_service="dataflow" run_number="0" run_time 297.348522 seconds End to End Runtime 463.946686 seconds ------------------------- For all tests: dpb_cluster_id="None" dpb_dataflow_sdk="None" perfkitbenchmarker_version="v1.12.0-1171-gd5f36c7" 2019-05-12 18:26:17,394 00b2679d MainThread INFO Publishing 2 samples to <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/00b2679d/perfkitbenchmarker_results.json> 2019-05-12 18:26:17,395 00b2679d MainThread INFO Benchmark run statuses: ------------------------------------------------------------------------------------ Name UID Status Failed Substatus ------------------------------------------------------------------------------------ beam_integration_benchmark beam_integration_benchmark0 SUCCEEDED ------------------------------------------------------------------------------------ Success rate: 100.00% (1/1) 2019-05-12 18:26:17,395 00b2679d MainThread INFO Complete logs can be found at: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/00b2679d/pkb.log> 2019-05-12 18:26:17,395 00b2679d MainThread INFO Completion statuses can be found at: <https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/runs/00b2679d/completion_statuses.json> [beam_PerformanceTests_TextIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins8864708523972409839.sh + kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_TextIOIT_HDFS/ws/config-beam-performancetests-textioit-hdfs-1630> delete namespace beam-performancetests-textioit-hdfs-1630 namespace "beam-performancetests-textioit-hdfs-1630" deleted Error from server (InternalError): an error on the server ("Internal Server Error: \"/api/v1/namespaces?fieldSelector=metadata.name%3Dbeam-performancetests-textioit-hdfs-1630\": the server is currently unable to handle the request") has prevented the request from succeeding Build step 'Execute shell' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
