See <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/95/display/redirect?page=changes>
Changes: [tgroh] Add OutputDeduplicator [tgroh] Update ExecutableStageMatcher to accept Matchers [tgroh] Use OutputDeduplicator in GreedyPipelineFuser [swegner] Upgrade Gradle to 4.7 [swegner] Upgrade Gradle protobuf plugin to latest. [tgroh] fixup! Add OutputDeduplicator ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on beam12 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision ac7bfb86ed241b46dbf5d249ea4464478f3fb180 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f ac7bfb86ed241b46dbf5d249ea4464478f3fb180 Commit message: "Merge pull request #4977: Deduplicate Unzipped Flattens after Pipeline Fusion" > git rev-list --no-walk bbabb4202cdbe234ab7be9a1cda57f0e8e226357 # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 [EnvInject] - Executing scripts and injecting environment variables after the SCM step. [EnvInject] - Injecting as environment variables the properties content SPARK_LOCAL_IP=127.0.0.1 [EnvInject] - Variables injected successfully. [beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins1825323276220131695.sh + gcloud container clusters get-credentials io-datastores --zone=us-central1-a --verbosity=debug DEBUG: Running [gcloud.container.clusters.get-credentials] with arguments: [--verbosity: "debug", --zone: "us-central1-a", NAME: "io-datastores"] Fetching cluster endpoint and auth data. DEBUG: Saved kubeconfig to /home/jenkins/.kube/config kubeconfig entry generated for io-datastores. INFO: Display format "default". DEBUG: SDK update checks are disabled. [beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins7234801034451994732.sh + cp /home/jenkins/.kube/config <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-filebasedioithdfs-1524718941628> [beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins7817746783588052647.sh + kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-filebasedioithdfs-1524718941628> create namespace filebasedioithdfs-1524718941628 namespace "filebasedioithdfs-1524718941628" created [beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins978264145741401057.sh ++ kubectl config current-context + kubectl --kubeconfig=<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-filebasedioithdfs-1524718941628> config set-context gke_apache-beam-testing_us-central1-a_io-datastores --namespace=filebasedioithdfs-1524718941628 Context "gke_apache-beam-testing_us-central1-a_io-datastores" modified. [beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins8934393002983009651.sh + rm -rf PerfKitBenchmarker [beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins8417668267297113899.sh + rm -rf .env [beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins8696566115428540475.sh + virtualenv .env --system-site-packages New python executable in <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/.env/bin/python2> Also creating executable in <https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/.env/bin/python> Installing setuptools, pkg_resources, pip, wheel... Complete output from command /home/jenkins/jenkin...DFS/.env/bin/python2 - setuptools pkg_resources pip wheel: Collecting setuptools Using cached https://files.pythonhosted.org/packages/20/d7/04a0b689d3035143e2ff288f4b9ee4bf6ed80585cc121c90bfd85a1a8c2e/setuptools-39.0.1-py2.py3-none-any.whl Collecting pkg_resources Collecting pip Using cached https://files.pythonhosted.org/packages/0f/74/ecd13431bcc456ed390b44c8a6e917c1820365cbebcb6a8974d1cd045ab4/pip-10.0.1-py2.py3-none-any.whl Collecting wheel Exception: Traceback (most recent call last): File "/usr/share/python-wheels/pip-8.1.1-py2.py3-none-any.whl/pip/basecommand.py", line 209, in main status = self.run(options, args) File "/usr/share/python-wheels/pip-8.1.1-py2.py3-none-any.whl/pip/commands/install.py", line 328, in run wb.build(autobuilding=True) File "/usr/share/python-wheels/pip-8.1.1-py2.py3-none-any.whl/pip/wheel.py", line 748, in build self.requirement_set.prepare_files(self.finder) File "/usr/share/python-wheels/pip-8.1.1-py2.py3-none-any.whl/pip/req/req_set.py", line 360, in prepare_files ignore_dependencies=self.ignore_dependencies)) File "/usr/share/python-wheels/pip-8.1.1-py2.py3-none-any.whl/pip/req/req_set.py", line 512, in _prepare_file finder, self.upgrade, require_hashes) File "/usr/share/python-wheels/pip-8.1.1-py2.py3-none-any.whl/pip/req/req_install.py", line 273, in populate_link self.link = finder.find_requirement(self, upgrade) File "/usr/share/python-wheels/pip-8.1.1-py2.py3-none-any.whl/pip/index.py", line 442, in find_requirement all_candidates = self.find_all_candidates(req.name) File "/usr/share/python-wheels/pip-8.1.1-py2.py3-none-any.whl/pip/index.py", line 400, in find_all_candidates for page in self._get_pages(url_locations, project_name): File "/usr/share/python-wheels/pip-8.1.1-py2.py3-none-any.whl/pip/index.py", line 545, in _get_pages page = self._get_page(location) File "/usr/share/python-wheels/pip-8.1.1-py2.py3-none-any.whl/pip/index.py", line 648, in _get_page return HTMLPage.get_page(link, session=self.session) File "/usr/share/python-wheels/pip-8.1.1-py2.py3-none-any.whl/pip/index.py", line 757, in get_page "Cache-Control": "max-age=600", File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/.env/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py";,> line 480, in get return self.request('GET', url, **kwargs) File "/usr/share/python-wheels/pip-8.1.1-py2.py3-none-any.whl/pip/download.py", line 378, in request return super(PipSession, self).request(method, url, *args, **kwargs) File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/.env/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py";,> line 468, in request resp = self.send(prep, **send_kwargs) File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/.env/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py";,> line 597, in send history = [resp for resp in gen] if allow_redirects else [] File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/.env/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py";,> line 195, in resolve_redirects **adapter_kwargs File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/.env/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/sessions.py";,> line 576, in send r = adapter.send(request, **kwargs) File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/.env/share/python-wheels/CacheControl-0.11.5-py2.py3-none-any.whl/cachecontrol/adapter.py";,> line 46, in send resp = super(CacheControlAdapter, self).send(request, **kw) File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/.env/share/python-wheels/requests-2.9.1-py2.py3-none-any.whl/requests/adapters.py";,> line 376, in send timeout=timeout File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/.env/share/python-wheels/urllib3-1.13.1-py2.py3-none-any.whl/urllib3/connectionpool.py";,> line 610, in urlopen _stacktrace=sys.exc_info()[2]) File "<https://builds.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/.env/share/python-wheels/urllib3-1.13.1-py2.py3-none-any.whl/urllib3/util/retry.py";,> line 228, in increment total -= 1 TypeError: unsupported operand type(s) for -=: 'Retry' and 'int' ---------------------------------------- ...Installing setuptools, pkg_resources, pip, wheel...done. Traceback (most recent call last): File "/usr/lib/python3/dist-packages/virtualenv.py", line 2363, in <module> main() File "/usr/lib/python3/dist-packages/virtualenv.py", line 719, in main symlink=options.symlink) File "/usr/lib/python3/dist-packages/virtualenv.py", line 988, in create_environment download=download, File "/usr/lib/python3/dist-packages/virtualenv.py", line 918, in install_wheel call_subprocess(cmd, show_stdout=False, extra_env=env, stdin=SCRIPT) File "/usr/lib/python3/dist-packages/virtualenv.py", line 812, in call_subprocess % (cmd_desc, proc.returncode)) OSError: Command /home/jenkins/jenkin...DFS/.env/bin/python2 - setuptools pkg_resources pip wheel failed with error code 2 Running virtualenv with interpreter /usr/bin/python2 Build step 'Execute shell' marked build as failure
