See <https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/23/display/redirect?page=changes>
Changes: [rmannibucau] better error message when there are conflicting anonymous names [rmannibucau] adding comments to make the code more obvious and extracting the [rmannibucau] adding tostring in ParDo.*Output and removing unstableNames variable [daniel.o.programmer] [BEAM-3126] Creating flatten operation for bundle processor. [rmannibucau] adressing comments [dariusz.aniszewski] use VirtualEnv for PerfKit jobs [lcwik] [BEAM-3125] Creating flatten runner, unfinished. [lcwik] [BEAM-3125] Creating flatten runner [lcwik] [BEAM-3125] Fixing little nitpicks and typos. [lcwik] [BEAM-3126] Fixes for flatten runner. ------------------------------------------ Started by timer [EnvInject] - Loading node environment variables. Building remotely on beam2 (beam) in workspace <https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/> > git rev-parse --is-inside-work-tree # timeout=10 Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 70a520ce9c0930742cf44efad6dbad1a5df29734 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 70a520ce9c0930742cf44efad6dbad1a5df29734 Commit message: "Merge pull request #4408 from youngoli/beam3126" > git rev-list da5020081bd474188ced5a00d064b2e0e12e1c86 # timeout=10 Cleaning workspace > git rev-parse --verify HEAD # timeout=10 Resetting working tree > git reset --hard # timeout=10 > git clean -fdx # timeout=10 [EnvInject] - Executing scripts and injecting environment variables after the SCM step. [EnvInject] - Injecting as environment variables the properties content SPARK_LOCAL_IP=127.0.0.1 [EnvInject] - Variables injected successfully. [beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe /tmp/jenkins5270319776418455454.sh + rm -rf PerfKitBenchmarker [beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe /tmp/jenkins4966490323848825866.sh + git clone https://github.com/GoogleCloudPlatform/PerfKitBenchmarker.git Cloning into 'PerfKitBenchmarker'... [beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe /tmp/jenkins6260894264123370180.sh + pip install --user -r PerfKitBenchmarker/requirements.txt Requirement already satisfied: absl-py in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 14)) Requirement already satisfied: jinja2>=2.7 in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 15)) Requirement already satisfied: setuptools in /usr/local/lib/python2.7/dist-packages (from -r PerfKitBenchmarker/requirements.txt (line 16)) Requirement already satisfied: colorlog[windows]==2.6.0 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 17)) Requirement already satisfied: blinker>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 18)) Requirement already satisfied: futures>=3.0.3 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 19)) Requirement already satisfied: PyYAML==3.12 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 20)) Requirement already satisfied: pint>=0.7 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 21)) Requirement already satisfied: numpy in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 22)) Requirement already satisfied: functools32 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 23)) Requirement already satisfied: contextlib2>=0.5.1 in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 24)) Requirement already satisfied: pywinrm in /home/jenkins/.local/lib/python2.7/site-packages (from -r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: six in /home/jenkins/.local/lib/python2.7/site-packages (from absl-py->-r PerfKitBenchmarker/requirements.txt (line 14)) Requirement already satisfied: MarkupSafe in /usr/local/lib/python2.7/dist-packages (from jinja2>=2.7->-r PerfKitBenchmarker/requirements.txt (line 15)) Requirement already satisfied: colorama; extra == "windows" in /usr/lib/python2.7/dist-packages (from colorlog[windows]==2.6.0->-r PerfKitBenchmarker/requirements.txt (line 17)) Requirement already satisfied: xmltodict in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: requests-ntlm>=0.3.0 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: requests>=2.9.1 in /home/jenkins/.local/lib/python2.7/site-packages (from pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: ntlm-auth>=1.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: cryptography>=1.3 in /home/jenkins/.local/lib/python2.7/site-packages (from requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: idna<2.6,>=2.5 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: urllib3<1.22,>=1.21.1 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: chardet<3.1.0,>=3.0.2 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: certifi>=2017.4.17 in /home/jenkins/.local/lib/python2.7/site-packages (from requests>=2.9.1->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: cffi>=1.7; platform_python_implementation != "PyPy" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: enum34; python_version < "3" in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: asn1crypto>=0.21.0 in /home/jenkins/.local/lib/python2.7/site-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: ipaddress; python_version < "3" in /usr/local/lib/python2.7/dist-packages (from cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) Requirement already satisfied: pycparser in /home/jenkins/.local/lib/python2.7/site-packages (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=1.3->requests-ntlm>=0.3.0->pywinrm->-r PerfKitBenchmarker/requirements.txt (line 25)) [beam_PerformanceTests_TFRecordIOIT] $ /bin/bash -xe /tmp/jenkins9109047121753739243.sh + pip install --user -e 'src/sdks/python/[gcp,test]' Obtaining file://<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/python> Complete output from command python setup.py egg_info: Traceback (most recent call last): File "<string>", line 1, in <module> File "<https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/python/setup.py";,> line 73, in <module> _CYTHON_VERSION = get_distribution('cython').version File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 564, in get_distribution dist = get_provider(dist) File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 436, in get_provider return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0] File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 984, in require needed = self.resolve(parse_requirements(requirements)) File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 857, in resolve env = Environment(self.entries) File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1068, in __init__ self.scan(search_path) File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1097, in scan for dist in find_distributions(item): File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2048, in find_on_path for dist in factory(fullpath): File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2144, in resolve_egg_link return next(dist_groups, ()) File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2140, in <genexpr> os.path.join(os.path.dirname(path), ref) File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2126, in non_empty_lines with open(path) as f: IOError: [Errno 2] No such file or directory: '/home/jenkins/.local/lib/python2.7/site-packages/apache-beam.egg-link' ---------------------------------------- Command "python setup.py egg_info" failed with error code 1 in <https://builds.apache.org/job/beam_PerformanceTests_TFRecordIOIT/ws/src/sdks/python/> Build step 'Execute shell' marked build as failure
