This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
The following commit(s) were added to refs/heads/master by this push:
new ede22a6 IMPALA-10608 followup: Detect the virtualenv tarball version
ede22a6 is described below
commit ede22a63a5acec36031dfeddb86691821c24d40c
Author: Joe McDonnell <joemcdonn...@cloudera.com>
AuthorDate: Wed Mar 31 08:38:29 2021 -0700
IMPALA-10608 followup: Detect the virtualenv tarball version
When rebasing from an older commit, the version change
in virtualenv can cause there to be multiple virtualenv
tarballs of different versions in the infra/python/deps
directory. bootstrap_virtualenv.py currently doesn't
handle this gracefully, because it is looking for all
virtualenv*.tar.gz files and fails when it finds more
than one.
This changes bootstrap_virtualenv.py to get the virtualenv
version from the requirements.txt file and only look
for the tarball with that version. If it fails to get
the version, it falls back to the old method.
Testing:
- Copied virtualenv-16.7.10.tar.gz to virtualenv-16.7.9.tar.gz
and verified that bootstrap_virtualenv.py works
Change-Id: Iebfa9ba5e223d5187414e02e24f34562418fae40
Reviewed-on: http://gerrit.cloudera.org:8080/17249
Reviewed-by: Joe McDonnell <joemcdonn...@cloudera.com>
Tested-by: Joe McDonnell <joemcdonn...@cloudera.com>
---
infra/python/bootstrap_virtualenv.py | 26 +++++++++++++++++++++++++-
1 file changed, 25 insertions(+), 1 deletion(-)
diff --git a/infra/python/bootstrap_virtualenv.py
b/infra/python/bootstrap_virtualenv.py
index 7f7e6fc..36aee7e 100644
--- a/infra/python/bootstrap_virtualenv.py
+++ b/infra/python/bootstrap_virtualenv.py
@@ -80,10 +80,34 @@ def delete_virtualenv_if_exist():
shutil.rmtree(ENV_DIR)
+def detect_virtualenv_version():
+ with open(REQS_PATH, "r") as reqs_file:
+ for line in reqs_file:
+ line = line.strip()
+ # Ignore blank lines and comments
+ if len(line) == 0 or line[0] == '#':
+ continue
+ if line.find("virtualenv") != -1 and line.find("==") != -1:
+ packagestring, version = [a.strip() for a in line.split("==")]
+ if packagestring == "virtualenv":
+ LOG.debug("Detected virtualenv version {0}".format(version))
+ return version
+ # If the parsing didn't work, don't raise an exception.
+ return None
+
+
def create_virtualenv():
LOG.info("Creating python virtualenv")
build_dir = tempfile.mkdtemp()
- file = tarfile.open(find_file(DEPS_DIR, "virtualenv*.tar.gz"), "r:gz")
+ # Try to find the virtualenv version by parsing the requirements file
+ # Default to "*" if we can't figure it out.
+ virtualenv_version = detect_virtualenv_version()
+ if virtualenv_version is None:
+ virtualenv_version = "*"
+ # Open the virtualenv tarball
+ virtualenv_tarball = \
+ find_file(DEPS_DIR, "virtualenv-{0}.tar.gz".format(virtualenv_version))
+ file = tarfile.open(virtualenv_tarball, "r:gz")
for member in file.getmembers():
file.extract(member, build_dir)
file.close()
