David Knupp has posted comments on this change.
Change subject: IMPALA-5181: Extract PYPI metadata from a webpage
......................................................................
Patch Set 1:
There may well be some historical context that I'm missing here, or just some
complication that I'm overlooking, so sincere apologies if these comments are
not relevant, but one thing I've noticed is that even though we create a python
virtualenv, instead of ever activating it, we use wrapper scripts to construct
elaborate paths to the enclosed binaries (not just Impala does this, but other
Cloudera teams do this too.)
E.g., if I'm in my system evironment:
dknupp@dknupp-desktop:0:~ $ which python pip virtualenv
/usr/bin/python
/usr/local/bin/pip
/usr/local/bin/virtualenv
dknupp@dknupp-desktop:0:~ $ pip --version
pip 8.1.2 from /usr/local/lib/python2.7/dist-packages (python 2.7)
dknupp@dknupp-desktop:0:~ $ virtualenv --version
15.0.2
But if I _activate_ the Impala python virtualenv first...
dknupp@dknupp-desktop:0:~ $ source
${IMPALA_HOME}/infra/python/env/bin/activate
(env)dknupp@dknupp-desktop:0:~ $ which python pip virtualenv
/home/dknupp/Impala/infra/python/env/bin/python
/home/dknupp/Impala/infra/python/env/bin/pip
/home/dknupp/Impala/infra/python/env/bin/virtualenv
...the system versions no longer matter, and I can upgrade these at will.
(env)dknupp@dknupp-desktop:0:~ $ pip install -U pip virtualenv
Collecting pip
Using cached pip-9.0.1-py2.py3-none-any.whl
Collecting virtualenv
Downloading virtualenv-15.1.0-py2.py3-none-any.whl (1.8MB)
100% |################################| 1.8MB 1.0MB/s
Installing collected packages: pip, virtualenv
Found existing installation: pip 8.1.2
Uninstalling pip-8.1.2:
Successfully uninstalled pip-8.1.2
Found existing installation: virtualenv 13.1.0
Uninstalling virtualenv-13.1.0:
Successfully uninstalled virtualenv-13.1.0
Successfully installed pip-9.0.1 virtualenv-15.1.0
(env)dknupp@dknupp-desktop:0:~ $ pip --version
pip 9.0.1 from
/home/dknupp/Impala/infra/python/env/local/lib/python2.7/site-packages (python
2.7)
(env)dknupp@dknupp-desktop:0:~ $ virtualenv --version
15.1.0
When I deactivate the virtualenv, the system versions are untouched.
dknupp@dknupp-desktop:0:~ $ pip --version
pip 8.1.2 from /usr/local/lib/python2.7/dist-packages (python 2.7)
dknupp@dknupp-desktop:0:~ $ virtualenv --version
15.0.2
So a possible workflow would be to:
1. Create the Impala virtualenv in ${IMPALA_HOME}/infra/python/env first
2. Activate the virtualenv
3. "pip install -U pip" to upgrade ${IMPALA_HOME}/infra/python/env/bin/pip
4. Using the upgraded pip to download & install the various things
5. Deactivate the virtualenv
--
To view, visit http://gerrit.cloudera.org:8080/6579
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: comment
Gerrit-Change-Id: If3845a0d5f568d4352e3cc4883596736974fd7de
Gerrit-PatchSet: 1
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Taras Bobrovytsky <[email protected]>
Gerrit-Reviewer: Alex Behm <[email protected]>
Gerrit-Reviewer: David Knupp <[email protected]>
Gerrit-Reviewer: Lars Volker <[email protected]>
Gerrit-Reviewer: Taras Bobrovytsky <[email protected]>
Gerrit-Reviewer: Tim Armstrong <[email protected]>
Gerrit-HasComments: No
