[
https://issues.apache.org/jira/browse/IMPALA-13551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17906555#comment-17906555
]
ASF subversion and git services commented on IMPALA-13551:
----------------------------------------------------------
Commit aefd1b0920150feff31922a6979affc005a6a7d4 in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=aefd1b092 ]
IMPALA-13551: Produce the shell tarball by pip installing impala-shell
Currently, the shell tarball maintains its own packaging code
and directory layout. This is very complicated and currently has
several Python packages directly checked into our repository.
To simplify it, this changes the shell tarball to be based on
pip installing the pypi package. Specifically, the new directory
structure for an unpack shell tarball is:
impala-shell-4.5.0-SNAPSHOT/
impala-shell
install_py${PYTHON_VERSION}/
install_py${ANOTHER_PYTHON_VERSION}/
For example, install_py2.7 is the Python 2.7 pip install of impala-shell.
install_py3.8 is a Python 3.8 pip install of impala-shell. This means
that the impala-shell script simply picks the install for the
specified version of python and uses that pip install directory.
To make this more consistent across different Linux distributions, this
upgrades pip in the virtualenv to the latest.
With this, ext-py and pkg_resources.py can be removed.
This requires rearranging the shell build code. Specifically, this splits
out the code that generates impala_build_version.py so that it can run
before generating the pypi package. The shell tarball now has a dependency
on the pypi package and must run after it.
This builds on Michael Smith's work from IMPALA-11399.
Testing:
- Ran shell tests locally
- Built on Centos 7, Redhat 8 & 9, Ubuntu 20 & 22, SLES 15
Change-Id: Ifbb66ab2c5bc7180221f98d9bf5e38d62f4ac036
Reviewed-on: http://gerrit.cloudera.org:8080/20171
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Produce impala-shell tarball using pip install
> ----------------------------------------------
>
> Key: IMPALA-13551
> URL: https://issues.apache.org/jira/browse/IMPALA-13551
> Project: IMPALA
> Issue Type: Task
> Components: Clients
> Affects Versions: Impala 4.5.0
> Reporter: Joe McDonnell
> Priority: Major
>
> Currently, the impala-shell tarball is produced by compiling the python
> packages in shell/ext-py into wheels then installing those wheels into
> directories that become part of the shell tarball. There are two problems
> with this. First, we have a lot of dependencies checked into shell/ext-py and
> it is annoying to update those when they change (see IMPALA-11399). Second,
> constructing the shell tarball involves a lot of copying code and
> dependencies into place. The steps are convoluted.
> Since we support pip installing impala-shell, the shell tarball can simply be
> pip installing impala-shell into directories for the supported python
> versions and having the top-level impala-shell script dispatch to the right
> directory for that python version. This would eliminate the shell/ext-py code
> and also make the shell tarball easier to understand.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
