[
https://issues.apache.org/jira/browse/IMPALA-12434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17766644#comment-17766644
]
ASF subversion and git services commented on IMPALA-12434:
----------------------------------------------------------
Commit 1a1a84ee23b85e3dc097c7e64d86418c7f289c90 in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=1a1a84ee2 ]
IMPALA-12434: Isolate pkg_resources.py to its own directory
In some build environments, the impala-shell Python 3
virtualenv install fails due to interactions with
shell/pkg_resources.py. This doesn't reproduce in the standard
development environment, but it is consistent. It seems to
be related to invoking a command in ${IMPALA_HOME}/shell
and the pkg_resources.py being in that directory.
To avoid any interactions, this moves shell/pkg_resources.py
to shell/legacy/pkg_resources.py. This keeps it off of the
path for the failing command, and it also keeps it off of
our PYTHONPATH (which includes ${IMPALA_HOME}/shell).
Testing:
- Ran a build in the affected build environment
- Ran a core job
Change-Id: Id8f2d8a8472c7bb405bf88673ed9779e23cde1d6
Reviewed-on: http://gerrit.cloudera.org:8080/20468
Reviewed-by: Michael Smith <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Isolate shell/pkg_resources.py to keep it off of the PYTHONPATH
> ---------------------------------------------------------------
>
> Key: IMPALA-12434
> URL: https://issues.apache.org/jira/browse/IMPALA-12434
> Project: IMPALA
> Issue Type: Task
> Components: Infrastructure
> Affects Versions: Impala 4.3.0
> Reporter: Joe McDonnell
> Priority: Major
>
> In some internal builds on Redhat 8, the pip install of impala-shell into the
> Python 3 virtualenv can fail with this error:
> {noformat}
> Exception:
> Traceback (most recent call last):
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/basecommand.py",
> line 215, in main
> status = self.run(options, args)
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/commands/install.py",
> line 357, in run
> wb.build(autobuilding=True)
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/wheel.py",
> line 753, in build
> self.requirement_set.prepare_files(self.finder)
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/req/req_set.py",
> line 381, in prepare_files
> ignore_dependencies=self.ignore_dependencies))
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/req/req_set.py",
> line 637, in _prepare_file
> abstract_dist.prep_for_dist()
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/req/req_set.py",
> line 130, in prep_for_dist
> self.req_to_install.run_egg_info()
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/req/req_install.py",
> line 417, in run_egg_info
> self.setup_py, self.link,
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/pip/req/req_install.py",
> line 387, in setup_py
> import setuptools # noqa
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/setuptools/__init__.py",
> line 12, in <module>
> import setuptools.version
> File
> "impala/shell/build/py3_venv/lib64/python3.6/site-packages/setuptools/version.py",
> line 1, in <module>
> import pkg_resources
> File "impala/shell/pkg_resources.py", line 50
> def _bypass_ensure_directory(name, mode=0777):
> ^
> SyntaxError: invalid token
> make[3]: *** [shell/CMakeFiles/shell_python3_install.dir/build.make:70:
> shell/CMakeFiles/shell_python3_install] Error 2
> {noformat}
> What seems to happen is shell/pkg_resources.py is on the PYTHONPATH and our
> pkg_resources.py doesn't work with Python 3. If I move pkg_resources.py to a
> different directory (like a legacy subdirectory) or if I add a
> WORKING_DIRECTORY for the CMake command, it works fine. In general, keeping
> pkg_resources.py away from our PYTHONPATH (which includes $IMPALA_HOME/shell)
> seems like a good idea.
> This problem doesn't reproduce on the regular upstream Impala build or on my
> local Ubuntu 20 workstation build, but it reliably reproduces on an internal
> build.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
