[
https://issues.apache.org/jira/browse/IMPALA-11980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17944795#comment-17944795
]
ASF subversion and git services commented on IMPALA-11980:
----------------------------------------------------------
Commit c5a0ec8bdffd958984c0f6d34e0fcb8acc138457 in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=c5a0ec8bd ]
IMPALA-11980 (part 1): Put all thrift-generated python code into the
impala_thrift_gen package
This puts all of the thrift-generated python code into the
impala_thrift_gen package. This is similar to what Impyla
does for its thrift-generated python code, except that it
uses the impala_thrift_gen package rather than impala._thrift_gen.
This is a preparatory patch for fixing the absolute import
issues.
This patches all of the thrift files to add the python namespace.
This has code to apply the patching to the thirdparty thrift
files (hive_metastore.thrift, fb303.thrift) to do the same.
Putting all the generated python into a package makes it easier
to understand where the imports are getting code. When the
subsequent change rearranges the shell code, the thrift generated
code can stay in a separate directory.
This uses isort to sort the imports for the affected Python files
with the provided .isort.cfg file. This also adds an impala-isort
shell script to make it easy to run.
Testing:
- Ran a core job
Change-Id: Ie2927f22c7257aa38a78084efe5bd76d566493c0
Reviewed-on: http://gerrit.cloudera.org:8080/20169
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Riza Suminto <[email protected]>
> Impala-shell should switch to absolute imports
> ----------------------------------------------
>
> Key: IMPALA-11980
> URL: https://issues.apache.org/jira/browse/IMPALA-11980
> Project: IMPALA
> Issue Type: Sub-task
> Components: Clients
> Affects Versions: Impala 4.3.0
> Reporter: Joe McDonnell
> Priority: Major
>
> Various tests import code from the shell codebase:
> {noformat}
> $ git grep 'from shell\..*import'
> custom_cluster/test_hs2_fault_injection.py:from shell.ImpalaHttpClient import
> ImpalaHttpClient
> custom_cluster/test_hs2_fault_injection.py:from shell.impala_client import
> ImpalaHS2Client
> custom_cluster/test_hs2_fault_injection.py:from shell.shell_exceptions import
> HttpError
> custom_cluster/test_shell_interactive_reconnect.py:from shell.impala_shell
> import ImpalaShell as ImpalaShellClass
> shell/test_cookie_util.py:from shell.cookie_util import (cookie_matches_path,
> get_cookie_expiry,
> shell/test_shell_client.py:from shell.impala_client import
> ImpalaBeeswaxClient, ImpalaHS2Client
> shell/test_shell_commandline.py:from shell.impala_shell import ImpalaShell as
> ImpalaShellClass
> shell/test_shell_interactive.py:from shell.impala_shell import ImpalaShell as
> ImpalaShellClass{noformat}
> The shell codebase currently uses relative imports, so when using Python 3,
> the imports within the shell codebase will not work. For example,
> impala_shell.py includes compatibility.py via a relative import:
> {noformat}
> from compatibility import _xrange as xrange{noformat}
> In order for this to work under Python 3, these relative imports will need to
> be absolute. However, some ways of invoking the shell will fail with absolute
> imports (e.g. bin/impala-shell.sh). Other form factors are fine with absolute
> imports (e.g. pip installs into a venv).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
