Hello Laszlo Gaal, Tim Armstrong, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/16960
to look at the new patch set (#2).
Change subject: IMPALA-10434: Fix impala-shell's unicode regressions on Python2
......................................................................
IMPALA-10434: Fix impala-shell's unicode regressions on Python2
To make impala-shell compatible for Python3, we explicitly distinguish
bytes and text in Python2 by decoding the bytes for all inputs.
Regression 1: multiple queries in one line with unicode chars will break
In precmd() of impala-shell, if there are multiple queries present in
one input line, we split it into individual queries (by
sqlparse.split()) and append them back to the 'cmdqueue'. They will be
passed to precmd() again. In our Python2 implementation, precmd()
expects them to be str type, and will decode them into unicode type.
However, the output type of sqlparse.split() is unicode which doesn't
have a decode() method. Calling decode() on a unicode var will let
Python2 implicitly encode it to str. This may cause UnicodeEncodeError
since implicitly encoding use 'ascii'.
Regression 2: multi-line query with unicode chars will break when
command history is enabled
In _check_for_command_completion(), when calling
readline.replace_history_item in Python2. We encode the completed_cmd
into bytes. However, we shouldn't replace it since the return type is
expected to be unicode.
Tests:
- Add tests for these two regressions in Python2.
Change-Id: Icc4a8d31311a5c59e5fc0e65fe09f770df41bea4
---
M shell/impala_shell.py
M tests/shell/test_shell_interactive.py
2 files changed, 30 insertions(+), 7 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/16960/2
--
To view, visit http://gerrit.cloudera.org:8080/16960
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Icc4a8d31311a5c59e5fc0e65fe09f770df41bea4
Gerrit-Change-Number: 16960
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Laszlo Gaal <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Tim Armstrong <[email protected]>
