Joe McDonnell created IMPALA-11251:
--------------------------------------
Summary: TestImpalaShellInteractive.test_unicode_input fails on
Ubuntu 20
Key: IMPALA-11251
URL: https://issues.apache.org/jira/browse/IMPALA-11251
Project: IMPALA
Issue Type: Bug
Components: Infrastructure
Affects Versions: Impala 4.1.0
Reporter: Joe McDonnell
Running TestImpalaShellInteractive.test_unicode_input() on Ubuntu results in
this failure:
{noformat}
shell/test_shell_interactive.py:377: in test_unicode_input
child_proc.expect(PROMPT_REGEX)
../infra/python/env-gcc7.5.0/lib/python2.7/site-packages/pexpect/__init__.py:1451:
in expect
timeout, searchwindowsize)
../infra/python/env-gcc7.5.0/lib/python2.7/site-packages/pexpect/__init__.py:1466:
in expect_list
timeout, searchwindowsize)
../infra/python/env-gcc7.5.0/lib/python2.7/site-packages/pexpect/__init__.py:1554:
in expect_loop
raise EOF(str(err) + '\n' + str(self))
E EOF: End Of File (EOF). Exception style platform.
E <pexpect.spawn object at 0x7fe089a9e890>
E version: 3.3
E command:
/home/impdev/Impala/shell/build/impala-shell-4.1.0-SNAPSHOT/impala-shell
E args:
['/home/impdev/Impala/shell/build/impala-shell-4.1.0-SNAPSHOT/impala-shell',
'--protocol=beeswax', '-ilocalhost:21000']
E searcher: <pexpect.searcher_re object at 0x7fe089a9ec10>
E buffer (last 100 chars): ''
E before (last 100 chars): ' (required by
/home/impdev/.python-eggs/sasl-0.2.1-py2.7-linux-x86_64.egg-tmp/sasl/saslwrapper.so)\r\n'
E after: <class 'pexpect.EOF'>
E match: None
E match_index: None
E exitstatus: None
E flag_eof: True
E pid: 1109550
E child_fd: 23
E closed: False
E timeout: 30
E delimiter: <class 'pexpect.EOF'>
E logfile: None
E logfile_read: None
E logfile_send: None
E maxread: 2000
E ignorecase: False
E searchwindowsize: None
E delaybeforesend: 0.05
E delayafterclose: 0.1
E delayafterterminate: 0.1{noformat}
What seems to be happening is that impala-shell is built against the system
compiler / system libstdc++ (GCC 9). If we then run it with LD_LIBRARY_PATH set
to point to toolchain compiler / libstdc++, then it won't find the symbols it
needs to run. The solution is to use shell/util.py's spawn_shell function,
which cleans up the environment for the pexpect:
{noformat}
def build_shell_env(env=None):
""" Construct the environment for the shell to run in based on 'env', or the
current
process's environment if env is None."""
if not env: env = os.environ
# Don't inherit PYTHONPATH or LD_LIBRARY_PATH - the shell launch script must
set
# these to include dependencies. Copy 'env' to avoid mutating argument or
os.environ.
env = dict(env)
if "PYTHONPATH" in env:
del env["PYTHONPATH"]
if "LD_LIBRARY_PATH" in env:
del env["LD_LIBRARY_PATH"]
return env
...
def spawn_shell(shell_cmd):
"""Spawn a shell process with the provided command line. Returns the Pexpect
object."""
return pexpect.spawn(shell_cmd[0], shell_cmd[1:],
env=build_shell_env()){noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
