[
https://issues.apache.org/jira/browse/IMPALA-15042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18088161#comment-18088161
]
ASF subversion and git services commented on IMPALA-15042:
----------------------------------------------------------
Commit 892b756dcc6c8caade1ecbf43a4eead5864359b0 in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=892b756dc ]
IMPALA-15042: Use a clean environment for single_node_perf_run.py's subcommands
bin/single_node_perf_run.py runs complicated things including
a whole build using Python subprocess. This is currently inheriting
environment variables, which can cause complicated interactions.
For example, single_node_perf_run.py uses impala-python3, which
sets LD_LIBRARY_PATH to point to the toolchain libstdc++. For
newer platforms (e.g. Ubuntu 22), this can cause issues for system
binaries like ccache that need a newer libstdc++. This can also
cause issues with stuck variables when switching back and forth
between the A and B git branches and sourcing bin/impala-config.sh.
This changes bin/single_node_perf_run.py to use a clean environment
for most subprocess invocations. This is a change in behavior as
it previously respected the variables in the environment. Users can
achieve the same impact by putting their environment variables in
bin/impala-config-local.sh.
Testing:
- Ran a perf-AB-test job with Ubuntu 22 (which was previously
failing)
Change-Id: I984b094be73b185ab151d5d1ba51602b7083d22a
Reviewed-on: http://gerrit.cloudera.org:8080/24430
Reviewed-by: Michael Smith <[email protected]>
Tested-by: Joe McDonnell <[email protected]>
> single_node_perf_run.py should execute subcommands with a clean environment
> ---------------------------------------------------------------------------
>
> Key: IMPALA-15042
> URL: https://issues.apache.org/jira/browse/IMPALA-15042
> Project: IMPALA
> Issue Type: Task
> Components: Infrastructure
> Affects Versions: Impala 5.0.0
> Reporter: Joe McDonnell
> Priority: Major
>
> single_node_perf_run.py executes using impala-python3 and then runs
> buildall.sh. impala-python3 sets LD_LIBRARY_PATH in the environment. If the
> system libstdc++ is newer than the toolchain libstdc++, then using this
> LD_LIBRARY_PATH is incompatible with running system binaries, which need the
> newer libstdc++. To avoid this issue, single_node_perf_run.py should run the
> subcommands without inheriting the environment. This prevents other issues as
> well.
> (An alternative is to only include the toolchain GCC home on the
> LD_LIBRARY_PATH if the toolchain libstdc++ is newer than system libstdc++.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
