[
https://issues.apache.org/jira/browse/HADOOP-16167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16787091#comment-16787091
]
Eric Yang commented on HADOOP-16167:
------------------------------------
{quote}Citation needed. I've never seen the usage of indirects being
discouraged anywhere.{quote}
Example of risk in using indirection reference:
{code}bash-3.2$ x='() { echo vulnerable; }; echo ""'
+ x='() { echo vulnerable; }; echo ""'
bash-3.2$ j=${x}
+ j='() { echo vulnerable; }; echo ""'
bash-3.2$ if [[ -n ${!j} ]]; then echo "not null"; fi
+ [[ -n '' ]]
{code}
x is a script that return last result as null. It is indirectly referenced and
triggered evaluation to not null check.
{code}
bash-3.2$ j=x
+ j=x
bash-3.2$ if [[ -n ${!j} ]]; then echo "not null"; fi
+ [[ -n () { echo vulnerable; }; echo "" ]]
+ echo 'not null'
not null
{code}
Both example are the same and only difference is assignment of j variable. As
the result the referenced value is different, and it can be risky. This only
works in small scale where developer is in control of input variables and copy
of variables. It becomes unclear in large project that caller has no control
over the value in the upstream variables. This is the reason indirection is
discouraged.
There are too few people that can do a full trace in Hadoop scripts to ensure
the string manipulation are done correctly. I trust aw has done his part to
make sure that 80% of the scripts are correct. However, standard practice with
shellcheck SC2086 can avoid bugs that not are apparent to developers. If the 4
lines of aw code can fix this issue, this is fine too.
> "hadoop CLASSFILE" prints error messages on Ubuntu 18
> -----------------------------------------------------
>
> Key: HADOOP-16167
> URL: https://issues.apache.org/jira/browse/HADOOP-16167
> Project: Hadoop Common
> Issue Type: Improvement
> Components: scripts
> Affects Versions: 3.2.0
> Reporter: Daniel Templeton
> Assignee: Daniel Templeton
> Priority: Major
> Attachments: HADOOP-16167.001.patch
>
>
> {noformat}
> # hadoop org.apache.hadoop.conf.Configuration
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2366:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2331:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_USER: bad substitution
> /usr/lib/hadoop/bin/../lib/hadoop/libexec//hadoop-functions.sh: line 2426:
> HADOOP_ORG.APACHE.HADOOP.CONF.CONFIGURATION_OPTS: bad substitution
> {noformat}
> The issue is a regression in bash 4.4. See
> [here|http://savannah.gnu.org/support/?109649]. The extraneous output can
> break scripts that read the command output.
> According to [~aw]:
> {quote}Oh, I think I see the bug. HADOOP_SUBCMD (and equivalents in yarn,
> hdfs, etc) just needs some special handling when a custom method is being
> called. For example, there’s no point in checking to see if it should run
> with privileges, so just skip over that. Probably a few other places too.
> Relatively easy fix. 2 lines of code, maybe.{quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
