deepujain opened a new pull request, #8670:
URL: https://github.com/apache/hadoop/pull/8670
### Description of PR
**Summary**
When `HADOOP_USER_CLASSPATH_FIRST=true`, a user-supplied classpath entry that
is already present in Hadoop's computed classpath can stay in its older
position instead of moving to the front. This happens because duplicate
classpath entries were treated as a no-op even when they were being re-added
with `before`.
This patch keeps the existing de-duplication behavior, but when a duplicate
entry is re-added with `before`, it is moved to the front of `CLASSPATH`.
That preserves the expected override behavior for cases such as
`HADOOP_CONF_DIR` also being present in `HADOOP_CLASSPATH`.
**Change**
- `hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh`:
update `hadoop_add_classpath()` so duplicate entries are reordered to the
front when added with `before`.
-
`hadoop-common-project/hadoop-common/src/test/scripts/hadoop_add_classpath.bats`:
add a regression that proves a duplicate entry can be moved to the front.
-
`hadoop-common-project/hadoop-common/src/test/scripts/hadoop_finalize_classpath.bats`:
add a higher-level regression that matches the JIRA scenario where
`HADOOP_CONF_DIR` is duplicated in `HADOOP_CLASSPATH` while
`HADOOP_USER_CLASSPATH_FIRST=true`.
**Evidence it works**
- `bash -n
hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh`
- `git diff --check`
- Direct shell validation of duplicate reordering:
`/bin/bash -lc 'set -eo pipefail; export HADOOP_SHELL_SCRIPT_DEBUG=true;
export QATESTMODE=true; export HADOOP_LIBEXEC_DIR=$(cd -P
hadoop-common-project/hadoop-common/src/main/bin >/dev/null && pwd -P); source
hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh;
tmpdir=$(mktemp -d); mkdir -p "$tmpdir/first" "$tmpdir/second"; CLASSPATH="";
hadoop_add_classpath "$tmpdir/first" after; hadoop_add_classpath
"$tmpdir/second" after; hadoop_add_classpath "$tmpdir/second" before; test
"$CLASSPATH" = "$tmpdir/second:$tmpdir/first"; rm -rf "$tmpdir"'`
- Direct shell validation of the JIRA-style finalize flow:
`/bin/bash -lc 'set -eo pipefail; export HADOOP_SHELL_SCRIPT_DEBUG=true;
export QATESTMODE=true; export HADOOP_LIBEXEC_DIR=$(cd -P
hadoop-common-project/hadoop-common/src/main/bin >/dev/null && pwd -P); source
hadoop-common-project/hadoop-common/src/main/bin/hadoop-functions.sh;
tmpdir=$(mktemp -d); mkdir -p "$tmpdir/new" "$tmpdir/old"; CLASSPATH="";
HADOOP_CONF_DIR="$tmpdir"; HADOOP_CLASSPATH="$tmpdir:$tmpdir/new:$tmpdir/old";
HADOOP_USER_CLASSPATH_FIRST=true; HADOOP_USE_CLIENT_CLASSLOADER="";
hadoop_translate_cygwin_path() { true; }; hadoop_finalize_classpath; test
"$CLASSPATH" = "$tmpdir:$tmpdir/new:$tmpdir/old"; rm -rf "$tmpdir"'`
- `bats` is not installed in this local environment, so the new `.bats`
regressions were validated through the equivalent sourced-function shell
invocations above.
**JIRA**
Fixes HADOOP-15019
### For code changes:
- [x] Does the title of this PR start with the corresponding JIRA issue id
(e.g. 'HADOOP-17799. Your PR title ...')?
- [ ] Object storage: Have the integration tests been executed and the
endpoint
declared according to the connector-specific documentation? Not
applicable to this shell-script change.
- [x] If adding new dependencies to the code, are these dependencies licensed
in a way that is compatible for inclusion under
[ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
No new dependencies are added.
- [x] If applicable, have you updated the `LICENSE`, `LICENSE-binary`,
`NOTICE-binary` files? No license or notice changes are required.
### AI Tooling
Contains content generated by Codex.
If an AI tool was used:
- [x] The PR includes the phrase "Contains content generated by <tool>"
where <tool> is the name of the AI tool used.
- [x] My use of AI contributions follows the ASF legal policy
https://www.apache.org/legal/generative-tooling.html
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
