Ramachandran Krishnan created RANGER-5660:
---------------------------------------------
Summary: YARN plugin: fix enable script lib path, add missing
template/configuration.xml, align impl packaging with HDFS agent
Key: RANGER-5660
URL: https://issues.apache.org/jira/browse/RANGER-5660
Project: Ranger
Issue Type: Task
Components: Ranger
Reporter: Ramachandran Krishnan
Assignee: Ramachandran Krishnan
Fix For: 3.0.0
h3. Problem
The Ranger YARN plugin fails to install and run correctly on Hadoop 3.x in
environments that use the standard plugin tarball and {{enable-yarn-plugin.sh}}
workflow.
Two upstream gaps and one packaging inconsistency were identified:
# Missing template for assembly
{{distro/src/main/assembly/plugin-yarn.xml}} includes a fileSet for
{{install/conf.templates/default}} from {{{}plugin-yarn/template/{}}}, but that
directory was never committed. A Maven-built tarball can be incomplete, and
{{enable-yarn-plugin.sh}} may fail before writing YARN security/audit
configuration.
# Wrong Hadoop lib directory for YARN
In {{{}agents-common/scripts/enable-agent.sh{}}}, YARN is grouped with HDFS and
plugin jars are installed under {{share/hadoop/hdfs/lib}} instead of
{{{}share/hadoop/yarn/lib{}}}. ResourceManager and NodeManager load from
{{{}yarn/lib{}}}, so the authorizer may not be wired correctly even when enable
appears to succeed.
# Impl dependency packaging gap
{{hdfs-agent.xml}} includes {{slf4j-api}} in the plugin impl module set;
{{plugin-yarn.xml}} does not. YARN impl packaging should match the HDFS agent
model for consistent child-first classloader behavior.
A related runtime symptom (mixed Jackson 2.12 + 2.17 in stale tarballs) is
resolved by rebuilding from assembly with pinned
{{{}${fasterxml.jackson.version}{}}}; no {{RangerPluginClassLoader}} change is
required for YARN (unlike Kafka).
----
h3. Steps to reproduce
# Build or use {{ranger-*-yarn-plugin.tar.gz}} from current {{{}main{}}}.
# On a Hadoop 3.x node, run {{./enable-yarn-plugin.sh}} with valid
{{{}install.properties{}}}.
# Observe one or more of:
** Enable script fails or does not produce {{ranger-yarn-security.xml}} /
{{ranger-yarn-audit.xml}}
** Plugin shim/classloader jars under {{share/hadoop/hdfs/lib}} but not
{{share/hadoop/yarn/lib}}
** With {{yarn.authorization-provider}} set to {{{}RangerYarnAuthorizer{}}},
RM fails with Jackson-related errors when impl contains mixed Jackson versions
from an outdated tarball
----
h3. Proposed fix
||File||Change||
|{{plugin-yarn/template/configuration.xml}}|Add empty Hadoop-style template so
assembly {{conf.templates/default}} is populated|
|{{agents-common/scripts/enable-agent.sh}}|Install YARN plugin jars to
{{${HCOMPONENT_INSTALL_DIR}/share/hadoop/yarn/lib}} (separate from HDFS)|
|{{distro/src/main/assembly/plugin-yarn.xml}}|Add
{{org.slf4j:slf4j-api:jar:${slf4j-api.version}}} to {{ranger-yarn-plugin-impl}}
includes (match {{{}hdfs-agent.xml{}}})|
----
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
