-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/74804/
-----------------------------------------------------------
Review request for ranger, Akshay Tupe, Anand Nadar, Ankita Sinha, Madhan
Neethiraj, Monika Kachhadiya, Subhrat Chaudhary, and Vanita Ubale.
Bugs: RANGER-4625
https://issues.apache.org/jira/browse/RANGER-4625
Repository: ranger
Description
-------
According to
https://github.com/apache/ranger/blob/master/dev-support/ranger-docker/README.md,
using docker to try start ranger, the container ranger-hadoop fails to start.
If the following information is displayed:
```
+ Thu Dec 21 09:29:17 UTC 2023 : Saving current config file:
/opt/hadoop/etc/hadoop/ranger-yarn-audit.xml to
/opt/hadoop/etc/hadoop/.ranger-yarn-audit.xml.20231221-092916 ...
+ Thu Dec 21 09:29:17 UTC 2023 : Saving current config file:
/opt/hadoop/etc/hadoop/ranger-yarn-security.xml to
/opt/hadoop/etc/hadoop/.ranger-yarn-security.xml.20231221-092916 ...
+ Thu Dec 21 09:29:17 UTC 2023 : Saving current config file:
/opt/hadoop/etc/hadoop/yarn-site.xml to
/opt/hadoop/etc/hadoop/.yarn-site.xml.20231221-092916 ...
+ Thu Dec 21 09:29:18 UTC 2023 : Saving lib file:
/opt/hadoop/share/hadoop/hdfs/lib/ranger-plugin-classloader-2.4.0.jar to
/opt/hadoop/share/hadoop/hdfs/lib/.ranger-plugin-classloader-2.4.0.jar.20231221092918
...
Unable to store password in non-plain text format. Error: [SLF4J: Failed to
load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further
details.
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/commons/logging/LogFactory
at org.apache.hadoop.fs.FileSystem.<clinit>(FileSystem.java:170)
at org.apache.hadoop.fs.Path.getFileSystem(Path.java:365)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.initFileSystem(JavaKeyStoreProvider.java:89)
at
org.apache.hadoop.security.alias.AbstractJavaKeyStoreProvider.<init>(AbstractJavaKeyStoreProvider.java:85)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:49)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider.<init>(JavaKeyStoreProvider.java:41)
at
org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory.createProvider(JavaKeyStoreProvider.java:100)
at
org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:73)
at
org.apache.ranger.credentialapi.CredentialReader.getDecryptedString(CredentialReader.java:74)
at
org.apache.ranger.credentialapi.buildks.createCredential(buildks.java:87)
at org.apache.ranger.credentialapi.buildks.main(buildks.java:41)
Caused by: java.lang.ClassNotFoundException:
org.apache.commons.logging.LogFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 11 more]
Exiting plugin installation
2023-12-21 09:29:20,453 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = ranger-hadoop.example.com/172.18.0.7
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 3.3.0
```
Diffs
-----
agents-cred/pom.xml 285217b51
distro/src/main/assembly/plugin-yarn.xml c0a8ca3af
Diff: https://reviews.apache.org/r/74804/diff/1/
Testing
-------
1) set working directory
```
$ cd ranger/dev-support/ranger-docker
```
2) download necessary archives
```
$ chmod +x download-archives.sh && ./download-archives.sh
```
3) set environment variables
```
$ export DOCKER_BUILDKIT=1
$ export COMPOSE_DOCKER_CLI_BUILD=1
$ export RANGER_DB_TYPE=postgres
```
4) build Apache Ranger in containers using docker-compose
```
$ docker-compose -f docker-compose.ranger-base.yml -f
docker-compose.ranger-build.yml up
```
5) start Ranger, Ranger enabled HDFS/YARN and dependent services (Solr, DB) in
containers:
```
$ docker-compose -f docker-compose.ranger-base.yml -f docker-compose.ranger.yml
-f docker-compose.ranger-${RANGER_DB_TYPE}.yml -f
docker-compose.ranger-usersync.yml -f docker-compose.ranger-tagsync.yml -f
docker-compose.ranger-hadoop.yml up -d
```
6) check whether the container `ranger-hadoop` is in a healthy status
```
$ docker ps # check container ranger-hadoop status
CONTAINER ID IMAGE COMMAND CREATED STATUS
PORTS
NAMES
d4484d2772ae ranger-usersync "/home/ranger/script…" 5 hours ago Up 5
hours
ranger-usersync
096a602d8dba ranger-tagsync "/home/ranger/script…" 5 hours ago Up 5
hours
ranger-tagsync
eb6beb3b6adb ranger-hadoop "/home/ranger/script…" 5 hours ago Up 5
hours (healthy) 0.0.0.0:8088->8088/tcp, :::8088->8088/tcp,
0.0.0.0:9000->9000/tcp, :::9000->9000/tcp ranger-hadoop
7b33bbdfacc4 ranger:latest "/home/ranger/script…" 5 hours ago Up 5
hours 0.0.0.0:6080->6080/tcp, :::6080->6080/tcp
ranger
d06e9489d24c ranger-zk "/docker-entrypoint.…" 5 hours ago Up 5
hours 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp,
:::2181->2181/tcp, 8080/tcp ranger-zk
02d44ec41a13 ranger-solr "docker-entrypoint.s…" 5 hours ago Up 5
hours 0.0.0.0:8983->8983/tcp, :::8983->8983/tcp
ranger-solr
65c47f73e71c ranger-postgres "docker-entrypoint.s…" 5 hours ago Up 5
hours (healthy) 5432/tcp
ranger-postgres
$
```
Thanks,
qingzhong li
