kirby zhou created RANGER-3595:
----------------------------------
Summary: Tar of KMS contains rubbish files
Key: RANGER-3595
URL: https://issues.apache.org/jira/browse/RANGER-3595
Project: Ranger
Issue Type: Improvement
Components: kms
Reporter: kirby zhou
There are lots of .class files under ews/webapp/. They wont be loaded by any
classpath. And they are duplicated against files inside
ews/webapp/lib/ranger-kms-3.0.0-SNAPSHOT.jar.
It seems dirty and may cause some security problem.
{code:bash}
#] tar tf target/ranger-3.0.0-SNAPSHOT-kms.tar.gz ranger-3.0.0-SNAPSHOT-kms/ |
egrep 'ews/webapp/org' | head
ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/
ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/
ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/
ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/kms/
ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/kms/biz/
ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/kms/dao/
ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/ranger/entity/
ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/hadoop/
ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/hadoop/crypto/
ranger-3.0.0-SNAPSHOT-kms/ews/webapp/org/apache/hadoop/crypto/key/
//代码占位符
{code}
* The reason is that:
distro/src/main/assembly/kms.xml
{code:java}
<moduleSet>
<useAllReactorProjects>true</useAllReactorProjects>
<includes>
<include>org.apache.ranger:ranger-kms</include>
</includes>
<binaries>
<outputDirectory>ews/webapp</outputDirectory>
<includeDependencies>false</includeDependencies>
<unpack>true</unpack>
</binaries>
</moduleSet> {code}
Why ?
The secret is in kms/scripts/setup.sh:
{code:java}
setup_kms(){
#copying ranger kms provider
oldP=${PWD}
cd $PWD/ews/webapp
log "[I] Adding ranger kms provider as services in hadoop-common jar"
for f in lib/hadoop-common*.jar
do
${JAVA_HOME}/bin/jar -uf ${f}
META-INF/services/org.apache.hadoop.crypto.key.KeyProviderFactory
chown ${unix_user}:${unix_group} ${f}
done
cd ${oldP}
}
{code}
The code above is VERY VERY DIRTY!
It hacks into hadoop-common.jar., Overwrite resource
"META-INF/services/org.apache.hadoop.crypto.key.KeyProviderFactory". Ensure the
following code can load
'org.apache.hadoop.crypto.key.RangerKeyStoreProvider$Factory' by
'META-INF/.../KeyProviderFactory'.
{code:java}
// org.apache.hadoop.crypto.key: KeyProviderFactory.java
private static final ServiceLoader<KeyProviderFactory> serviceLoader =
ServiceLoader.load(KeyProviderFactory.class,
KeyProviderFactory.class.getClassLoader());
{code}
But this is unnecessary.
ServiceLoader will read all resources with the same name using the ClassLoader
of KeyProviderFactory. We just need to put a jar contains that property side by
side of hadoop-common.jar ( ews/webapp/lib/ ). And
ranger-kms-3.0.0-SNAPSHOT.jar already here.
{code:java}
% tar tf ../target/ranger-*-kms.tar.gz | egrep 'kms[^/]*\.jar|hadoop-common'
ranger-3.0.0-SNAPSHOT-kms/ews/webapp/lib/ranger-kms-3.0.0-SNAPSHOT.jar
ranger-3.0.0-SNAPSHOT-kms/ews/webapp/lib/hadoop-common-3.3.0.jar
...
% tar tf target/ranger-kms-3.0.0-SNAPSHOT.jar | fgrep ProviderFactory
META-INF/services/org.apache.hadoop.crypto.key.KeyProviderFactory
{code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
