[jira] [Commented] (RANGER-4327) Make ranger2.4 hdfs plugin compatible with hadoop 3.3.5 +

2024-03-28 Thread Chingachgook (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17832030#comment-17832030
 ] 

Chingachgook commented on RANGER-4327:
--

I confirm that we have removed jackson-1.x from the Ranger Snapshot-3.0.0 
distribution and managed to create packages to work with hadoop 3.3.6

> Make ranger2.4 hdfs plugin compatible with hadoop 3.3.5 +
> -
>
> Key: RANGER-4327
> URL: https://issues.apache.org/jira/browse/RANGER-4327
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Affects Versions: 2.3.0, 2.4.0
>Reporter: caijialiang
>Assignee: caijialiang
>Priority: Major
> Attachments: image-2023-07-25-17-30-18-136.png, 
> image-2023-07-26-13-40-10-555.png, patch3-RANGER-4327.diff
>
>
> hadoop 3.3.5 has upgraded its jackson dependency and removed all jackson 1.x 
> packages, including the jackson-jaxrs jar. As a result, when starting with 
> the Ranger plugin enabled, Hadoop namenode fails to start due to the 
> inability to load jackson 1.x.
> To address this issue and ensure compatibility with the Hadoop 3.3.5 upgrade, 
> we need to package jackson 1.x jars in the hdfs plugin as well,like the Ozone 
> did https://issues.apache.org/jira/browse/RANGER-2818. This will allow the 
> namenode to start successfully.
> !image-2023-07-26-13-40-10-555.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (RANGER-4327) Make ranger2.4 hdfs plugin compatible with hadoop 3.3.5 +

2024-03-21 Thread Chingachgook (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829546#comment-17829546
 ] 

Chingachgook commented on RANGER-4327:
--

I applied the attached patch and the error with jackson went away, but now I 
get another error
How can this problem be solved?
{code:java}
2024-03-21 15:57:09,993 DEBUG classloader.RangerPluginClassLoader: 
RangerPluginClassLoader.loadClass(java.util.Iterator): calling 
childClassLoader.findClass()
2024-03-21 15:57:09,993 DEBUG classloader.RangerPluginClassLoader: <== 
RangerPluginClassLoader.loadClass(java.util.Iterator): interface 
java.util.Iterator
2024-03-21 15:57:09,993 DEBUG classloader.RangerPluginClassLoader: ==> 
RangerPluginClassLoader.deactivate()
2024-03-21 15:57:09,993 DEBUG classloader.RangerPluginClassLoader: <== 
RangerPluginClassLoader.deactivate()
2024-03-21 15:57:09,993 ERROR namenode.FSNamesystem: FSNamesystem 
initialization failed.
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
        at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:138)
        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.(FSNamesystem.java:1026)
        at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:800)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:694)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:781)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:1033)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:1008)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1782)
        at 
org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1847)
Caused by: java.lang.reflect.InvocationTargetException
        at 
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
 Method)
        at 
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at 
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at 
java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
        at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
        ... 8 more
Caused by: java.lang.LinkageError: loader constraint violation: when resolving 
method 'org.slf4j.ILoggerFactory 
org.slf4j.impl.StaticLoggerBinder.getLoggerFactory()' the class loader 
org.apache.ranger.plugin.classloader.RangerPluginClassLoader @36c0d0bd of the 
current class, org/slf4j/LoggerFactory, and the class loader 'app' for the 
method's defining class, org/slf4j/impl/StaticLoggerBinder, have different 
Class objects for the type org/slf4j/ILoggerFactory used in the signature 
(org.slf4j.LoggerFactory is in unnamed module of loader 
org.apache.ranger.plugin.classloader.RangerPluginClassLoader @36c0d0bd, parent 
loader 'bootstrap'; org.slf4j.impl.StaticLoggerBinder is in unnamed module of 
loader 'app')
        at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:423)
        at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:362)
        at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:388)
        at 
org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.(RangerHdfsAuthorizer.java:83)
        at java.base/java.lang.Class.forName0(Native Method)
        at java.base/java.lang.Class.forName(Class.java:398)
        at 
org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.init(RangerHdfsAuthorizer.java:60)
        at 
org.apache.ranger.authorization.hadoop.RangerHdfsAuthorizer.(RangerHdfsAuthorizer.java:43)
        ... 13 more
2024-03-21 15:57:10,038 ERROR blockmanagement.DatanodeManager: Slow peers 
collection thread interrupted
java.lang.InterruptedException: sleep interrupted
        at java.base/java.lang.Thread.sleep(Native Method)
        at 
org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager$1.run(DatanodeManager.java:395)
        at java.base/java.lang.Thread.run(Thread.java:829)
2024-03-21 15:57:10,306 INFO namenode.FSNamesystem: Stopping services started 
for active state
2024-03-21 15:57:10,306 DEBUG delegation.AbstractDelegationTokenSecretManager: 
Stopping expired delegation token remover thread
2024-03-21 15:57:10,312 INFO namenode.FSNamesystem: Stopping services started 
for standby state
2024-03-21 15:57:10,312 DEBUG namenode.FSEditLog: Closing log when already 
closed
2024-03-21 15:57:10,313 DEBUG namenode.FSEditLog: Closing log when already 
closed
2024-03-21 15:57:10,313 ERROR namenode.NameNode: Failed to start namenode.
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
        at 
org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:138)
        at 

[jira] [Commented] (RANGER-4327) Make ranger2.4 hdfs plugin compatible with hadoop 3.3.5 +

2024-01-11 Thread Bhavik Patel (Jira)


[ 
https://issues.apache.org/jira/browse/RANGER-4327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17805901#comment-17805901
 ] 

Bhavik Patel commented on RANGER-4327:
--

This issue is observed as jackson-1.x version is removed in the hadoop-3.4 : 
[https://github.com/apache/hadoop/commit/04b6b9a87bf024e82aa50328b4994aea4268d98e#diff-635a9ebcc70f58d8ca1c55b0d78cc90064a5081d189162a432271778da3cf018]

>From ranger end also we have to remove jackson-1.x version as it's vulnerable.

> Make ranger2.4 hdfs plugin compatible with hadoop 3.3.5 +
> -
>
> Key: RANGER-4327
> URL: https://issues.apache.org/jira/browse/RANGER-4327
> Project: Ranger
>  Issue Type: Improvement
>  Components: plugins
>Affects Versions: 2.3.0, 2.4.0
>Reporter: caijialiang
>Assignee: caijialiang
>Priority: Major
> Attachments: image-2023-07-25-17-30-18-136.png, 
> image-2023-07-26-13-40-10-555.png, patch3-RANGER-4327.diff
>
>
> hadoop 3.3.5 has upgraded its jackson dependency and removed all jackson 1.x 
> packages, including the jackson-jaxrs jar. As a result, when starting with 
> the Ranger plugin enabled, Hadoop namenode fails to start due to the 
> inability to load jackson 1.x.
> To address this issue and ensure compatibility with the Hadoop 3.3.5 upgrade, 
> we need to package jackson 1.x jars in the hdfs plugin as well,like the Ozone 
> did https://issues.apache.org/jira/browse/RANGER-2818. This will allow the 
> namenode to start successfully.
> !image-2023-07-26-13-40-10-555.png!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)