[
https://issues.apache.org/jira/browse/HDFS-15226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
bianqi updated HDFS-15226:
--------------------------
Description:
When I integrated ranger1.2 with Hadoop2.7.6, the following NPE error occurred
when executing hdfs dfs -ls /.
However, when I integrated ranger1.2 with Hadoop2.7.1, executing hdfs dfs -ls
/ without any errors, and the directory list can be displayed normally.
{quote}java.lang.NullPointerException
at java.lang.String.checkBounds(String.java:384)
at java.lang.String.<init>(String.java:425)
at org.apache.hadoop.hdfs.DFSUtil.bytes2String(DFSUtil.java:337)
at org.apache.hadoop.hdfs.DFSUtil.bytes2String(DFSUtil.java:319)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.getINodeAttrs(FSPermissionChecker.java:238)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:183)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1752)
at
org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getFileInfo(FSDirStatAndListingOp.java:100)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:3832)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getFileInfo(NameNodeRpcServer.java:1012)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getFileInfo(ClientNamenodeProtocolServerSideTranslatorPB.java:855)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2217)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2213)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1758)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2213)
DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8020: responding
to org.apache.hadoop.hdfs.protocol.ClientProtocol.getFileInfo from xxxxxx:8502
Call#0 Retry#0
{quote}
When I checked the HDFS source code, I compared hadoop2.7.1 and hadoop2.7.6 and
found that 2.7.6 added the following methods
[^image-2020-03-16-14-01-03-078.png]
I found that the code of the latest master branch has been fixed and currently
has not been merged in the latest 3.2.1 version.
{quote}private INodeAttributes getINodeAttrs(byte[][] pathByNameArr, int
pathIdx,
INode inode, int snapshotId) {
INodeAttributes inodeAttrs = inode.getSnapshotINode(snapshotId);
if (getAttributesProvider() != null) {
String[] elements = new String[pathIdx + 1];
/**
* \{@link INode#getPathComponents(String)} returns a null component
* for the root only path "/". Assign an empty string if so.
*/
if (pathByNameArr.length == 1 && pathByNameArr[0] == null) {
elements[0] = "";
} else {
for (int i = 0; i < elements.length; i++) {
elements[i] = DFSUtil.bytes2String(pathByNameArr[i]);
}
}
inodeAttrs = getAttributesProvider().getAttributes(elements, inodeAttrs);
}
return inodeAttrs;
}
{quote}
was:
When I integrated ranger1.2 with Hadoop2.7.6, the following NPE error
occurred when executing hdfs dfs -ls /.
However, when I integrated ranger1.2 with Hadoop2.7.1, executing hdfs dfs
-ls / without any errors, and the directory list can be displayed normally.
{quote}java.lang.NullPointerException
at java.lang.String.checkBounds(String.java:384)
at java.lang.String.<init>(String.java:425)
at org.apache.hadoop.hdfs.DFSUtil.bytes2String(DFSUtil.java:337)
at org.apache.hadoop.hdfs.DFSUtil.bytes2String(DFSUtil.java:319)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.getINodeAttrs(FSPermissionChecker.java:238)
at
org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:183)
at
org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1752)
at
org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getFileInfo(FSDirStatAndListingOp.java:100)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:3832)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getFileInfo(NameNodeRpcServer.java:1012)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getFileInfo(ClientNamenodeProtocolServerSideTranslatorPB.java:855)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2217)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2213)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1758)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2213)
DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8020: responding to
org.apache.hadoop.hdfs.protocol.ClientProtocol.getFileInfo from xxxxxx:8502
Call#0 Retry#0{quote}
When I checked the HDFS source code, I compared hadoop2.7.1 and
hadoop2.7.6 and found that 2.7.6 added the following methods
!image-2020-03-16-14-01-03-078.png|thumbnail!
> Ranger integrates HDFS and discovers NPE
> ----------------------------------------
>
> Key: HDFS-15226
> URL: https://issues.apache.org/jira/browse/HDFS-15226
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: hdfs
> Affects Versions: 2.7.6
> Environment: Apache Ranger1.2 && Hadoop2.7.6
> Reporter: bianqi
> Priority: Critical
> Fix For: 3.2.0, 3.2.1
>
> Attachments: image-2020-03-16-14-01-03-078.png
>
>
> When I integrated ranger1.2 with Hadoop2.7.6, the following NPE error
> occurred when executing hdfs dfs -ls /.
> However, when I integrated ranger1.2 with Hadoop2.7.1, executing hdfs dfs
> -ls / without any errors, and the directory list can be displayed normally.
> {quote}java.lang.NullPointerException
> at java.lang.String.checkBounds(String.java:384)
> at java.lang.String.<init>(String.java:425)
> at org.apache.hadoop.hdfs.DFSUtil.bytes2String(DFSUtil.java:337)
> at org.apache.hadoop.hdfs.DFSUtil.bytes2String(DFSUtil.java:319)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.getINodeAttrs(FSPermissionChecker.java:238)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:183)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirectory.checkPermission(FSDirectory.java:1752)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getFileInfo(FSDirStatAndListingOp.java:100)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getFileInfo(FSNamesystem.java:3832)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getFileInfo(NameNodeRpcServer.java:1012)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getFileInfo(ClientNamenodeProtocolServerSideTranslatorPB.java:855)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:982)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2217)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2213)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1758)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2213)
> DEBUG org.apache.hadoop.ipc.Server: IPC Server handler 1 on 8020: responding
> to org.apache.hadoop.hdfs.protocol.ClientProtocol.getFileInfo from
> xxxxxx:8502 Call#0 Retry#0
> {quote}
> When I checked the HDFS source code, I compared hadoop2.7.1 and hadoop2.7.6
> and found that 2.7.6 added the following methods
> [^image-2020-03-16-14-01-03-078.png]
> I found that the code of the latest master branch has been fixed and
> currently has not been merged in the latest 3.2.1 version.
>
> {quote}private INodeAttributes getINodeAttrs(byte[][] pathByNameArr, int
> pathIdx,
> INode inode, int snapshotId) {
> INodeAttributes inodeAttrs = inode.getSnapshotINode(snapshotId);
> if (getAttributesProvider() != null) {
> String[] elements = new String[pathIdx + 1];
> /**
> * \{@link INode#getPathComponents(String)} returns a null component
> * for the root only path "/". Assign an empty string if so.
> */
> if (pathByNameArr.length == 1 && pathByNameArr[0] == null) {
> elements[0] = "";
> } else {
> for (int i = 0; i < elements.length; i++) {
> elements[i] = DFSUtil.bytes2String(pathByNameArr[i]);
> }
> }
> inodeAttrs = getAttributesProvider().getAttributes(elements, inodeAttrs);
> }
> return inodeAttrs;
> }
> {quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
