[
https://issues.apache.org/jira/browse/HDFS-16496?focusedWorklogId=737295&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-737295
]
ASF GitHub Bot logged work on HDFS-16496:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 06/Mar/22 22:03
Start Date: 06/Mar/22 22:03
Worklog Time Spent: 10m
Work Description: sodonnel opened a new pull request #4051:
URL: https://github.com/apache/hadoop/pull/4051
### Description of PR
Running a snapshot diff against some snapshotable folders gives an error:
```
org.apache.hadoop.hdfs.protocol.SnapshotException: Directory is neither
snapshottable nor under a snap root!
at
org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.checkAndGetSnapshottableAncestorDir(SnapshotManager.java:395)
at
org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.diff(SnapshotManager.java:744)
at
org.apache.hadoop.hdfs.server.namenode.FSDirSnapshotOp.getSnapshotDiffReportListing(FSDirSnapshotOp.java:200)
at
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getSnapshotDiffReportListing(FSNamesystem.java:6983)
at
org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getSnapshotDiffReportListing(NameNodeRpcServer.java:1977)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getSnapshotDiffReportListing(ClientNamenodeProtocolServerSideTranslatorPB.java:1387)
at
org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:533)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:989)
at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:917)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2894)
```
This is caused by
[HDFS-15483](https://issues.apache.org/jira/browse/HDFS-15483) (in order
snapshot delete), and the issue is in the following method in SnapshotManager:
```
public INodeDirectory getSnapshottableAncestorDir(final INodesInPath iip)
throws IOException {
final String path = iip.getPath();
final INode inode = iip.getLastINode();
final INodeDirectory dir;
if (inode instanceof INodeDirectory) { // THIS SHOULD BE TRUE - change
to inode.isDirectory()
dir = INodeDirectory.valueOf(inode, path);
} else {
dir = INodeDirectory.valueOf(iip.getINode(-2), iip.getParentPath());
}
if (dir.isSnapshottable()) {
return dir;
}
for (INodeDirectory snapRoot : this.snapshottables.values()) {
if (dir.isAncestorDirectory(snapRoot)) {
return snapRoot;
}
}
return null;
}
```
After adding some debug, I found the directory which is the snapshot root is
not an instance of INodeDirectory, but instead is an
"INodeReference$DstReference". I think the directory becomes an instance of
this class, if the directory is renamed and one of its children has been moved
out of another snapshot.
The fix is simple - just check `inode.isDirectory()` instead.
### How was this patch tested?
Manually against an affected image with the problem.
### For code changes:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 737295)
Remaining Estimate: 0h
Time Spent: 10m
> Snapshot diff on snapshotable directory fails with not snapshottable error
> --------------------------------------------------------------------------
>
> Key: HDFS-16496
> URL: https://issues.apache.org/jira/browse/HDFS-16496
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namanode
> Affects Versions: 3.4.0
> Reporter: Stephen O'Donnell
> Assignee: Stephen O'Donnell
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Running a snapshot diff against some snapshotable folders gives an error:
> {code}
> org.apache.hadoop.hdfs.protocol.SnapshotException: Directory is neither
> snapshottable nor under a snap root!
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.checkAndGetSnapshottableAncestorDir(SnapshotManager.java:395)
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.diff(SnapshotManager.java:744)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirSnapshotOp.getSnapshotDiffReportListing(FSDirSnapshotOp.java:200)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getSnapshotDiffReportListing(FSNamesystem.java:6983)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getSnapshotDiffReportListing(NameNodeRpcServer.java:1977)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getSnapshotDiffReportListing(ClientNamenodeProtocolServerSideTranslatorPB.java:1387)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:533)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1070)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:989)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:917)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1898)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2894)
> {code}
> This is caused by HDFS-15483 (in order snapshot delete), and the issue is in
> the following method in SnapshotManager:
> {code}
> public INodeDirectory getSnapshottableAncestorDir(final INodesInPath iip)
> throws IOException {
> final String path = iip.getPath();
> final INode inode = iip.getLastINode();
> final INodeDirectory dir;
> if (inode instanceof INodeDirectory) { // THIS SHOULD BE TRUE - change to
> inode.isDirectory()
> dir = INodeDirectory.valueOf(inode, path);
> } else {
> dir = INodeDirectory.valueOf(iip.getINode(-2), iip.getParentPath());
> }
> if (dir.isSnapshottable()) {
> return dir;
> }
> for (INodeDirectory snapRoot : this.snapshottables.values()) {
> if (dir.isAncestorDirectory(snapRoot)) {
> return snapRoot;
> }
> }
> return null;
> }
> {code}
> After adding some debug, I found the directory which is the snapshot root is
> not an instance of INodeDirectory, but instead is an
> "INodeReference$DstReference". I think the directory becomes an instance of
> this class, if the directory is renamed and one of its children has been
> moved out of another snapshot.
> The fix is simple - just check `inode.isDirectory()` instead.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
