[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[
https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=485609=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-485609
]
ASF GitHub Bot logged work on HDFS-15568:
-
Author: ASF GitHub Bot
Created on: 17/Sep/20 09:20
Start Date: 17/Sep/20 09:20
Worklog Time Spent: 10m
Work Description: bshashikant merged pull request #2296:
URL: https://github.com/apache/hadoop/pull/2296
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 485609)
Time Spent: 3h 10m (was: 3h)
> namenode start failed to start when dfs.namenode.snapshot.max.limit set
> ---
>
> Key: HDFS-15568
> URL: https://issues.apache.org/jira/browse/HDFS-15568
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Nilotpal Nandi
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> {code:java}
> 11:35:05.872 AM ERROR NameNode
> Failed to start namenode.
> org.apache.hadoop.hdfs.protocol.SnapshotException: Failed to add snapshot:
> there are already 20 snapshot(s) and the max snapshot limit is 20
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.DirectorySnapshottableFeature.addSnapshot(DirectorySnapshottableFeature.java:181)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeDirectory.addSnapshot(INodeDirectory.java:285)
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.createSnapshot(SnapshotManager.java:447)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:802)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:287)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:182)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:912)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:760)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:337)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1164)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:755)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:646)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:717)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:960)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:933)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1670)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1737)
> {code}
> Steps to reproduce:
> --
> directory level snapshot limit set - 100
> Created 100 snapshots
> deleted all 100 snapshots (in-oder)
> No snapshot exist
> Then, directory level snapshot limit set - 20
> HDFS restart
> Namenode start failed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[
https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=485610=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-485610
]
ASF GitHub Bot logged work on HDFS-15568:
-
Author: ASF GitHub Bot
Created on: 17/Sep/20 09:20
Start Date: 17/Sep/20 09:20
Worklog Time Spent: 10m
Work Description: bshashikant commented on pull request #2296:
URL: https://github.com/apache/hadoop/pull/2296#issuecomment-694109588
Thanks @szetszwo and @goiri for the review.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 485610)
Time Spent: 3h 20m (was: 3h 10m)
> namenode start failed to start when dfs.namenode.snapshot.max.limit set
> ---
>
> Key: HDFS-15568
> URL: https://issues.apache.org/jira/browse/HDFS-15568
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Nilotpal Nandi
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> {code:java}
> 11:35:05.872 AM ERROR NameNode
> Failed to start namenode.
> org.apache.hadoop.hdfs.protocol.SnapshotException: Failed to add snapshot:
> there are already 20 snapshot(s) and the max snapshot limit is 20
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.DirectorySnapshottableFeature.addSnapshot(DirectorySnapshottableFeature.java:181)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeDirectory.addSnapshot(INodeDirectory.java:285)
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.createSnapshot(SnapshotManager.java:447)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:802)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:287)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:182)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:912)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:760)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:337)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1164)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:755)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:646)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:717)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:960)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:933)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1670)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1737)
> {code}
> Steps to reproduce:
> --
> directory level snapshot limit set - 100
> Created 100 snapshots
> deleted all 100 snapshots (in-oder)
> No snapshot exist
> Then, directory level snapshot limit set - 20
> HDFS restart
> Namenode start failed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[
https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=484946=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-484946
]
ASF GitHub Bot logged work on HDFS-15568:
-
Author: ASF GitHub Bot
Created on: 16/Sep/20 05:51
Start Date: 16/Sep/20 05:51
Worklog Time Spent: 10m
Work Description: bshashikant commented on pull request #2296:
URL: https://github.com/apache/hadoop/pull/2296#issuecomment-693187529
/retest
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 484946)
Time Spent: 3h (was: 2h 50m)
> namenode start failed to start when dfs.namenode.snapshot.max.limit set
> ---
>
> Key: HDFS-15568
> URL: https://issues.apache.org/jira/browse/HDFS-15568
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Nilotpal Nandi
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: pull-request-available
> Time Spent: 3h
> Remaining Estimate: 0h
>
> {code:java}
> 11:35:05.872 AM ERROR NameNode
> Failed to start namenode.
> org.apache.hadoop.hdfs.protocol.SnapshotException: Failed to add snapshot:
> there are already 20 snapshot(s) and the max snapshot limit is 20
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.DirectorySnapshottableFeature.addSnapshot(DirectorySnapshottableFeature.java:181)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeDirectory.addSnapshot(INodeDirectory.java:285)
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.createSnapshot(SnapshotManager.java:447)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:802)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:287)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:182)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:912)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:760)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:337)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1164)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:755)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:646)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:717)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:960)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:933)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1670)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1737)
> {code}
> Steps to reproduce:
> --
> directory level snapshot limit set - 100
> Created 100 snapshots
> deleted all 100 snapshots (in-oder)
> No snapshot exist
> Then, directory level snapshot limit set - 20
> HDFS restart
> Namenode start failed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[
https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=484944=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-484944
]
ASF GitHub Bot logged work on HDFS-15568:
-
Author: ASF GitHub Bot
Created on: 16/Sep/20 05:50
Start Date: 16/Sep/20 05:50
Worklog Time Spent: 10m
Work Description: bshashikant commented on pull request #2296:
URL: https://github.com/apache/hadoop/pull/2296#issuecomment-693187393
retest
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 484944)
Time Spent: 2h 50m (was: 2h 40m)
> namenode start failed to start when dfs.namenode.snapshot.max.limit set
> ---
>
> Key: HDFS-15568
> URL: https://issues.apache.org/jira/browse/HDFS-15568
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Nilotpal Nandi
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> {code:java}
> 11:35:05.872 AM ERROR NameNode
> Failed to start namenode.
> org.apache.hadoop.hdfs.protocol.SnapshotException: Failed to add snapshot:
> there are already 20 snapshot(s) and the max snapshot limit is 20
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.DirectorySnapshottableFeature.addSnapshot(DirectorySnapshottableFeature.java:181)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeDirectory.addSnapshot(INodeDirectory.java:285)
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.createSnapshot(SnapshotManager.java:447)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:802)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:287)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:182)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:912)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:760)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:337)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1164)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:755)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:646)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:717)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:960)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:933)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1670)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1737)
> {code}
> Steps to reproduce:
> --
> directory level snapshot limit set - 100
> Created 100 snapshots
> deleted all 100 snapshots (in-oder)
> No snapshot exist
> Then, directory level snapshot limit set - 20
> HDFS restart
> Namenode start failed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[
https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=484140=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-484140
]
ASF GitHub Bot logged work on HDFS-15568:
-
Author: ASF GitHub Bot
Created on: 14/Sep/20 17:39
Start Date: 14/Sep/20 17:39
Worklog Time Spent: 10m
Work Description: goiri commented on a change in pull request #2296:
URL: https://github.com/apache/hadoop/pull/2296#discussion_r488109432
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotManager.java
##
@@ -133,4 +138,57 @@ public void testValidateSnapshotIDWidth() throws Exception
{
getMaxSnapshotID() < Snapshot.CURRENT_STATE_ID);
}
+ @Test
+ public void testSnapshotLimitOnRestart() throws Exception {
+final Configuration conf = new Configuration();
+final Path snapshottableDir
+= new Path("/" + getClass().getSimpleName());
+int numSnapshots = 5;
+conf.setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_MAX_LIMIT, numSnapshots);
+conf.setInt(DFSConfigKeys.DFS_NAMENODE_SNAPSHOT_FILESYSTEM_LIMIT,
+numSnapshots * 2);
+MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).
+numDataNodes(0).build();
+cluster.waitActive();
+DistributedFileSystem hdfs = cluster.getFileSystem();
+hdfs.mkdirs(snapshottableDir);
+hdfs.allowSnapshot(snapshottableDir);
+for (int i = 0; i < numSnapshots; i++) {
+ hdfs.createSnapshot(snapshottableDir, "s" + i);
+}
+LambdaTestUtils.intercept(SnapshotException.class,
+"snapshot limit",
+() -> hdfs.createSnapshot(snapshottableDir, "s5"));
+
+// now change max snapshot directory limit to 2 and restart namenode
+cluster.getNameNode().getConf().setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_MAX_LIMIT, 2);
+cluster.restartNameNodes();
+SnapshotManager snapshotManager = cluster.getNamesystem().
+getSnapshotManager();
+
+// make sure edits of all previous 5 create snapshots are replayed
+Assert.assertEquals(numSnapshots, snapshotManager.getNumSnapshots());
+
+// make sure namenode has the new snapshot limit configured as 2
+Assert.assertEquals(2, snapshotManager.getMaxSnapshotLimit());
+
+// Any new snapshot creation should still fail
+LambdaTestUtils.intercept(SnapshotException.class,
+"snapshot limit", () -> hdfs.createSnapshot(snapshottableDir, "s5"));
+// now change max snapshot FS limit to 2 and restart namenode
+cluster.getNameNode().getConf().setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_FILESYSTEM_LIMIT, 2);
+cluster.restartNameNodes();
+snapshotManager = cluster.getNamesystem().
+getSnapshotManager();
+// make sure edits of all previous 5 create snapshots are replayed
+Assert.assertEquals(numSnapshots, snapshotManager.getNumSnapshots());
+
+// make sure namenode has the new snapshot limit configured as 2
+Assert.assertEquals(2, snapshotManager.getMaxSnapshotLimit());
+cluster.shutdown();
Review comment:
We may need to do it in a finally to make sure we always clean it. Also
check for null.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 484140)
Time Spent: 2h 40m (was: 2.5h)
> namenode start failed to start when dfs.namenode.snapshot.max.limit set
> ---
>
> Key: HDFS-15568
> URL: https://issues.apache.org/jira/browse/HDFS-15568
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Nilotpal Nandi
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> {code:java}
> 11:35:05.872 AM ERROR NameNode
> Failed to start namenode.
> org.apache.hadoop.hdfs.protocol.SnapshotException: Failed to add snapshot:
> there are already 20 snapshot(s) and the max snapshot limit is 20
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.DirectorySnapshottableFeature.addSnapshot(DirectorySnapshottableFeature.java:181)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeDirectory.addSnapshot(INodeDirectory.java:285)
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.createSnapshot(SnapshotManager.java:447)
> at
>
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[
https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=483523=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-483523
]
ASF GitHub Bot logged work on HDFS-15568:
-
Author: ASF GitHub Bot
Created on: 12/Sep/20 22:02
Start Date: 12/Sep/20 22:02
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2296:
URL: https://github.com/apache/hadoop/pull/2296#issuecomment-691065657
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 483523)
Time Spent: 2.5h (was: 2h 20m)
> namenode start failed to start when dfs.namenode.snapshot.max.limit set
> ---
>
> Key: HDFS-15568
> URL: https://issues.apache.org/jira/browse/HDFS-15568
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Nilotpal Nandi
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> {code:java}
> 11:35:05.872 AM ERROR NameNode
> Failed to start namenode.
> org.apache.hadoop.hdfs.protocol.SnapshotException: Failed to add snapshot:
> there are already 20 snapshot(s) and the max snapshot limit is 20
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.DirectorySnapshottableFeature.addSnapshot(DirectorySnapshottableFeature.java:181)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeDirectory.addSnapshot(INodeDirectory.java:285)
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.createSnapshot(SnapshotManager.java:447)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:802)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:287)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:182)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:912)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:760)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:337)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1164)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:755)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:646)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:717)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:960)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:933)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1670)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1737)
> {code}
> Steps to reproduce:
> --
> directory level snapshot limit set - 100
> Created 100 snapshots
> deleted all 100 snapshots (in-oder)
> No snapshot exist
> Then, directory level snapshot limit set - 20
> HDFS restart
> Namenode start failed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[
https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=483213=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-483213
]
ASF GitHub Bot logged work on HDFS-15568:
-
Author: ASF GitHub Bot
Created on: 12/Sep/20 21:32
Start Date: 12/Sep/20 21:32
Worklog Time Spent: 10m
Work Description: goiri commented on a change in pull request #2296:
URL: https://github.com/apache/hadoop/pull/2296#discussion_r487238144
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotManager.java
##
@@ -133,4 +137,58 @@ public void testValidateSnapshotIDWidth() throws Exception
{
getMaxSnapshotID() < Snapshot.CURRENT_STATE_ID);
}
+ @Test
+ public void testSnapshotLimitOnRestart() throws Exception {
+final Configuration conf = new Configuration();
+final Path snapshottableDir
+= new Path("/" + getClass().getSimpleName());
+int numSnapshots = 5;
+conf.setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_MAX_LIMIT, numSnapshots);
+conf.setInt(DFSConfigKeys.DFS_NAMENODE_SNAPSHOT_FILESYSTEM_LIMIT,
+numSnapshots * 2);
+MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).
+numDataNodes(0).build();
+cluster.waitActive();
+DistributedFileSystem hdfs = cluster.getFileSystem();
+hdfs.mkdirs(snapshottableDir);
+hdfs.allowSnapshot(snapshottableDir);
+int i = 0;
+for (; i < numSnapshots; i++) {
+ hdfs.createSnapshot(snapshottableDir, "s" + i);
+}
+LambdaTestUtils.intercept(SnapshotException.class,
+"snapshot limit",
+() -> hdfs.createSnapshot(snapshottableDir, "s5"));
+
+// now change max snapshot directory limit to 2 and restart namenode
+cluster.getNameNode().getConf().setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_MAX_LIMIT, 2);
+cluster.restartNameNodes();
+SnapshotManager snapshotManager = cluster.getNamesystem().
+getSnapshotManager();
+
+// make sure edits of all previous 5 create snapshots are replayed
+Assert.assertEquals(numSnapshots, snapshotManager.getNumSnapshots());
+
+// make sure namenode has the new snapshot limit configured as 2
+Assert.assertEquals(2, snapshotManager.getMaxSnapshotLimit());
+
+// Any new snapshot creation should still fail
+LambdaTestUtils.intercept(SnapshotException.class,
+"snapshot limit",
+() -> hdfs.createSnapshot(snapshottableDir, "s5"));
+// now change max snapshot FS limit to 2 and restart namenode
+cluster.getNameNode().getConf().setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_FILESYSTEM_LIMIT, 2);
+cluster.restartNameNodes();
+snapshotManager = cluster.getNamesystem().
Review comment:
1 line?
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotManager.java
##
@@ -133,4 +137,58 @@ public void testValidateSnapshotIDWidth() throws Exception
{
getMaxSnapshotID() < Snapshot.CURRENT_STATE_ID);
}
+ @Test
+ public void testSnapshotLimitOnRestart() throws Exception {
+final Configuration conf = new Configuration();
+final Path snapshottableDir
+= new Path("/" + getClass().getSimpleName());
+int numSnapshots = 5;
+conf.setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_MAX_LIMIT, numSnapshots);
+conf.setInt(DFSConfigKeys.DFS_NAMENODE_SNAPSHOT_FILESYSTEM_LIMIT,
+numSnapshots * 2);
+MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).
+numDataNodes(0).build();
+cluster.waitActive();
+DistributedFileSystem hdfs = cluster.getFileSystem();
+hdfs.mkdirs(snapshottableDir);
+hdfs.allowSnapshot(snapshottableDir);
+int i = 0;
Review comment:
Who uses this "i" later? in for loop def should be good.
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotManager.java
##
@@ -133,4 +137,58 @@ public void testValidateSnapshotIDWidth() throws Exception
{
getMaxSnapshotID() < Snapshot.CURRENT_STATE_ID);
}
+ @Test
+ public void testSnapshotLimitOnRestart() throws Exception {
+final Configuration conf = new Configuration();
+final Path snapshottableDir
+= new Path("/" + getClass().getSimpleName());
+int numSnapshots = 5;
+conf.setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_MAX_LIMIT, numSnapshots);
+conf.setInt(DFSConfigKeys.DFS_NAMENODE_SNAPSHOT_FILESYSTEM_LIMIT,
+numSnapshots * 2);
+MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).
Review comment:
should we clean this cluster?
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotManager.java
##
@@
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[
https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=483135=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-483135
]
ASF GitHub Bot logged work on HDFS-15568:
-
Author: ASF GitHub Bot
Created on: 12/Sep/20 21:19
Start Date: 12/Sep/20 21:19
Worklog Time Spent: 10m
Work Description: bshashikant commented on pull request #2296:
URL: https://github.com/apache/hadoop/pull/2296#issuecomment-690974334
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 483135)
Time Spent: 2h 10m (was: 2h)
> namenode start failed to start when dfs.namenode.snapshot.max.limit set
> ---
>
> Key: HDFS-15568
> URL: https://issues.apache.org/jira/browse/HDFS-15568
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Nilotpal Nandi
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> {code:java}
> 11:35:05.872 AM ERROR NameNode
> Failed to start namenode.
> org.apache.hadoop.hdfs.protocol.SnapshotException: Failed to add snapshot:
> there are already 20 snapshot(s) and the max snapshot limit is 20
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.DirectorySnapshottableFeature.addSnapshot(DirectorySnapshottableFeature.java:181)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeDirectory.addSnapshot(INodeDirectory.java:285)
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.createSnapshot(SnapshotManager.java:447)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:802)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:287)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:182)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:912)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:760)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:337)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1164)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:755)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:646)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:717)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:960)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:933)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1670)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1737)
> {code}
> Steps to reproduce:
> --
> directory level snapshot limit set - 100
> Created 100 snapshots
> deleted all 100 snapshots (in-oder)
> No snapshot exist
> Then, directory level snapshot limit set - 20
> HDFS restart
> Namenode start failed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[
https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=483117=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-483117
]
ASF GitHub Bot logged work on HDFS-15568:
-
Author: ASF GitHub Bot
Created on: 12/Sep/20 21:14
Start Date: 12/Sep/20 21:14
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2296:
URL: https://github.com/apache/hadoop/pull/2296#issuecomment-691065657
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 483117)
Time Spent: 2h (was: 1h 50m)
> namenode start failed to start when dfs.namenode.snapshot.max.limit set
> ---
>
> Key: HDFS-15568
> URL: https://issues.apache.org/jira/browse/HDFS-15568
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Nilotpal Nandi
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: pull-request-available
> Time Spent: 2h
> Remaining Estimate: 0h
>
> {code:java}
> 11:35:05.872 AM ERROR NameNode
> Failed to start namenode.
> org.apache.hadoop.hdfs.protocol.SnapshotException: Failed to add snapshot:
> there are already 20 snapshot(s) and the max snapshot limit is 20
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.DirectorySnapshottableFeature.addSnapshot(DirectorySnapshottableFeature.java:181)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeDirectory.addSnapshot(INodeDirectory.java:285)
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.createSnapshot(SnapshotManager.java:447)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:802)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:287)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:182)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:912)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:760)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:337)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1164)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:755)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:646)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:717)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:960)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:933)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1670)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1737)
> {code}
> Steps to reproduce:
> --
> directory level snapshot limit set - 100
> Created 100 snapshots
> deleted all 100 snapshots (in-oder)
> No snapshot exist
> Then, directory level snapshot limit set - 20
> HDFS restart
> Namenode start failed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[
https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=482806=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-482806
]
ASF GitHub Bot logged work on HDFS-15568:
-
Author: ASF GitHub Bot
Created on: 12/Sep/20 20:23
Start Date: 12/Sep/20 20:23
Worklog Time Spent: 10m
Work Description: goiri commented on a change in pull request #2296:
URL: https://github.com/apache/hadoop/pull/2296#discussion_r487238144
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotManager.java
##
@@ -133,4 +137,58 @@ public void testValidateSnapshotIDWidth() throws Exception
{
getMaxSnapshotID() < Snapshot.CURRENT_STATE_ID);
}
+ @Test
+ public void testSnapshotLimitOnRestart() throws Exception {
+final Configuration conf = new Configuration();
+final Path snapshottableDir
+= new Path("/" + getClass().getSimpleName());
+int numSnapshots = 5;
+conf.setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_MAX_LIMIT, numSnapshots);
+conf.setInt(DFSConfigKeys.DFS_NAMENODE_SNAPSHOT_FILESYSTEM_LIMIT,
+numSnapshots * 2);
+MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).
+numDataNodes(0).build();
+cluster.waitActive();
+DistributedFileSystem hdfs = cluster.getFileSystem();
+hdfs.mkdirs(snapshottableDir);
+hdfs.allowSnapshot(snapshottableDir);
+int i = 0;
+for (; i < numSnapshots; i++) {
+ hdfs.createSnapshot(snapshottableDir, "s" + i);
+}
+LambdaTestUtils.intercept(SnapshotException.class,
+"snapshot limit",
+() -> hdfs.createSnapshot(snapshottableDir, "s5"));
+
+// now change max snapshot directory limit to 2 and restart namenode
+cluster.getNameNode().getConf().setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_MAX_LIMIT, 2);
+cluster.restartNameNodes();
+SnapshotManager snapshotManager = cluster.getNamesystem().
+getSnapshotManager();
+
+// make sure edits of all previous 5 create snapshots are replayed
+Assert.assertEquals(numSnapshots, snapshotManager.getNumSnapshots());
+
+// make sure namenode has the new snapshot limit configured as 2
+Assert.assertEquals(2, snapshotManager.getMaxSnapshotLimit());
+
+// Any new snapshot creation should still fail
+LambdaTestUtils.intercept(SnapshotException.class,
+"snapshot limit",
+() -> hdfs.createSnapshot(snapshottableDir, "s5"));
+// now change max snapshot FS limit to 2 and restart namenode
+cluster.getNameNode().getConf().setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_FILESYSTEM_LIMIT, 2);
+cluster.restartNameNodes();
+snapshotManager = cluster.getNamesystem().
Review comment:
1 line?
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotManager.java
##
@@ -133,4 +137,58 @@ public void testValidateSnapshotIDWidth() throws Exception
{
getMaxSnapshotID() < Snapshot.CURRENT_STATE_ID);
}
+ @Test
+ public void testSnapshotLimitOnRestart() throws Exception {
+final Configuration conf = new Configuration();
+final Path snapshottableDir
+= new Path("/" + getClass().getSimpleName());
+int numSnapshots = 5;
+conf.setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_MAX_LIMIT, numSnapshots);
+conf.setInt(DFSConfigKeys.DFS_NAMENODE_SNAPSHOT_FILESYSTEM_LIMIT,
+numSnapshots * 2);
+MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).
+numDataNodes(0).build();
+cluster.waitActive();
+DistributedFileSystem hdfs = cluster.getFileSystem();
+hdfs.mkdirs(snapshottableDir);
+hdfs.allowSnapshot(snapshottableDir);
+int i = 0;
Review comment:
Who uses this "i" later? in for loop def should be good.
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotManager.java
##
@@ -133,4 +137,58 @@ public void testValidateSnapshotIDWidth() throws Exception
{
getMaxSnapshotID() < Snapshot.CURRENT_STATE_ID);
}
+ @Test
+ public void testSnapshotLimitOnRestart() throws Exception {
+final Configuration conf = new Configuration();
+final Path snapshottableDir
+= new Path("/" + getClass().getSimpleName());
+int numSnapshots = 5;
+conf.setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_MAX_LIMIT, numSnapshots);
+conf.setInt(DFSConfigKeys.DFS_NAMENODE_SNAPSHOT_FILESYSTEM_LIMIT,
+numSnapshots * 2);
+MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).
Review comment:
should we clean this cluster?
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotManager.java
##
@@
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[
https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=482722=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-482722
]
ASF GitHub Bot logged work on HDFS-15568:
-
Author: ASF GitHub Bot
Created on: 12/Sep/20 20:15
Start Date: 12/Sep/20 20:15
Worklog Time Spent: 10m
Work Description: bshashikant commented on pull request #2296:
URL: https://github.com/apache/hadoop/pull/2296#issuecomment-690974334
/retest
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 482722)
Time Spent: 1h 40m (was: 1.5h)
> namenode start failed to start when dfs.namenode.snapshot.max.limit set
> ---
>
> Key: HDFS-15568
> URL: https://issues.apache.org/jira/browse/HDFS-15568
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Nilotpal Nandi
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> {code:java}
> 11:35:05.872 AM ERROR NameNode
> Failed to start namenode.
> org.apache.hadoop.hdfs.protocol.SnapshotException: Failed to add snapshot:
> there are already 20 snapshot(s) and the max snapshot limit is 20
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.DirectorySnapshottableFeature.addSnapshot(DirectorySnapshottableFeature.java:181)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeDirectory.addSnapshot(INodeDirectory.java:285)
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.createSnapshot(SnapshotManager.java:447)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:802)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:287)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:182)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:912)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:760)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:337)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1164)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:755)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:646)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:717)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:960)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:933)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1670)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1737)
> {code}
> Steps to reproduce:
> --
> directory level snapshot limit set - 100
> Created 100 snapshots
> deleted all 100 snapshots (in-oder)
> No snapshot exist
> Then, directory level snapshot limit set - 20
> HDFS restart
> Namenode start failed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[
https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=482702=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-482702
]
ASF GitHub Bot logged work on HDFS-15568:
-
Author: ASF GitHub Bot
Created on: 12/Sep/20 20:13
Start Date: 12/Sep/20 20:13
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #2296:
URL: https://github.com/apache/hadoop/pull/2296#issuecomment-691065657
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 482702)
Time Spent: 1.5h (was: 1h 20m)
> namenode start failed to start when dfs.namenode.snapshot.max.limit set
> ---
>
> Key: HDFS-15568
> URL: https://issues.apache.org/jira/browse/HDFS-15568
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Nilotpal Nandi
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> {code:java}
> 11:35:05.872 AM ERROR NameNode
> Failed to start namenode.
> org.apache.hadoop.hdfs.protocol.SnapshotException: Failed to add snapshot:
> there are already 20 snapshot(s) and the max snapshot limit is 20
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.DirectorySnapshottableFeature.addSnapshot(DirectorySnapshottableFeature.java:181)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeDirectory.addSnapshot(INodeDirectory.java:285)
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.createSnapshot(SnapshotManager.java:447)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:802)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:287)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:182)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:912)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:760)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:337)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1164)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:755)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:646)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:717)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:960)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:933)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1670)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1737)
> {code}
> Steps to reproduce:
> --
> directory level snapshot limit set - 100
> Created 100 snapshots
> deleted all 100 snapshots (in-oder)
> No snapshot exist
> Then, directory level snapshot limit set - 20
> HDFS restart
> Namenode start failed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[
https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=482242=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-482242
]
ASF GitHub Bot logged work on HDFS-15568:
-
Author: ASF GitHub Bot
Created on: 11/Sep/20 19:13
Start Date: 11/Sep/20 19:13
Worklog Time Spent: 10m
Work Description: goiri commented on a change in pull request #2296:
URL: https://github.com/apache/hadoop/pull/2296#discussion_r487238144
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotManager.java
##
@@ -133,4 +137,58 @@ public void testValidateSnapshotIDWidth() throws Exception
{
getMaxSnapshotID() < Snapshot.CURRENT_STATE_ID);
}
+ @Test
+ public void testSnapshotLimitOnRestart() throws Exception {
+final Configuration conf = new Configuration();
+final Path snapshottableDir
+= new Path("/" + getClass().getSimpleName());
+int numSnapshots = 5;
+conf.setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_MAX_LIMIT, numSnapshots);
+conf.setInt(DFSConfigKeys.DFS_NAMENODE_SNAPSHOT_FILESYSTEM_LIMIT,
+numSnapshots * 2);
+MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).
+numDataNodes(0).build();
+cluster.waitActive();
+DistributedFileSystem hdfs = cluster.getFileSystem();
+hdfs.mkdirs(snapshottableDir);
+hdfs.allowSnapshot(snapshottableDir);
+int i = 0;
+for (; i < numSnapshots; i++) {
+ hdfs.createSnapshot(snapshottableDir, "s" + i);
+}
+LambdaTestUtils.intercept(SnapshotException.class,
+"snapshot limit",
+() -> hdfs.createSnapshot(snapshottableDir, "s5"));
+
+// now change max snapshot directory limit to 2 and restart namenode
+cluster.getNameNode().getConf().setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_MAX_LIMIT, 2);
+cluster.restartNameNodes();
+SnapshotManager snapshotManager = cluster.getNamesystem().
+getSnapshotManager();
+
+// make sure edits of all previous 5 create snapshots are replayed
+Assert.assertEquals(numSnapshots, snapshotManager.getNumSnapshots());
+
+// make sure namenode has the new snapshot limit configured as 2
+Assert.assertEquals(2, snapshotManager.getMaxSnapshotLimit());
+
+// Any new snapshot creation should still fail
+LambdaTestUtils.intercept(SnapshotException.class,
+"snapshot limit",
+() -> hdfs.createSnapshot(snapshottableDir, "s5"));
+// now change max snapshot FS limit to 2 and restart namenode
+cluster.getNameNode().getConf().setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_FILESYSTEM_LIMIT, 2);
+cluster.restartNameNodes();
+snapshotManager = cluster.getNamesystem().
Review comment:
1 line?
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotManager.java
##
@@ -133,4 +137,58 @@ public void testValidateSnapshotIDWidth() throws Exception
{
getMaxSnapshotID() < Snapshot.CURRENT_STATE_ID);
}
+ @Test
+ public void testSnapshotLimitOnRestart() throws Exception {
+final Configuration conf = new Configuration();
+final Path snapshottableDir
+= new Path("/" + getClass().getSimpleName());
+int numSnapshots = 5;
+conf.setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_MAX_LIMIT, numSnapshots);
+conf.setInt(DFSConfigKeys.DFS_NAMENODE_SNAPSHOT_FILESYSTEM_LIMIT,
+numSnapshots * 2);
+MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).
+numDataNodes(0).build();
+cluster.waitActive();
+DistributedFileSystem hdfs = cluster.getFileSystem();
+hdfs.mkdirs(snapshottableDir);
+hdfs.allowSnapshot(snapshottableDir);
+int i = 0;
Review comment:
Who uses this "i" later? in for loop def should be good.
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotManager.java
##
@@ -133,4 +137,58 @@ public void testValidateSnapshotIDWidth() throws Exception
{
getMaxSnapshotID() < Snapshot.CURRENT_STATE_ID);
}
+ @Test
+ public void testSnapshotLimitOnRestart() throws Exception {
+final Configuration conf = new Configuration();
+final Path snapshottableDir
+= new Path("/" + getClass().getSimpleName());
+int numSnapshots = 5;
+conf.setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_MAX_LIMIT, numSnapshots);
+conf.setInt(DFSConfigKeys.DFS_NAMENODE_SNAPSHOT_FILESYSTEM_LIMIT,
+numSnapshots * 2);
+MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).
Review comment:
should we clean this cluster?
This is an automated message from the Apache Git Service.
To respond to the message, please
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[ https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=482162=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-482162 ] ASF GitHub Bot logged work on HDFS-15568: - Author: ASF GitHub Bot Created on: 11/Sep/20 16:35 Start Date: 11/Sep/20 16:35 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2296: URL: https://github.com/apache/hadoop/pull/2296#issuecomment-691196975 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 31m 51s | trunk passed | | +1 :green_heart: | compile | 1m 17s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | compile | 1m 10s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | checkstyle | 0m 50s | trunk passed | | +1 :green_heart: | mvnsite | 1m 15s | trunk passed | | +1 :green_heart: | shadedclient | 17m 51s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 51s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 1m 25s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +0 :ok: | spotbugs | 3m 8s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 6s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 8s | the patch passed | | +1 :green_heart: | compile | 1m 11s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javac | 1m 11s | the patch passed | | +1 :green_heart: | compile | 1m 4s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | javac | 1m 4s | the patch passed | | +1 :green_heart: | checkstyle | 0m 41s | the patch passed | | +1 :green_heart: | mvnsite | 1m 10s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedclient | 15m 34s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 45s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 1m 16s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | findbugs | 3m 11s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 112m 34s | hadoop-hdfs in the patch passed. | | +1 :green_heart: | asflicense | 0m 37s | The patch does not generate ASF License warnings. | | | | 201m 39s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier | | | hadoop.hdfs.TestGetFileChecksum | | | hadoop.hdfs.TestFileChecksumCompositeCrc | | | hadoop.hdfs.server.namenode.snapshot.TestSnapshotManager | | | hadoop.hdfs.TestBlockStoragePolicy | | | hadoop.hdfs.TestSnapshotCommands | | | hadoop.hdfs.TestFileChecksum | | | hadoop.hdfs.TestDFSOutputStream | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2296/3/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2296 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux b21b996bec02 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 2a6fdcedeb4 | | Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | unit |
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[ https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=482059=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-482059 ] ASF GitHub Bot logged work on HDFS-15568: - Author: ASF GitHub Bot Created on: 11/Sep/20 12:31 Start Date: 11/Sep/20 12:31 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2296: URL: https://github.com/apache/hadoop/pull/2296#issuecomment-691065657 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 12s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 33m 38s | trunk passed | | +1 :green_heart: | compile | 1m 16s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | compile | 1m 8s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | checkstyle | 0m 47s | trunk passed | | +1 :green_heart: | mvnsite | 1m 16s | trunk passed | | +1 :green_heart: | shadedclient | 17m 44s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 50s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 1m 18s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +0 :ok: | spotbugs | 3m 7s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 5s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 8s | the patch passed | | +1 :green_heart: | compile | 1m 11s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javac | 1m 11s | the patch passed | | +1 :green_heart: | compile | 1m 2s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | javac | 1m 2s | the patch passed | | -0 :warning: | checkstyle | 0m 43s | hadoop-hdfs-project/hadoop-hdfs: The patch generated 1 new + 89 unchanged - 0 fixed = 90 total (was 89) | | +1 :green_heart: | mvnsite | 1m 8s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedclient | 15m 41s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 45s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 1m 16s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | findbugs | 3m 10s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 108m 47s | hadoop-hdfs in the patch passed. | | +1 :green_heart: | asflicense | 0m 36s | The patch does not generate ASF License warnings. | | | | 199m 30s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier | | | hadoop.hdfs.TestFileChecksum | | | hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes | | | hadoop.hdfs.TestSnapshotCommands | | | hadoop.hdfs.server.namenode.ha.TestBootstrapAliasmap | | | hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits | | | hadoop.hdfs.server.namenode.snapshot.TestSnapshotManager | | | hadoop.hdfs.TestFileChecksumCompositeCrc | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2296/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2296 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux aa36c5473309 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / 89428f142fe | | Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[
https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=481985=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481985
]
ASF GitHub Bot logged work on HDFS-15568:
-
Author: ASF GitHub Bot
Created on: 11/Sep/20 09:11
Start Date: 11/Sep/20 09:11
Worklog Time Spent: 10m
Work Description: bshashikant commented on pull request #2296:
URL: https://github.com/apache/hadoop/pull/2296#issuecomment-690974334
/retest
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 481985)
Time Spent: 50m (was: 40m)
> namenode start failed to start when dfs.namenode.snapshot.max.limit set
> ---
>
> Key: HDFS-15568
> URL: https://issues.apache.org/jira/browse/HDFS-15568
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Nilotpal Nandi
>Assignee: Shashikant Banerjee
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> {code:java}
> 11:35:05.872 AM ERROR NameNode
> Failed to start namenode.
> org.apache.hadoop.hdfs.protocol.SnapshotException: Failed to add snapshot:
> there are already 20 snapshot(s) and the max snapshot limit is 20
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.DirectorySnapshottableFeature.addSnapshot(DirectorySnapshottableFeature.java:181)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeDirectory.addSnapshot(INodeDirectory.java:285)
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.createSnapshot(SnapshotManager.java:447)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:802)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:287)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:182)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:912)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:760)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:337)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1164)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:755)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:646)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:717)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:960)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:933)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1670)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1737)
> {code}
> Steps to reproduce:
> --
> directory level snapshot limit set - 100
> Created 100 snapshots
> deleted all 100 snapshots (in-oder)
> No snapshot exist
> Then, directory level snapshot limit set - 20
> HDFS restart
> Namenode start failed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[
https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=481668=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481668
]
ASF GitHub Bot logged work on HDFS-15568:
-
Author: ASF GitHub Bot
Created on: 10/Sep/20 19:49
Start Date: 10/Sep/20 19:49
Worklog Time Spent: 10m
Work Description: szetszwo commented on a change in pull request #2296:
URL: https://github.com/apache/hadoop/pull/2296#discussion_r486594478
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/SnapshotManager.java
##
@@ -448,22 +455,31 @@ public String createSnapshot(final LeaseManager
leaseManager,
"snapshot IDs and ID rollover is not supported.");
}
int n = numSnapshots.get();
-if (n >= maxSnapshotFSLimit) {
- // We have reached the maximum snapshot limit
- throw new SnapshotException(
- "Failed to create snapshot: there are already " + (n + 1)
- + " snapshot(s) and the max snapshot limit is "
- + maxSnapshotFSLimit);
-}
-
-srcRoot.addSnapshot(snapshotCounter, snapshotName, leaseManager,
-this.captureOpenFiles, maxSnapshotLimit, mtime);
+checkSnapshotLimit(maxSnapshotFSLimit, n);
+srcRoot.addSnapshot(this, snapshotName, leaseManager, mtime);
//create success, update id
snapshotCounter++;
numSnapshots.getAndIncrement();
return Snapshot.getSnapshotPath(snapshotRoot, snapshotName);
}
+
+ void checkSnapshotLimit(int limit, int numSnapshots)
Review comment:
I suggest to add limit type to the error message as below.
```
diff --git
a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/DirectorySnapshottableFeature.java
b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/DirectorySnapshottableFeature.java
index 266c0a71241..7a47ab4000d 100644
---
a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/DirectorySnapshottableFeature.java
+++
b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/DirectorySnapshottableFeature.java
@@ -190,8 +190,7 @@ public Snapshot addSnapshot(INodeDirectory snapshotRoot,
+ n + " snapshot(s) and the snapshot quota is "
+ snapshotQuota);
}
-snapshotManager.checkSnapshotLimit(snapshotManager.
-getMaxSnapshotLimit(), n);
+snapshotManager.checkPerDirectorySnapshotLimit(n);
final Snapshot s = new Snapshot(id, name, snapshotRoot);
final byte[] nameBytes = s.getRoot().getLocalNameBytes();
final int i = searchSnapshot(nameBytes);
diff --git
a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/SnapshotManager.java
b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/SnapshotManager.java
index 0a2e18c3dc3..7c482074486 100644
---
a/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/SnapshotManager.java
+++
b/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/SnapshotManager.java
@@ -455,7 +455,7 @@ public String createSnapshot(final LeaseManager
leaseManager,
"snapshot IDs and ID rollover is not supported.");
}
int n = numSnapshots.get();
-checkSnapshotLimit(maxSnapshotFSLimit, n);
+checkFileSystemSnapshotLimit(n);
srcRoot.addSnapshot(this, snapshotName, leaseManager, mtime);
//create success, update id
@@ -464,12 +464,19 @@ public String createSnapshot(final LeaseManager
leaseManager,
return Snapshot.getSnapshotPath(snapshotRoot, snapshotName);
}
- void checkSnapshotLimit(int limit, int numSnapshots)
- throws SnapshotException {
+ void checkFileSystemSnapshotLimit(int n) throws SnapshotException {
+checkSnapshotLimit(maxSnapshotFSLimit, n, "file system");
+ }
+
+ void checkPerDirectorySnapshotLimit(int n) throws SnapshotException {
+checkSnapshotLimit(maxSnapshotLimit, n, "per directory");
+ }
+
+ private void checkSnapshotLimit(int limit, int numSnapshots,
+ String type) throws SnapshotException {
if (numSnapshots >= limit) {
- String msg = "there are already " + (numSnapshots + 1)
- + " snapshot(s) and the max snapshot limit is "
- + limit;
+ String msg = "There are already " + (numSnapshots + 1)
+ + " snapshot(s) and the " + type + " snapshot limit is " + limit;
if (fsdir.isImageLoaded()) {
// We have reached the maximum snapshot limit
throw new SnapshotException(
```
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[
https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=481660=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481660
]
ASF GitHub Bot logged work on HDFS-15568:
-
Author: ASF GitHub Bot
Created on: 10/Sep/20 19:21
Start Date: 10/Sep/20 19:21
Worklog Time Spent: 10m
Work Description: goiri commented on a change in pull request #2296:
URL: https://github.com/apache/hadoop/pull/2296#discussion_r486579348
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirectory.java
##
@@ -508,6 +508,10 @@ FSNamesystem getFSNamesystem() {
return namesystem;
}
+ public boolean isImageLoaded() {
Review comment:
Add javadoc
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/SnapshotManager.java
##
@@ -368,6 +368,13 @@ void assertFirstSnapshot(INodeDirectory dir,
}
}
+ boolean captureOpenFiles() {
Review comment:
javadoc
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/SnapshotManager.java
##
@@ -368,6 +368,13 @@ void assertFirstSnapshot(INodeDirectory dir,
}
}
+ boolean captureOpenFiles() {
+return captureOpenFiles;
+ }
+
+ int getMaxSnapshotLimit() {
Review comment:
VisibleForTesting
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotManager.java
##
@@ -133,4 +137,68 @@ public void testValidateSnapshotIDWidth() throws Exception
{
getMaxSnapshotID() < Snapshot.CURRENT_STATE_ID);
}
+ @Test
+ public void SnapshotLimitOnRestart() throws Exception {
+final Configuration conf = new Configuration();
+final Path snapshottableDir
+= new Path("/" + getClass().getSimpleName());
+int numSnapshots = 5;
+conf.setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_MAX_LIMIT, numSnapshots);
+conf.setInt(DFSConfigKeys.DFS_NAMENODE_SNAPSHOT_FILESYSTEM_LIMIT,
+numSnapshots * 2);
+MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).
+numDataNodes(0).build();
+cluster.waitActive();
+DistributedFileSystem hdfs = cluster.getFileSystem();
+hdfs.mkdirs(snapshottableDir);
+hdfs.allowSnapshot(snapshottableDir);
+int i = 0;
+for (; i < numSnapshots; i++) {
+ hdfs.createSnapshot(snapshottableDir, "s" + i);
+}
+try {
+ hdfs.createSnapshot(snapshottableDir, "s" + i);
+ Assert.fail("Expected SnapshotException not thrown");
+} catch (SnapshotException se) {
+ Assert.assertTrue(
+ StringUtils.toLowerCase(se.getMessage()).contains(
+ "max snapshot limit"));
+}
+
+// now change max snapshot directory limit to 2 and restart namenode
+cluster.getNameNode().getConf().setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_MAX_LIMIT, 2);
+cluster.restartNameNodes();
+
+// make sure edits of all previous 5 create snapshots are replayed
+Assert.assertEquals(numSnapshots, cluster.getNamesystem().
+getSnapshotManager().getNumSnapshots());
+
+// make sure namenode has the new snapshot limit configured as 2
+Assert.assertEquals(2,
+cluster.getNamesystem().getSnapshotManager().getMaxSnapshotLimit());
+
+// Any new snapshot creation should still fail
+try {
+ hdfs.createSnapshot(snapshottableDir, "s" + i);
+ Assert.fail("Expected SnapshotException not thrown");
+} catch (SnapshotException se) {
+ Assert.assertTrue(
Review comment:
LambdaTestUtils
##
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotManager.java
##
@@ -133,4 +137,68 @@ public void testValidateSnapshotIDWidth() throws Exception
{
getMaxSnapshotID() < Snapshot.CURRENT_STATE_ID);
}
+ @Test
+ public void SnapshotLimitOnRestart() throws Exception {
+final Configuration conf = new Configuration();
+final Path snapshottableDir
+= new Path("/" + getClass().getSimpleName());
+int numSnapshots = 5;
+conf.setInt(DFSConfigKeys.
+DFS_NAMENODE_SNAPSHOT_MAX_LIMIT, numSnapshots);
+conf.setInt(DFSConfigKeys.DFS_NAMENODE_SNAPSHOT_FILESYSTEM_LIMIT,
+numSnapshots * 2);
+MiniDFSCluster cluster = new MiniDFSCluster.Builder(conf).
+numDataNodes(0).build();
+cluster.waitActive();
+DistributedFileSystem hdfs = cluster.getFileSystem();
+hdfs.mkdirs(snapshottableDir);
+hdfs.allowSnapshot(snapshottableDir);
+int i = 0;
+for (; i < numSnapshots; i++) {
+ hdfs.createSnapshot(snapshottableDir, "s" + i);
+}
+try {
+ hdfs.createSnapshot(snapshottableDir, "s" + i);
+ Assert.fail("Expected
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[ https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=481397=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481397 ] ASF GitHub Bot logged work on HDFS-15568: - Author: ASF GitHub Bot Created on: 10/Sep/20 11:49 Start Date: 10/Sep/20 11:49 Worklog Time Spent: 10m Work Description: hadoop-yetus commented on pull request #2296: URL: https://github.com/apache/hadoop/pull/2296#issuecomment-690215954 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 40m 35s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ trunk Compile Tests _ | | +1 :green_heart: | mvninstall | 32m 7s | trunk passed | | +1 :green_heart: | compile | 1m 16s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | compile | 1m 8s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | checkstyle | 0m 47s | trunk passed | | +1 :green_heart: | mvnsite | 1m 15s | trunk passed | | +1 :green_heart: | shadedclient | 17m 51s | branch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 50s | trunk passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 1m 19s | trunk passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +0 :ok: | spotbugs | 3m 13s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 11s | trunk passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 8s | the patch passed | | +1 :green_heart: | compile | 1m 11s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javac | 1m 11s | the patch passed | | +1 :green_heart: | compile | 1m 1s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | javac | 1m 1s | the patch passed | | -0 :warning: | checkstyle | 0m 40s | hadoop-hdfs-project/hadoop-hdfs: The patch generated 3 new + 89 unchanged - 0 fixed = 92 total (was 89) | | +1 :green_heart: | mvnsite | 1m 10s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedclient | 15m 27s | patch has no errors when building and testing our client artifacts. | | +1 :green_heart: | javadoc | 0m 46s | the patch passed with JDK Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 | | +1 :green_heart: | javadoc | 1m 16s | the patch passed with JDK Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | +1 :green_heart: | findbugs | 3m 27s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 80m 10s | hadoop-hdfs in the patch passed. | | -1 :x: | asflicense | 0m 35s | The patch generated 4 ASF License warnings. | | | | 209m 18s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hdfs.server.sps.TestExternalStoragePolicySatisfier | | | hadoop.fs.viewfs.TestViewFSOverloadSchemeWithMountTableConfigInHDFS | | | hadoop.fs.viewfs.TestViewFileSystemHdfs | | | hadoop.fs.viewfs.TestViewFileSystemLinkMergeSlash | | | hadoop.hdfs.TestSnapshotCommands | | | hadoop.fs.viewfs.TestViewFsAtHdfsRoot | | | hadoop.hdfs.TestFileChecksumCompositeCrc | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2296/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hadoop/pull/2296 | | Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 94a92cfd8c29 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/bin/hadoop.sh | | git revision | trunk / e5fe3262702 | | Default Java | Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | Multi-JDK versions | /usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.8+10-post-Ubuntu-0ubuntu118.04.1 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_265-8u265-b01-0ubuntu2~18.04-b01 | | checkstyle |
[jira] [Work logged] (HDFS-15568) namenode start failed to start when dfs.namenode.snapshot.max.limit set
[
https://issues.apache.org/jira/browse/HDFS-15568?focusedWorklogId=481302=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-481302
]
ASF GitHub Bot logged work on HDFS-15568:
-
Author: ASF GitHub Bot
Created on: 10/Sep/20 08:19
Start Date: 10/Sep/20 08:19
Worklog Time Spent: 10m
Work Description: bshashikant opened a new pull request #2296:
URL: https://github.com/apache/hadoop/pull/2296
please see https://issues.apache.org/jira/browse/HDFS-15568
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 481302)
Remaining Estimate: 0h
Time Spent: 10m
> namenode start failed to start when dfs.namenode.snapshot.max.limit set
> ---
>
> Key: HDFS-15568
> URL: https://issues.apache.org/jira/browse/HDFS-15568
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: snapshots
>Reporter: Nilotpal Nandi
>Assignee: Shashikant Banerjee
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code:java}
> 11:35:05.872 AM ERROR NameNode
> Failed to start namenode.
> org.apache.hadoop.hdfs.protocol.SnapshotException: Failed to add snapshot:
> there are already 20 snapshot(s) and the max snapshot limit is 20
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.DirectorySnapshottableFeature.addSnapshot(DirectorySnapshottableFeature.java:181)
> at
> org.apache.hadoop.hdfs.server.namenode.INodeDirectory.addSnapshot(INodeDirectory.java:285)
> at
> org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.createSnapshot(SnapshotManager.java:447)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.applyEditLogOp(FSEditLogLoader.java:802)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:287)
> at
> org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:182)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadEdits(FSImage.java:912)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.loadFSImage(FSImage.java:760)
> at
> org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:337)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1164)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:755)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:646)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:717)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:960)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.(NameNode.java:933)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1670)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1737)
> {code}
> Steps to reproduce:
> --
> directory level snapshot limit set - 100
> Created 100 snapshots
> deleted all 100 snapshots (in-oder)
> No snapshot exist
> Then, directory level snapshot limit set - 20
> HDFS restart
> Namenode start failed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
