[
https://issues.apache.org/jira/browse/HDFS-15614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17319274#comment-17319274
]
Ayush Saxena commented on HDFS-15614:
-------------------------------------
Thanx [~shashikant] for the responses, I will bother you a bit more:
{quote}If a new directory is made snapshottable with feature flahg turned ,
.Trash directory gets created implicitly as a part of allowSnapshot call. I
don't think there is an ambiguity here.
{quote}
I think I cleared this up in my question itself, So here is a test for that:
{code:java}
@Test
public void testClientAmbiguity() throws Exception {
Configuration conf = new HdfsConfiguration();
// Enable the feature
conf.setBoolean("dfs.namenode.snapshot.trashroot.enabled", true);
try (MiniDFSCluster cluster =
new MiniDFSCluster.Builder(conf).build()) {
cluster.waitActive();
final DistributedFileSystem dfs = cluster.getFileSystem();
// Create two directories, on 1 allowSnapshot through DFS & on other
// from HDFS ADMIN
Path dir1 = new Path("/dir1");
Path dir2 = new Path("/dir2");
dfs.mkdirs(dir1);
dfs.mkdirs(dir2);
// AllowSnapshot on dir1 through dfs
dfs.allowSnapshot(dir1);
// AllowSnapshot on dir2 through dfsadmin
DFSAdmin dfsAdmin = new DFSAdmin(conf);
ToolRunner.run(conf, dfsAdmin,new String[]{"-allowSnapshot",
dir2.toString()});
// Check for trash directory in dir1(allowed through dfs)
assertFalse(dfs.exists(new Path(dir1,FileSystem.TRASH_PREFIX))); // (1)
// Check for trash directory in dir2(allowed through DfsAdmin)
assertTrue(dfs.exists(new Path(dir2,FileSystem.TRASH_PREFIX)));
// Failover/Restart namenode and stuff
cluster.restartNameNodes();
cluster.waitActive();
// Nothing should change
// Check for trash directory in dir1(allowed through dfs)
// Will fail here. stuff changed post restart of namenode. Such thing
// will happen with uupgrade as well.
assertFalse(dfs.exists(new Path(dir1,FileSystem.TRASH_PREFIX))); // (1)
assertTrue(dfs.exists(new Path(dir2,FileSystem.TRASH_PREFIX)));
}
}
{code}
And this fails, And yep there is an ambiguity.
{quote}This is important for provisioning snapshot trash to use ordered
snapshot deletion feature if the system already had pre existing snapshottable
directories.
{quote}
How come a client side feature that important, that can make the cluster go
down in times of critical situation like failover, Again a test to show that:
{code:java}
@Test
public void testFailureAfterFailoverOrRestart() throws Exception {
Configuration conf = new HdfsConfiguration();
// Enable the feature
conf.setBoolean("dfs.namenode.snapshot.trashroot.enabled", true);
try (MiniDFSCluster cluster =
new MiniDFSCluster.Builder(conf).build()) {
cluster.waitActive();
final DistributedFileSystem dfs = cluster.getFileSystem();
// Create a directory
Path dir1 = new Path("/dir1");
dfs.mkdirs(dir1);
// AllowSnapshot on dir1
dfs.allowSnapshot(dir1);
// SetQuota
dfs.setQuota(dir1, 1, 1);
// Check if the cluster is working and happy.
dfs.mkdirs(new Path("/dir2"));
assertTrue(dfs.exists(new Path("/dir2")));
// Failover/Restart namenode or such stuff
cluster.restartNameNodes(); // Namenode Crashed, It was failover, then
// standby would also crash & ultimately whole of cluster.
// Will not reach here itself. :-(
cluster.waitActive();
dfs.listStatus(new Path("/dir1"));
}
}
{code}
{quote}The "getAllSnapshottableDirs()" in itslef is not a heavy call IMO. It
does not depend on the no of snapshots present in the system.
{quote}
Ok If you say, getAllSnapshottableDirs() might not be heavy, even if there are
tons of snapshottable directories, So, getFileInfo for all these directories
and then mkdirs for all these in worst case.
So a normal scenario is like:
1 call getAllSnapshottableDirs -> say fetches 2 million dirs
2 million getFileInfo() calls -> say avg case 1 million doesn't have trash
1 million mkdirs() -> a write call isn't considered cheap & fast you go to JNs
and stuff.
If you get this creation of snapshots in the filesystem spec as well, still you
won't get rid of any of these problems,
Nevertheless, what ever be the case a normal running cluster shouldn't crash
due to any feature, and that too during failovers, that is some crazy stuff.
And regarding encryption zone stuff are you talking it is similar to
HDFS-10324(I see this only linked on HDFS-15607)? Well I don't think it is
doing create like stuff during startup. Will see if [~weichiu] can confirm
that, He worked on it. Didn't dig in much though
Well not very sure of the use case and things here, so would leave it to you
guys. Please don't hold anything for me in the context, might not be able to
follow up.
Disc: Test ran on top of your PR 2881, since that fixes a bug in this code.
Just dropping an FYI. to some more folks if they have any comments on such
behaviour. Namenode calling mkdirs and other API to itself during startup/
failover/ leave safemode and ultimately crashing in some situations......
cc. [~aajisaka] [~kihwal] [~liuml07] [~elgoiri] [~hexiaoqiao]
> Initialize snapshot trash root during NameNode startup if enabled
> -----------------------------------------------------------------
>
> Key: HDFS-15614
> URL: https://issues.apache.org/jira/browse/HDFS-15614
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> This is a follow-up to HDFS-15607.
> Goal:
> Initialize (create) snapshot trash root for all existing snapshottable
> directories if {{dfs.namenode.snapshot.trashroot.enabled}} is set to
> {{true}}. So admins won't have to run {{dfsadmin -provisionTrash}} manually
> on all those existing snapshottable directories.
> The change is expected to land in {{FSNamesystem}}.
> Discussion:
> 1. Currently in HDFS-15607, the snapshot trash root creation logic is on the
> client side. But in order for NN to create it at startup, the logic must
> (also) be implemented on the server side as well. -- which is also a
> requirement by WebHDFS (HDFS-15612).
> 2. Alternatively, we can provide an extra parameter to the
> {{-provisionTrash}} command like: {{dfsadmin -provisionTrash -all}} to
> initialize/provision trash root on all existing snapshottable dirs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
