[
https://issues.apache.org/jira/browse/HDFS-15614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17320927#comment-17320927
]
Siyao Meng edited comment on HDFS-15614 at 4/14/21, 11:31 AM:
--------------------------------------------------------------
Thanks for bringing this up [~ayushtkn].
{quote}
And this fails, And yep there is an ambiguity.
{quote}
The reason is that
[{{DFS#provisionSnapshotTrash}}|https://github.com/apache/hadoop/blob/c6539e3289711d29f508930bbda40302f48ddf4c/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java#L2984]
followed EZ counterpart's
[{{DFS#provisionEZTrash}}|https://github.com/apache/hadoop/blob/c6539e3289711d29f508930bbda40302f48ddf4c/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java#L2913]
implementation. {{dfs.provisionSnapshotTrash}} is not automatically called
from {{dfs.allowSnapshot}}, following what encryption zone has done similarly.
Therefore this would be the same deal for encryption zone trash root creation.
When replacing {{dfs.allowSnapshot}} calls with {{dfs.createEncryptionZone}} in
the first test case we should also find trash root inside encryption zone
missing with {{dfs.provisionSnapshotTrash}} call *alone*.
I suggest some guidelines should be posted and in javadoc adding that
allowSnapshot should better performed with dfsadmin CLI (and
createEncryptionZone if there aren't already).
{quote}
How come a client side feature that important, that can make the cluster go
down in times of critical situation like failover, Again a test to show that:
{quote}
[name
quota|https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsQuotaAdminGuide.html#Name_Quotas]
can become an issue indeed.
I think I got your point. Maybe a better way to create those necessary Trash
dirs is to ask an admin to run dfsadmin command *manually* after flipping
{{dfs.namenode.snapshot.trashroot.enabled}} to {{true}}.
Currently we already have {{dfsadmin -provisionSnapshotTrash}} but can only be
done one by one. {{dfsadmin -provisionSnapshotTrash -all}} can be implemented
to achieve this.
Cheers,
Siyao
was (Author: smeng):
Thanks for bringing this up [~ayushtkn].
{quote}
And this fails, And yep there is an ambiguity.
{quote}
The reason is that
[{{DFS#provisionSnapshotTrash}}|https://github.com/apache/hadoop/blob/c6539e3289711d29f508930bbda40302f48ddf4c/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java#L2984]
followed EZ counterpart's
[{{DFS#provisionEZTrash}}|https://github.com/apache/hadoop/blob/c6539e3289711d29f508930bbda40302f48ddf4c/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java#L2913]
implementation. {{dfs.provisionSnapshotTrash}} is not automatically called
from {{dfs.allowSnapshot}}, following what .
Therefore this would be the same deal for encryption zone trash root creation.
When replacing {{dfs.allowSnapshot}} calls with {{dfs.createEncryptionZone}} in
the first test case we should also find trash root inside encryption zone
missing with {{dfs.provisionSnapshotTrash}} call *alone*.
I suggest some guidelines should be posted and in javadoc adding that
allowSnapshot should better performed with dfsadmin CLI (and
createEncryptionZone if there aren't already).
{quote}
How come a client side feature that important, that can make the cluster go
down in times of critical situation like failover, Again a test to show that:
{quote}
[name
quota|https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsQuotaAdminGuide.html#Name_Quotas]
can become an issue indeed.
I think I got your point. Maybe a better way to create those necessary Trash
dirs is to ask an admin to run dfsadmin command *manually* after flipping
{{dfs.namenode.snapshot.trashroot.enabled}} to {{true}}.
Currently we already have {{dfsadmin -provisionSnapshotTrash}} but can only be
done one by one. {{dfsadmin -provisionSnapshotTrash -all}} can be implemented
to achieve this.
Cheers,
Siyao
> Initialize snapshot trash root during NameNode startup if enabled
> -----------------------------------------------------------------
>
> Key: HDFS-15614
> URL: https://issues.apache.org/jira/browse/HDFS-15614
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> This is a follow-up to HDFS-15607.
> Goal:
> Initialize (create) snapshot trash root for all existing snapshottable
> directories if {{dfs.namenode.snapshot.trashroot.enabled}} is set to
> {{true}}. So admins won't have to run {{dfsadmin -provisionTrash}} manually
> on all those existing snapshottable directories.
> The change is expected to land in {{FSNamesystem}}.
> Discussion:
> 1. Currently in HDFS-15607, the snapshot trash root creation logic is on the
> client side. But in order for NN to create it at startup, the logic must
> (also) be implemented on the server side as well. -- which is also a
> requirement by WebHDFS (HDFS-15612).
> 2. Alternatively, we can provide an extra parameter to the
> {{-provisionTrash}} command like: {{dfsadmin -provisionTrash -all}} to
> initialize/provision trash root on all existing snapshottable dirs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
