[
https://issues.apache.org/jira/browse/HDFS-15614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17320982#comment-17320982
]
Ayush Saxena commented on HDFS-15614:
-------------------------------------
Quoting [~shashikant] from the
[PR|https://github.com/apache/hadoop/pull/2881#issuecomment-819454476]
{quote}Coming to think of it, if providing an external command to create the
Trash directory by admins is feasible and makes sense, i think its ok to remove
the NN startup logic to create Trash directories inside snapshottable root.
{quote}
Yeps, thats exactly my point, We seems to be on same page now. :)
[~smeng]
{quote}Therefore this would be the same deal for encryption zone trash root
creation. When replacing dfs.allowSnapshot calls with dfs.createEncryptionZone
in the first test case we should also find trash root inside encryption zone
missing with dfs.provisionSnapshotTrash call alone.
{quote}
Well I didn't actually catch this, but the Ambiguity I am talking about is not
between commands executed from DFSAdmin and DFS, The ambiguity I am talking
about is the change coming post namenode startup/failover. If it was there
before failover/restart. It should be there post failover/restart. If it wasn't
there before so it shouldn't surface also after failover/restart.
EncryptionZones doesn't create any trash dir themselves during restart or
failover AFAIK. So, this ambiguity will not be there. And the existence of
Trash in case of DFSAdmin and not through DFS, is OK for both yours and
Encryption zone cases. That is just a behavioural aspect.
{quote}name quota can become an issue indeed.
{quote}
{quote}name quota can become an issue indeed.
I think I got your point. Maybe a better way to create those necessary Trash
dirs is to ask an admin to run dfsadmin command manually after flipping
dfs.namenode.snapshot.trashroot.enabled to true.
Currently we already have dfsadmin -provisionSnapshotTrash but can only be
done one by one. dfsadmin -provisionSnapshotTrash -all can be implemented to
achieve this.
{quote}
Yeps, We have an agreement again. :) This way sounds very appropriate. Having
{{dfsadmin -provisionSnapshotTrash -all}} will save efforts and will be really
good, Should be very doable as well at client side itself. Can go to Namenode
too, but going to the namenode to create for all in one shot might lead to lock
starvation and stuff.
So, If everyone is on same page(I suppose). We can get rid of HDFS-15614 and
HDFS-15820 and if any other fix on top. and open up a new jira for -all stuff.
Thanx Shashikant and Siyao!!!
> Initialize snapshot trash root during NameNode startup if enabled
> -----------------------------------------------------------------
>
> Key: HDFS-15614
> URL: https://issues.apache.org/jira/browse/HDFS-15614
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> This is a follow-up to HDFS-15607.
> Goal:
> Initialize (create) snapshot trash root for all existing snapshottable
> directories if {{dfs.namenode.snapshot.trashroot.enabled}} is set to
> {{true}}. So admins won't have to run {{dfsadmin -provisionTrash}} manually
> on all those existing snapshottable directories.
> The change is expected to land in {{FSNamesystem}}.
> Discussion:
> 1. Currently in HDFS-15607, the snapshot trash root creation logic is on the
> client side. But in order for NN to create it at startup, the logic must
> (also) be implemented on the server side as well. -- which is also a
> requirement by WebHDFS (HDFS-15612).
> 2. Alternatively, we can provide an extra parameter to the
> {{-provisionTrash}} command like: {{dfsadmin -provisionTrash -all}} to
> initialize/provision trash root on all existing snapshottable dirs.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
