[jira] [Commented] (HADOOP-7310) Trash location needs to be revisited
[
https://issues.apache.org/jira/browse/HADOOP-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924818#comment-16924818
]
Fei Hui commented on HADOOP-7310:
-
Extend on solution A.
Solution A occurs the problem: Admins should create a trash directory before
users remove, othersize users don't allow to create the trash directory.
Create a new Service to move /user/${user}/.Trash/* to
/trash/user/${user}/.Trash/* periodically.
With this we can
1 Resolve users' quota problem.
2 Forward Compatible.
> Trash location needs to be revisited
>
>
> Key: HADOOP-7310
> URL: https://issues.apache.org/jira/browse/HADOOP-7310
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 2.0.0-alpha
>Reporter: Sanjay Radia
>Assignee: Sanjay Radia
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-7310) Trash location needs to be revisited
[
https://issues.apache.org/jira/browse/HADOOP-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16852098#comment-16852098
]
Ahmed Hussein commented on HADOOP-7310:
---
+1 to solution A
I suggest to modify the {{DistributedFileSystem}} trash implementation to do
the following:
# Add a configuration option that represents a list of trash directories
"{{TrashRoots"}}.
# Each time a file gets deleted, the trash is located by looping on the
TrashRoots and returning the first existing directory. If no such directory
exists, the system defaults to the old policy (i.e, {{./user/user1/.Trash}})
# In order to be backward compatible, the {{TrashPolicy.Emptier}} looks in two
different locations: the {{TrashRoots}} and to the default user trash folders (
i.e, {{./user/user1/.Trash}}).
# Since EZ already put trash at the top of respective zone, the EZ won't be
modified.
*Requirements*:
# Define a comma separated list of trash directories in the config file.
# Admins create trash directories. If all the {{TrashRoots}} directories are
missing, the deleted files will be moved to user trash directories.
This approach does not change the FS API and endures minimum changes to the
code.
> Trash location needs to be revisited
>
>
> Key: HADOOP-7310
> URL: https://issues.apache.org/jira/browse/HADOOP-7310
> Project: Hadoop Common
> Issue Type: Improvement
>Affects Versions: 2.0.0-alpha
>Reporter: Sanjay Radia
>Assignee: Sanjay Radia
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-7310) Trash location needs to be revisited
[ https://issues.apache.org/jira/browse/HADOOP-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14626988#comment-14626988 ] Sanjay Radia commented on HADOOP-7310: -- Looking at HDFS-8747, perhaps the better solution is: Whenever a file is deleted, the trash is located by searching up the path and finding the closest parent dir that has trash. It solves trash in encryption zone, trash in any quota based subtree (such as home directory) . Ie the system creates trash in /, home dirs, encryotion zones etc. The increased complication is that the NN needs to deal with multiple trash locations. Trash location needs to be revisited Key: HADOOP-7310 URL: https://issues.apache.org/jira/browse/HADOOP-7310 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.0-alpha Reporter: Sanjay Radia Assignee: Sanjay Radia -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HADOOP-7310) Trash location needs to be revisited
[ https://issues.apache.org/jira/browse/HADOOP-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13461090#comment-13461090 ] Harsh J commented on HADOOP-7310: - I'd like Solution B. bq. Do folks see *real* compatibility problem here? Not sure what a *real* compatibility means. But if you mean code/API wise, I don't see anything abnormal in doing this, nope. Though, the point I think we're missing is that its not always the admin who uses trash. It is certainly the admin who uses quotas, but trash is hardly ever queried by the admin. The problem is breakage of user-training. All users do fs-command X to check trash today. Tomorrow they'll have to do fs-command Y, and thats not easy to ask them to do. If their admin wants to change it for good, he may instruct the users after having made the change; but its no good to break it by default. Hence, I'd vote for solution B, with same-old format, and with a proper doc added along. Trash location needs to be revisited Key: HADOOP-7310 URL: https://issues.apache.org/jira/browse/HADOOP-7310 Project: Hadoop Common Issue Type: Improvement Affects Versions: 2.0.0-alpha Reporter: Sanjay Radia Assignee: Sanjay Radia -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7310) Trash location needs to be revisited
[ https://issues.apache.org/jira/browse/HADOOP-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13038250#comment-13038250 ] Todd Lipcon commented on HADOOP-7310: - If we go with a config, can we use a format string like /user/%s/.Trash? In my experience, the semantics of mapreduce.jobtracker.staging.root.dir have confused many users. Trash location needs to be revisited Key: HADOOP-7310 URL: https://issues.apache.org/jira/browse/HADOOP-7310 Project: Hadoop Common Issue Type: Improvement Reporter: Sanjay Radia Assignee: Sanjay Radia Fix For: 0.23.0 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7310) Trash location needs to be revisited
[ https://issues.apache.org/jira/browse/HADOOP-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13037533#comment-13037533 ] Eli Collins commented on HADOOP-7310: - Solution A sounds good to me as well. This also has the nice property that you can set a single quota on all trash. With federation do you expect that trash will live on the same NN as home directories? If home directories span multiple NNs (probably uncommon) then it seems like you would want trash to as well, and this is more complex for admins when Trash is not a child of the user directory. In terms of compatibility, post this change, I assume /user/joe/.Trash be considered a regular directory, and admins can move the contents to /Trash/joe as part of the upgrade. Trash location needs to be revisited Key: HADOOP-7310 URL: https://issues.apache.org/jira/browse/HADOOP-7310 Project: Hadoop Common Issue Type: Improvement Reporter: Sanjay Radia Assignee: Sanjay Radia Fix For: 0.23.0 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7310) Trash location needs to be revisited
[
https://issues.apache.org/jira/browse/HADOOP-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13037644#comment-13037644
]
Aaron T. Myers commented on HADOOP-7310:
Could we perhaps just make the trash prefix configurable in the same vain as
{{mapreduce.jobtracker.staging.root.dir}}? I'm thinking some configurable path
onto which ${user.name}/.Trash would be appended. This way admins who wanted
to could set the prefix to {{/user}}, and thereby retain the current location
of the trash directories, and admins who wanted a single directory for the
trash at the root could set it to {{/Trash}}, much like Sanjay's proposed
solution A above.
Trash location needs to be revisited
Key: HADOOP-7310
URL: https://issues.apache.org/jira/browse/HADOOP-7310
Project: Hadoop Common
Issue Type: Improvement
Reporter: Sanjay Radia
Assignee: Sanjay Radia
Fix For: 0.23.0
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7310) Trash location needs to be revisited
[ https://issues.apache.org/jira/browse/HADOOP-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13037460#comment-13037460 ] Bharath Mundlapudi commented on HADOOP-7310: +1 to solution A. IMO, Trash is a system wide abstract. Placing /trash/username is cleaner and permissions are honored like USERA can't delete/peek USERB's trash content. Trash location needs to be revisited Key: HADOOP-7310 URL: https://issues.apache.org/jira/browse/HADOOP-7310 Project: Hadoop Common Issue Type: Improvement Reporter: Sanjay Radia Assignee: Sanjay Radia Fix For: 0.23.0 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7310) Trash location needs to be revisited
[ https://issues.apache.org/jira/browse/HADOOP-7310?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13036475#comment-13036475 ] Sanjay Radia commented on HADOOP-7310: -- Trash is located at /user/userName/.Trash Its location needs to be revisted: # Does not work well with Quotas. For example, /user/joe and /project/xx may have quotas. When files are deleted from /project/xx they move to /user/joe/.Trash which causes problems since the user quota is usually much less than that of project-xx. # If customer has a special data hdfs cluster where there are no home directories, then one is forced to have home directories on this data cluster just to accommodate Trash; this can be confusing or may be used as a way get around quotas. # With federation, one is likely to have a NN for user accounts and other NNs for /projects, /data etc. The issues listed for point 2 apply here. I propose two solutions: * A. Put trash in /Trash/userName; generally no quotas will be configured for /Trash since it is cleaned up periodically. This addresses all the three issues raised above. This solution however breaks compatibility. Note Trash applies ONLY to rm's performed via the shell's -rm command. * B. The trash location is configured as SS configuration variable. I prefer solution A, since it is one less configuration variable. Do folks see *real* compatibility problem here? Trash location needs to be revisited Key: HADOOP-7310 URL: https://issues.apache.org/jira/browse/HADOOP-7310 Project: Hadoop Common Issue Type: Improvement Reporter: Sanjay Radia Assignee: Sanjay Radia Fix For: 0.23.0 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
