[
https://issues.apache.org/jira/browse/MAPREDUCE-5809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated MAPREDUCE-5809:
-------------------------------------
Attachment: MAPREDUCE-5809.1.patch
The attached patch introduces the "distcp -pa" command line argument for
preserving ACLs. I found that it helped the implementation to reuse some code
currently in HDFS, so I've done some refactoring. If this gets +1, then I'll
split it into separate patches for HADOOP and HDFS.
# {{AclUtil}}: This is a new utility class containing methods that previously
resided in HDFS or privately in Hadoop Common. It's annotated
{{LimitedPrivate}} for HDFS and MapReduce. There is no new logic here, just
moving the methods around.
# {{ScopedAclEntries}}: This whole class has moved from HDFS to Common with
addition of the {{LimitedPrivate}} annotation.
# {{AclCommands}}: The implementation of the ls shell command is simplified by
using the new utility code.
# {{CopyListing}}: Checks if source file system supports ACLs before attempting
to run the distcp job (fail fast). Renamed {{checkForDuplicates}} to
{{validateFinalListing}} since the validation is now more than just checking
for duplicates.
# {{DistCp}}: Checks if target file system supports ACLs.
# {{DistCpOptionSwitch}}: Documented meaning of -pa and also made it clear
which attributes get preserved when passing -p with no additional flags.
# {{DistCpUtils}}: Preserve ACLs if requested.
# {{TestDistCpWithAcls}}: New test suite that runs distcp with -pa and asserts
that ACLs and permission bits are the same at the destination. Also tests the
fail-fast behavior when ACLs are not enabled in the NameNode or ACLs are
unimplemented in the file system.
The one thing we can't cover in our automated tests here is the case of
attempting distcp with -pa where the NameNode is pre-2.4.0. I'll check that
with a manual test.
> Enhance distcp to support preserving HDFS ACLs.
> -----------------------------------------------
>
> Key: MAPREDUCE-5809
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5809
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: distcp
> Affects Versions: 2.4.0
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: MAPREDUCE-5809.1.patch
>
>
> This issue tracks enhancing distcp to add a new command-line argument for
> preserving HDFS ACLs from the source at the copy destination.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
