[ https://issues.apache.org/jira/browse/MAPREDUCE-7292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
John Liu updated MAPREDUCE-7292: -------------------------------- Attachment: MAPREDUCE-7292.002.patch > Use snapshot diff to build file listing when copying to blob storage > -------------------------------------------------------------------- > > Key: MAPREDUCE-7292 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7292 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distcp > Reporter: John Liu > Priority: Minor > Attachments: MAPREDUCE-7292.001.patch, MAPREDUCE-7292.002.patch > > > The DistCp sync option should be extensible for copying to blob storage, > which is not a distributed filesystem. Clients of DistCp could benefit from > using the HDFS snapshot diff report to create the file listing in less time. > A valid use case is to copy new files added to HDFS to a remote blob storage. > The client ensures all new files are copied over but does not require the > destination filesystem to be a distributed filesystem or have the previous > snapshot. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org