[
https://issues.apache.org/jira/browse/HDFS-7312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14222163#comment-14222163
]
Joseph Prosser commented on HDFS-7312:
--------------------------------------
Added additional testcases and usage note.
Removed extra line.
This code has been tested on a cluster running Hadoop 2.0.0-cdh4.5.0 with S3
access using an S3 bucket.
When using the -skiptmp flag I do not see these output lines:
14/11/22 15:20:07 INFO s3a.S3AFileSystem: Delete path
s3a://BUCKETNAME/_distcp_tmp_57fmri - recursive true
14/11/22 15:20:07 INFO s3a.S3AFileSystem: Getting path status for
s3a://BUCKETNAME/_distcp_tmp_57fmri (_distcp_tmp_57fmri)
> Update DistCp v1 to optionally not use tmp location
> ---------------------------------------------------
>
> Key: HDFS-7312
> URL: https://issues.apache.org/jira/browse/HDFS-7312
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: tools
> Affects Versions: 2.5.1
> Reporter: Joseph Prosser
> Assignee: Joseph Prosser
> Priority: Minor
> Attachments: HDFS-7312.001.patch, HDFS-7312.002.patch,
> HDFS-7312.003.patch, HDFS-7312.004.patch, HDFS-7312.005.patch,
> HDFS-7312.006.patch, HDFS-7312.patch
>
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> DistCp v1 currently copies files to a tmp location and then renames that to
> the specified destination. This can cause performance issues on filesystems
> such as S3. A -skiptmp flag will be added to bypass this step and copy
> directly to the destination. This feature mirrors a similar one added to
> HBase ExportSnapshot
> [HBASE-11119|https://issues.apache.org/jira/browse/HBASE-11119]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
