[
https://issues.apache.org/jira/browse/HADOOP-13114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15283079#comment-15283079
]
Suraj Nayak commented on HADOOP-13114:
--------------------------------------
JIRA was not accepting comments when I uploaded the latest patch with
{{CodecPool}} changes. Adding the details of Jenkins build here with this
comment :
Jenkins Console output Link :
[https://builds.apache.org/job/PreCommit-HADOOP-Build/9414/console]
Jenkins output :
+1 overall
| Vote | Subsystem | Runtime | Comment
============================================================================
| 0 | reexec | 0m 13s | Docker mode activated.
| +1 | @author | 0m 0s | The patch does not contain any @author
| | | | tags.
| +1 | test4tests | 0m 0s | The patch appears to include 2 new or
| | | | modified test files.
| +1 | mvninstall | 7m 1s | trunk passed
| +1 | compile | 0m 14s | trunk passed with JDK v1.8.0_91
| +1 | compile | 0m 17s | trunk passed with JDK v1.7.0_95
| +1 | checkstyle | 0m 17s | trunk passed
| +1 | mvnsite | 0m 22s | trunk passed
| +1 | mvneclipse | 0m 15s | trunk passed
| +1 | findbugs | 0m 28s | trunk passed
| +1 | javadoc | 0m 12s | trunk passed with JDK v1.8.0_91
| +1 | javadoc | 0m 15s | trunk passed with JDK v1.7.0_95
| +1 | mvninstall | 0m 17s | the patch passed
| +1 | compile | 0m 13s | the patch passed with JDK v1.8.0_91
| +1 | javac | 0m 13s | the patch passed
| +1 | compile | 0m 15s | the patch passed with JDK v1.7.0_95
| +1 | javac | 0m 15s | the patch passed
| +1 | checkstyle | 0m 14s | the patch passed
| +1 | mvnsite | 0m 20s | the patch passed
| +1 | mvneclipse | 0m 11s | the patch passed
| +1 | whitespace | 0m 0s | The patch has no whitespace issues.
| +1 | findbugs | 0m 36s | the patch passed
| +1 | javadoc | 0m 10s | the patch passed with JDK v1.8.0_91
| +1 | javadoc | 0m 12s | the patch passed with JDK v1.7.0_95
| +1 | unit | 8m 40s | hadoop-distcp in the patch passed with
| | | | JDK v1.8.0_91.
| +1 | unit | 7m 55s | hadoop-distcp in the patch passed with
| | | | JDK v1.7.0_95.
| +1 | asflicense | 0m 17s | The patch does not generate ASF License
| | | | warnings.
| | | 29m 51s |
|| Subsystem || Report/Notes ||
============================================================================
| Docker | Image:yetus/hadoop:cf2ee45 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12803827/HADOOP-13114-trunk_2016-05-12-1.patch
|
| JIRA Issue | HADOOP-13114 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 62e2be2ea3c4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / fa440a3 |
| Default Java | 1.7.0_95 |
| Multi-JDK versions | /usr/lib/jvm/java-8-oracle:1.8.0_91
/usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 |
| findbugs | v3.0.0 |
| JDK v1.7.0_95 Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9414/testReport/ |
| modules | C: hadoop-tools/hadoop-distcp U: hadoop-tools/hadoop-distcp |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/9414/console |
| Powered by | Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org |
> DistCp should have option to compress data on write
> ---------------------------------------------------
>
> Key: HADOOP-13114
> URL: https://issues.apache.org/jira/browse/HADOOP-13114
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 3.0.0-alpha1
> Reporter: Suraj Nayak
> Assignee: Suraj Nayak
> Priority: Minor
> Labels: distcp
> Fix For: 3.0.0-alpha1
>
> Attachments: HADOOP-13114-trunk_2016-05-07-1.patch,
> HADOOP-13114-trunk_2016-05-08-1.patch, HADOOP-13114-trunk_2016-05-10-1.patch,
> HADOOP-13114-trunk_2016-05-12-1.patch
>
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> DistCp utility should have capability to store data in user specified
> compression format. This avoids one hop of compressing data after transfer.
> Backup strategies to different cluster also get benefit of saving one IO
> operation to and from HDFS, thus saving resources, time and effort.
> * Create an option -compressOutput defaulting to
> {{org.apache.hadoop.io.compress.BZip2Codec}}.
> * Users will be able to change codec with {{-D
> mapreduce.output.fileoutputformat.compress.codec=org.apache.hadoop.io.compress.GzipCodec}}
> * If distcp compression is enabled, suffix the filenames with default codec
> extension to indicate the file is compressed. Thus users can be aware of what
> codec was used to compress the data.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
