[jira] [Commented] (MAPREDUCE-6780) Add support for striping files in benchmarking of TeraGen and TeraSort
[ https://issues.apache.org/jira/browse/MAPREDUCE-6780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15559716#comment-15559716 ] Hudson commented on MAPREDUCE-6780: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10575 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10575/]) MAPREDUCE-6780. Add support for HDFS directory with erasure code policy (kai.zheng: rev bea004eaeb7ba33bf324ef3e7065cfdd614d8198) * (edit) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java > Add support for striping files in benchmarking of TeraGen and TeraSort > -- > > Key: MAPREDUCE-6780 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6780 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: SammiChen >Assignee: SammiChen > Fix For: 3.0.0-alpha2 > > Attachments: MAPREDUCE-6780-v1.patch, MAPREDUCE-6780-v2.patch > > > So far, HDFS file with erasure code policy doesn't support hflush and hsync > operation. This task is going to find a way to support writing data to > erasure code policy files in TeraGen and TeraSort. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6780) Add support for striping files in benchmarking of TeraGen and TeraSort
[ https://issues.apache.org/jira/browse/MAPREDUCE-6780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated MAPREDUCE-6780: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2 Status: Resolved (was: Patch Available) Committed to trunk. Thanks [~Sammi] for the contribution! > Add support for striping files in benchmarking of TeraGen and TeraSort > -- > > Key: MAPREDUCE-6780 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6780 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: SammiChen >Assignee: SammiChen > Fix For: 3.0.0-alpha2 > > Attachments: MAPREDUCE-6780-v1.patch, MAPREDUCE-6780-v2.patch > > > So far, HDFS file with erasure code policy doesn't support hflush and hsync > operation. This task is going to find a way to support writing data to > erasure code policy files in TeraGen and TeraSort. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6780) Add support for striping files in benchmarking of TeraGen and TeraSort
[ https://issues.apache.org/jira/browse/MAPREDUCE-6780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated MAPREDUCE-6780: - Summary: Add support for striping files in benchmarking of TeraGen and TeraSort (was: Add support for HDFS directory with erasure code policy to TeraGen and TeraSort) > Add support for striping files in benchmarking of TeraGen and TeraSort > -- > > Key: MAPREDUCE-6780 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6780 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: SammiChen >Assignee: SammiChen > Attachments: MAPREDUCE-6780-v1.patch, MAPREDUCE-6780-v2.patch > > > So far, HDFS file with erasure code policy doesn't support hflush and hsync > operation. This task is going to find a way to support writing data to > erasure code policy files in TeraGen and TeraSort. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6780) Add support for HDFS directory with erasure code policy to TeraGen and TeraSort
[ https://issues.apache.org/jira/browse/MAPREDUCE-6780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15559673#comment-15559673 ] Kai Zheng commented on MAPREDUCE-6780: -- Thanks [~Sammi] for the update per our off-line reviewing discussion. The latest patch LGTM and +1. > Add support for HDFS directory with erasure code policy to TeraGen and > TeraSort > --- > > Key: MAPREDUCE-6780 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6780 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: SammiChen >Assignee: SammiChen > Attachments: MAPREDUCE-6780-v1.patch, MAPREDUCE-6780-v2.patch > > > So far, HDFS file with erasure code policy doesn't support hflush and hsync > operation. This task is going to find a way to support writing data to > erasure code policy files in TeraGen and TeraSort. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6780) Add support for HDFS directory with erasure code policy to TeraGen and TeraSort
[ https://issues.apache.org/jira/browse/MAPREDUCE-6780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15559526#comment-15559526 ] Hadoop QA commented on MAPREDUCE-6780: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s {color} | {color:blue} Docker mode activated. {color} | | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s {color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 55s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 18s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 14s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 22s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 15s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 27s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s {color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 16s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 13s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 13s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 10s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 19s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 12s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 32s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s {color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 30s {color} | {color:green} hadoop-mapreduce-examples in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 17s {color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 12m 27s {color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:9560f25 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12832324/MAPREDUCE-6780-v2.patch | | JIRA Issue | MAPREDUCE-6780 | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux b7fe10ff2eb3 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 4d10621 | | Default Java | 1.8.0_101 | | findbugs | v3.0.0 | | Test Results | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6757/testReport/ | | modules | C: hadoop-mapreduce-project/hadoop-mapreduce-examples U: hadoop-mapreduce-project/hadoop-mapreduce-examples | | Console output | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6757/console | | Powered by | Apache Yetus 0.3.0 http://yetus.apache.org | This message was automatically generated. > Add support for HDFS directory with erasure code policy to TeraGen and > TeraSort > --- > > Key: MAPREDUCE-6780 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6780 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: SammiChen >Assignee: SammiChen > Attachments:
[jira] [Updated] (MAPREDUCE-6780) Add support for HDFS directory with erasure code policy to TeraGen and TeraSort
[ https://issues.apache.org/jira/browse/MAPREDUCE-6780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated MAPREDUCE-6780: - Attachment: MAPREDUCE-6780-v2.patch Refine some wording > Add support for HDFS directory with erasure code policy to TeraGen and > TeraSort > --- > > Key: MAPREDUCE-6780 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6780 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: SammiChen >Assignee: SammiChen > Attachments: MAPREDUCE-6780-v1.patch, MAPREDUCE-6780-v2.patch > > > So far, HDFS file with erasure code policy doesn't support hflush and hsync > operation. This task is going to find a way to support writing data to > erasure code policy files in TeraGen and TeraSort. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org