[jira] [Updated] (MAPREDUCE-6947) Moving logging APIs over to slf4j in hadoop-mapreduce-examples
[ https://issues.apache.org/jira/browse/MAPREDUCE-6947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated MAPREDUCE-6947: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.1.0 3.0.0-beta1 2.9.0 Status: Resolved (was: Patch Available) Committed this to trunk, branch-3.0, and branch-2. Thanks [~GergelyNovak] for the contribution! > Moving logging APIs over to slf4j in hadoop-mapreduce-examples > --- > > Key: MAPREDUCE-6947 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6947 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Gergely Novák >Assignee: Gergely Novák > Fix For: 2.9.0, 3.0.0-beta1, 3.1.0 > > Attachments: MAPREDUCE-6947.001.patch, MAPREDUCE-6947.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6947) Moving logging APIs over to slf4j in hadoop-mapreduce-examples
[ https://issues.apache.org/jira/browse/MAPREDUCE-6947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16171006#comment-16171006 ] Hudson commented on MAPREDUCE-6947: --- FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #12907 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12907/]) MAPREDUCE-6947. Moving logging APIs over to slf4j in (aajisaka: rev 2018538fdba1a95a6556187569e872fce7f9e1c3) * (edit) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraGen.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/DBCountPageView.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/test/java/org/apache/hadoop/examples/terasort/TestTeraSort.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/BaileyBorweinPlouffe.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/dancing/DancingLinks.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/pi/DistSum.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraScheduler.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java > Moving logging APIs over to slf4j in hadoop-mapreduce-examples > --- > > Key: MAPREDUCE-6947 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6947 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Gergely Novák >Assignee: Gergely Novák > Attachments: MAPREDUCE-6947.001.patch, MAPREDUCE-6947.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6947) Moving logging APIs over to slf4j in hadoop-mapreduce-examples
[ https://issues.apache.org/jira/browse/MAPREDUCE-6947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170999#comment-16170999 ] Akira Ajisaka commented on MAPREDUCE-6947: -- Yes, your patch is exactly what I meant. +1, checking this in. > Moving logging APIs over to slf4j in hadoop-mapreduce-examples > --- > > Key: MAPREDUCE-6947 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6947 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Gergely Novák >Assignee: Gergely Novák > Attachments: MAPREDUCE-6947.001.patch, MAPREDUCE-6947.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6958) Shuffle audit logger should log size of shuffle transfer
[
https://issues.apache.org/jira/browse/MAPREDUCE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170856#comment-16170856
]
Hadoop QA commented on MAPREDUCE-6958:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 16m
45s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
34s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
18s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
28s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
17s{color} | {color:green} hadoop-mapreduce-client-shuffle in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 34s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:eaf5c66 |
| JIRA Issue | MAPREDUCE-6958 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887770/MAPREDUCE-6958-branch-2.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 92227d8c29f4 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | branch-2 / 6aaca3d |
| Default Java | 1.7.0_151 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7143/testReport/ |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle
U:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle
|
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7143/console |
| Powered by | Apache Yetus 0.5.0 http://yetus.apache.org |
This message was automatically generated.
> Shuffle audit logger should log size of shuffle transfer
>
>
> Key: MAPREDUCE-6958
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6958
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Reporter: Jason Lowe
>Assignee: Jason Lowe
>Priority:
[jira] [Commented] (MAPREDUCE-6958) Shuffle audit logger should log size of shuffle transfer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170837#comment-16170837 ] Chris Douglas commented on MAPREDUCE-6958: -- bq. Since 3.0 hasn't officially shipped yet, I propose to revert the 003 patch I committed to trunk and branch-3.0 and instead commit patch version 002 which preserves the job-then-reducer ordering already established in the 2.x line. Objections? Nope, that sounds reasonable. Thanks for the extra audit > Shuffle audit logger should log size of shuffle transfer > > > Key: MAPREDUCE-6958 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6958 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Minor > Fix For: 3.0.0-beta1 > > Attachments: MAPREDUCE-6958.001.patch, MAPREDUCE-6958.002.patch, > MAPREDUCE-6958.003.patch, MAPREDUCE-6958-branch-2.002.patch > > > The shuffle audit logger currently logs the job ID and reducer ID but nothing > about the size of the requested transfer. It calculates this as part of the > HTTP response headers, so it would be trivial to log the response size. This > would be very valuable for debugging network traffic storms from the shuffle > handler. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6958) Shuffle audit logger should log size of shuffle transfer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-6958: -- Attachment: MAPREDUCE-6958-branch-2.002.patch Attaching a branch-2 version of the 002 patch. > Shuffle audit logger should log size of shuffle transfer > > > Key: MAPREDUCE-6958 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6958 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Minor > Fix For: 3.0.0-beta1 > > Attachments: MAPREDUCE-6958.001.patch, MAPREDUCE-6958.002.patch, > MAPREDUCE-6958.003.patch, MAPREDUCE-6958-branch-2.002.patch > > > The shuffle audit logger currently logs the job ID and reducer ID but nothing > about the size of the requested transfer. It calculates this as part of the > HTTP response headers, so it would be trivial to log the response size. This > would be very valuable for debugging network traffic storms from the shuffle > handler. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6958) Shuffle audit logger should log size of shuffle transfer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170810#comment-16170810 ] Hudson commented on MAPREDUCE-6958: --- FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #12903 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12903/]) MAPREDUCE-6958. Shuffle audit logger should log size of shuffle (jlowe: rev b3d61304f2fa4a99526f7a60ccaac9f262083079) * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java > Shuffle audit logger should log size of shuffle transfer > > > Key: MAPREDUCE-6958 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6958 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Minor > Fix For: 3.0.0-beta1 > > Attachments: MAPREDUCE-6958.001.patch, MAPREDUCE-6958.002.patch, > MAPREDUCE-6958.003.patch > > > The shuffle audit logger currently logs the job ID and reducer ID but nothing > about the size of the requested transfer. It calculates this as part of the > HTTP response headers, so it would be trivial to log the response size. This > would be very valuable for debugging network traffic storms from the shuffle > handler. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6958) Shuffle audit logger should log size of shuffle transfer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-6958: -- Fix Version/s: 3.0.0-beta1 Hmm, so we have a bit of a quandry. I committed this through branch-3.0, but when I tried picking this to branch-2 and 2.8, but the patch doesn't apply because the mappers field was not there. I thought it had been there awhile, but it turns out it was added in MAPREDUCE-6808 in a backwards-incompatible way. The original format was job then reducer, and that JIRA made it job, maps, reducer. Since 3.0 hasn't officially shipped yet, I propose to revert the 003 patch I committed to trunk and branch-3.0 and instead commit patch version 002 which preserves the job-then-reducer ordering already established in the 2.x line. Objections? > Shuffle audit logger should log size of shuffle transfer > > > Key: MAPREDUCE-6958 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6958 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Minor > Fix For: 3.0.0-beta1 > > Attachments: MAPREDUCE-6958.001.patch, MAPREDUCE-6958.002.patch, > MAPREDUCE-6958.003.patch > > > The shuffle audit logger currently logs the job ID and reducer ID but nothing > about the size of the requested transfer. It calculates this as part of the > HTTP response headers, so it would be trivial to log the response size. This > would be very valuable for debugging network traffic storms from the shuffle > handler. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6958) Shuffle audit logger should log size of shuffle transfer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170796#comment-16170796 ] Jason Lowe commented on MAPREDUCE-6958: --- Thanks for the review! Committing this. > Shuffle audit logger should log size of shuffle transfer > > > Key: MAPREDUCE-6958 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6958 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Minor > Attachments: MAPREDUCE-6958.001.patch, MAPREDUCE-6958.002.patch, > MAPREDUCE-6958.003.patch > > > The shuffle audit logger currently logs the job ID and reducer ID but nothing > about the size of the requested transfer. It calculates this as part of the > HTTP response headers, so it would be trivial to log the response size. This > would be very valuable for debugging network traffic storms from the shuffle > handler. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6960) Shuffle Handler prints disk error stack traces for every read failure.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170725#comment-16170725
]
Hadoop QA commented on MAPREDUCE-6960:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
20s{color} | {color:green} hadoop-mapreduce-client-shuffle in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | MAPREDUCE-6960 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887748/MAPREDUCE-6960.001.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux d0fe073f5ae8 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 29dd551 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7142/testReport/ |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle
U:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle
|
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7142/console |
| Powered by | Apache Yetus 0.5.0 http://yetus.apache.org |
This message was automatically generated.
> Shuffle Handler prints disk error stack traces for every read failure.
> --
>
> Key: MAPREDUCE-6960
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6960
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Kuhu Shukla
>Assignee: Kuhu Shukla
> Attachments:
[jira] [Commented] (MAPREDUCE-6491) Environment variable handling assumes values should be appended
[ https://issues.apache.org/jira/browse/MAPREDUCE-6491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170696#comment-16170696 ] Jason Lowe commented on MAPREDUCE-6491: --- Thanks for the review, Allen! bq. DYLD_LIBRARY_PATH should be in the list Ah, good catch. Should it also include other dynamic linker related variables like the following? DYLD_FALLBACK_LIBRARY_PATH DYLD_VERSIONED_LIBRARY_PATH DYLD_FRAMEWORK_PATH DYLD_FALLBACK_FRAMEWORK_PATH DYLD_VERSIONED_FRAMEWORK_PATH LD_RUN_PATH bq. As mentioned several times, this won't work if a job (e.g., streaming) calls the shell scripts with the default values. Is this any different that what will happen without this patch? If HADOOP_CLASSPATH shouldn't be appended to in trunk but the code _always_ appends new values to environment variables, then this seems like it's already broken whether this is fixed or not. My apologies if I'm missing something. > Environment variable handling assumes values should be appended > --- > > Key: MAPREDUCE-6491 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6491 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Jason Lowe >Assignee: Dustin Cote > Attachments: MAPREDUCE-6491.011.patch, YARN-2369-10.patch, > YARN-2369-1.patch, YARN-2369-2.patch, YARN-2369-3.patch, YARN-2369-4.patch, > YARN-2369-5.patch, YARN-2369-6.patch, YARN-2369-7.patch, YARN-2369-8.patch, > YARN-2369-9.patch > > > When processing environment variables for a container context the code > assumes that the value should be appended to any pre-existing value in the > environment. This may be desired behavior for handling path-like environment > variables such as PATH, LD_LIBRARY_PATH, CLASSPATH, etc. but it is a > non-intuitive and harmful way to handle any variable that does not have > path-like semantics. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6960) Shuffle Handler prints disk error stack traces for every read failure.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kuhu Shukla updated MAPREDUCE-6960:
---
Status: Patch Available (was: Open)
> Shuffle Handler prints disk error stack traces for every read failure.
> --
>
> Key: MAPREDUCE-6960
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6960
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Kuhu Shukla
>Assignee: Kuhu Shukla
> Attachments: MAPREDUCE-6960.001.patch
>
>
> {code}
> } catch (IOException e) {
> LOG.error("Shuffle error :", e);
> {code}
> In cases where the read from a disk fails and throws a DiskErrorException,
> the shuffle handler prints the entire stack trace for each and every one of
> the failures causing the nodemanager logs to quickly fill up the disk.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6960) Shuffle Handler prints disk error stack traces for every read failure.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kuhu Shukla updated MAPREDUCE-6960:
---
Attachment: MAPREDUCE-6960.001.patch
v1 patch that logs the exception without the stack trace only in the case of
DiskErrorException. We can do the same for all IOExceptions but that would make
any other kinds of IOException debugging difficult.
> Shuffle Handler prints disk error stack traces for every read failure.
> --
>
> Key: MAPREDUCE-6960
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6960
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Kuhu Shukla
>Assignee: Kuhu Shukla
> Attachments: MAPREDUCE-6960.001.patch
>
>
> {code}
> } catch (IOException e) {
> LOG.error("Shuffle error :", e);
> {code}
> In cases where the read from a disk fails and throws a DiskErrorException,
> the shuffle handler prints the entire stack trace for each and every one of
> the failures causing the nodemanager logs to quickly fill up the disk.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6960) Shuffle Handler prints disk error stack traces for every read failure.
Kuhu Shukla created MAPREDUCE-6960:
--
Summary: Shuffle Handler prints disk error stack traces for every
read failure.
Key: MAPREDUCE-6960
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6960
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Kuhu Shukla
Assignee: Kuhu Shukla
{code}
} catch (IOException e) {
LOG.error("Shuffle error :", e);
{code}
In cases where the read from a disk fails and throws a DiskErrorException, the
shuffle handler prints the entire stack trace for each and every one of the
failures causing the nodemanager logs to quickly fill up the disk.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6958) Shuffle audit logger should log size of shuffle transfer
[
https://issues.apache.org/jira/browse/MAPREDUCE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170525#comment-16170525
]
Hadoop QA commented on MAPREDUCE-6958:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
14s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
21s{color} | {color:green} hadoop-mapreduce-client-shuffle in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:71bbb86 |
| JIRA Issue | MAPREDUCE-6958 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12887718/MAPREDUCE-6958.003.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux a423f3a9b770 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 29dd551 |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7141/testReport/ |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle
U:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle
|
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7141/console |
| Powered by | Apache Yetus 0.5.0 http://yetus.apache.org |
This message was automatically generated.
> Shuffle audit logger should log size of shuffle transfer
>
>
> Key: MAPREDUCE-6958
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6958
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
>Reporter: Jason Lowe
>Assignee: Jason Lowe
>Priority: Minor
> Attachments:
[jira] [Commented] (MAPREDUCE-6958) Shuffle audit logger should log size of shuffle transfer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170487#comment-16170487 ] Chris Douglas commented on MAPREDUCE-6958: -- Thanks for updating the patch, [~jlowe]. +1 > Shuffle audit logger should log size of shuffle transfer > > > Key: MAPREDUCE-6958 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6958 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Minor > Attachments: MAPREDUCE-6958.001.patch, MAPREDUCE-6958.002.patch, > MAPREDUCE-6958.003.patch > > > The shuffle audit logger currently logs the job ID and reducer ID but nothing > about the size of the requested transfer. It calculates this as part of the > HTTP response headers, so it would be trivial to log the response size. This > would be very valuable for debugging network traffic storms from the shuffle > handler. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6959) Understanding on process to start contribution
[ https://issues.apache.org/jira/browse/MAPREDUCE-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170486#comment-16170486 ] Mehul commented on MAPREDUCE-6959: -- Thank you [~jlowe] - I just found the wiki link "How to Contribute" you mentioned in your response and going through now. I was just confused with the mirror repo of mapreduce but now I understood its sub project of hadoop common repo. Thanks again! > Understanding on process to start contribution > -- > > Key: MAPREDUCE-6959 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6959 > Project: Hadoop Map/Reduce > Issue Type: Wish >Reporter: Mehul >Priority: Trivial > > I was trying to find process/steps to start with contribution into following > repo i.e. https://github.com/apache/hadoop-mapreduce. Can someone please help > with the detail so that I can create appropriate git/jira issue and start > woking on it? > Any direction would be really appreciated! > Thanks, > Mehul -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6958) Shuffle audit logger should log size of shuffle transfer
[ https://issues.apache.org/jira/browse/MAPREDUCE-6958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-6958: -- Attachment: MAPREDUCE-6958.003.patch Updated the patch to reorder the log message for better backwards compatibility. > Shuffle audit logger should log size of shuffle transfer > > > Key: MAPREDUCE-6958 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6958 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Minor > Attachments: MAPREDUCE-6958.001.patch, MAPREDUCE-6958.002.patch, > MAPREDUCE-6958.003.patch > > > The shuffle audit logger currently logs the job ID and reducer ID but nothing > about the size of the requested transfer. It calculates this as part of the > HTTP response headers, so it would be trivial to log the response size. This > would be very valuable for debugging network traffic storms from the shuffle > handler. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Resolved] (MAPREDUCE-6959) Understanding on process to start contribution
[ https://issues.apache.org/jira/browse/MAPREDUCE-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe resolved MAPREDUCE-6959. --- Resolution: Invalid JIRA is for tracking features and bugs in Hadoop and not for general support. Questions such as these can be directed to the [mailing lists|http://hadoop.apache.org/mailing_lists.html]. Specifically if you're interested in contributing I highly recommend checking out the [How To Contribute|https://wiki.apache.org/hadoop/HowToContribute] wiki page. Note that https://github.com/apache/hadoop-mapreduce is a mirror of just the MapReduce code from what looks like Hadoop 1.x or even earlier code that is no longer supported. All active development is on Hadoop 2. x and Hadoop 3.x. > Understanding on process to start contribution > -- > > Key: MAPREDUCE-6959 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6959 > Project: Hadoop Map/Reduce > Issue Type: Wish >Reporter: Mehul >Priority: Trivial > > I was trying to find process/steps to start with contribution into following > repo i.e. https://github.com/apache/hadoop-mapreduce. Can someone please help > with the detail so that I can create appropriate git/jira issue and start > woking on it? > Any direction would be really appreciated! > Thanks, > Mehul -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6954) Disable erasure coding for files that are uploaded to the MR staging area
[ https://issues.apache.org/jira/browse/MAPREDUCE-6954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170416#comment-16170416 ] Hudson commented on MAPREDUCE-6954: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12898 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12898/]) MAPREDUCE-6954. Disable erasure coding for files that are uploaded to (rkanter: rev 0adc0471d0c06f66a31060f270dcb50a7b4ffafa) * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestJobResourceUploader.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobResourceUploader.java * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java > Disable erasure coding for files that are uploaded to the MR staging area > - > > Key: MAPREDUCE-6954 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6954 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: client >Affects Versions: 3.0.0-alpha4 >Reporter: Peter Bacsko >Assignee: Peter Bacsko > Fix For: 3.0.0-beta1 > > Attachments: MAPREDUCE-6954-001.patch, MAPREDUCE-6954-002.patch, > MAPREDUCE-6954-003.patch, MAPREDUCE-6954-004.patch > > > Depending on the encoder/decoder used and the type or MR workload, EC might > negatively affect the performance of an MR job if too many files are > localized. > In such a scenario, users might want to disable EC in the staging area to > speed up the execution. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6959) Understanding on process to start contribution
[ https://issues.apache.org/jira/browse/MAPREDUCE-6959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mehul updated MAPREDUCE-6959: - Description: I was trying to find process/steps to start with contribution into following repo i.e. https://github.com/apache/hadoop-mapreduce. Can someone please help with the detail so that I can create appropriate git/jira issue and start woking on it? Any direction would be really appreciated! Thanks, Mehul was: I was trying to find process/steps to start with contribution to following repo i.e. https://github.com/apache/hadoop-mapreduce. Can someone please help with that understanding so I can create appropriate git/jira issue and assign myself and start woking on it? Any direction would be really appreciated! Thanks, Mehul > Understanding on process to start contribution > -- > > Key: MAPREDUCE-6959 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6959 > Project: Hadoop Map/Reduce > Issue Type: Wish >Reporter: Mehul >Priority: Trivial > > I was trying to find process/steps to start with contribution into following > repo i.e. https://github.com/apache/hadoop-mapreduce. Can someone please help > with the detail so that I can create appropriate git/jira issue and start > woking on it? > Any direction would be really appreciated! > Thanks, > Mehul -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6954) Disable erasure coding for files that are uploaded to the MR staging area
[ https://issues.apache.org/jira/browse/MAPREDUCE-6954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170389#comment-16170389 ] Peter Bacsko commented on MAPREDUCE-6954: - Thanks for fixing it. > Disable erasure coding for files that are uploaded to the MR staging area > - > > Key: MAPREDUCE-6954 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6954 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: client >Affects Versions: 3.0.0-alpha4 >Reporter: Peter Bacsko >Assignee: Peter Bacsko > Fix For: 3.0.0-beta1 > > Attachments: MAPREDUCE-6954-001.patch, MAPREDUCE-6954-002.patch, > MAPREDUCE-6954-003.patch, MAPREDUCE-6954-004.patch > > > Depending on the encoder/decoder used and the type or MR workload, EC might > negatively affect the performance of an MR job if too many files are > localized. > In such a scenario, users might want to disable EC in the staging area to > speed up the execution. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6954) Disable erasure coding for files that are uploaded to the MR staging area
[ https://issues.apache.org/jira/browse/MAPREDUCE-6954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170383#comment-16170383 ] Robert Kanter commented on MAPREDUCE-6954: -- Oops, I was committing some other stuff that day too, and accidentally put the name of one of those in the commit messages :( trunk: https://github.com/apache/hadoop/commit/3a8d57a0a2e047b34be82f602a2b6cf5593d2125 branch-3.0: https://github.com/apache/hadoop/commit/293ab432e116b93ada6ee56e518ae07aa853ab41 I've just reverted those two: trunk: https://github.com/apache/hadoop/commit/5f496683fb00ba26a6bf5a506ae87d4bc4088727 https://github.com/apache/hadoop/commit/3847b8c861205d9c85c866ec3477e8b0d19bd659 And recommitted the code with the correct message: trunk: https://github.com/apache/hadoop/commit/0adc0471d0c06f66a31060f270dcb50a7b4ffafa https://github.com/apache/hadoop/commit/917e875e511589c297971f6a14fd85de7925409f > Disable erasure coding for files that are uploaded to the MR staging area > - > > Key: MAPREDUCE-6954 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6954 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: client >Affects Versions: 3.0.0-alpha4 >Reporter: Peter Bacsko >Assignee: Peter Bacsko > Fix For: 3.0.0-beta1 > > Attachments: MAPREDUCE-6954-001.patch, MAPREDUCE-6954-002.patch, > MAPREDUCE-6954-003.patch, MAPREDUCE-6954-004.patch > > > Depending on the encoder/decoder used and the type or MR workload, EC might > negatively affect the performance of an MR job if too many files are > localized. > In such a scenario, users might want to disable EC in the staging area to > speed up the execution. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6959) Understanding on process to start contribution
Mehul created MAPREDUCE-6959: Summary: Understanding on process to start contribution Key: MAPREDUCE-6959 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6959 Project: Hadoop Map/Reduce Issue Type: Wish Reporter: Mehul Priority: Trivial I was trying to find process/steps to start with contribution to following repo i.e. https://github.com/apache/hadoop-mapreduce. Can someone please help with that understanding so I can create appropriate git/jira issue and assign myself and start woking on it? Any direction would be really appreciated! Thanks, Mehul -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6947) Moving logging APIs over to slf4j in hadoop-mapreduce-examples
[ https://issues.apache.org/jira/browse/MAPREDUCE-6947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16170023#comment-16170023 ] Gergely Novák commented on MAPREDUCE-6947: -- [~ajisakaa] Is this what you meant? If it is, I'd be happy to create more subtasks for the other parts of MapReduce. > Moving logging APIs over to slf4j in hadoop-mapreduce-examples > --- > > Key: MAPREDUCE-6947 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6947 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Gergely Novák >Assignee: Gergely Novák > Attachments: MAPREDUCE-6947.001.patch, MAPREDUCE-6947.002.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6954) Disable erasure coding for files that are uploaded to the MR staging area
[ https://issues.apache.org/jira/browse/MAPREDUCE-6954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169882#comment-16169882 ] Peter Bacsko commented on MAPREDUCE-6954: - [~rkanter] something is not quite right. I can't see this change on any of the branches you mentioned. > Disable erasure coding for files that are uploaded to the MR staging area > - > > Key: MAPREDUCE-6954 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6954 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: client >Affects Versions: 3.0.0-alpha4 >Reporter: Peter Bacsko >Assignee: Peter Bacsko > Fix For: 3.0.0-beta1 > > Attachments: MAPREDUCE-6954-001.patch, MAPREDUCE-6954-002.patch, > MAPREDUCE-6954-003.patch, MAPREDUCE-6954-004.patch > > > Depending on the encoder/decoder used and the type or MR workload, EC might > negatively affect the performance of an MR job if too many files are > localized. > In such a scenario, users might want to disable EC in the staging area to > speed up the execution. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
