[jira] [Commented] (MAPREDUCE-6676) NNBench should Throw IOException when rename and delete fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-6676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041238#comment-16041238 ] Hudson commented on MAPREDUCE-6676: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11840 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/11840/]) MAPREDUCE-6676. NNBench should Throw IOException when rename and delete (brahma: rev 24181f559f008fc7945702c0fad7d1028f75bfc6) * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/hdfs/NNBench.java > NNBench should Throw IOException when rename and delete fails > - > > Key: MAPREDUCE-6676 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6676 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula > Fix For: 2.9.0, 3.0.0-alpha4, 2.8.2 > > Attachments: MAPREDUCE-6676.patch > > > Throw IOException when rename,delete fails, currently it's unknown to user > when rename and delte fails.. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6896) Document wrong spelling in usage of MapredTestDriver tools.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16041236#comment-16041236
]
Daniel Templeton commented on MAPREDUCE-6896:
-
LGTM +1
> Document wrong spelling in usage of MapredTestDriver tools.
> ---
>
> Key: MAPREDUCE-6896
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6896
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0-alpha3
>Reporter: LiXin Ge
>Assignee: LiXin Ge
> Labels: easyfix
> Attachments: MAPREDUCE-6896.001.patch
>
>
> When the user run performance test of MapredTestDriver, a spelling mistake
> seems exposed to user:
> {quote}
> ./hadoop org.apache.hadoop.test.MapredTestDriver
> An example program must be given as the first argument.
> Valid program names are:
> ..
> timelineperformance: A job that launches mappers to test *timline* service
> performance.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Assigned] (MAPREDUCE-6896) Document wrong spelling in usage of MapredTestDriver tools.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton reassigned MAPREDUCE-6896:
---
Assignee: LiXin Ge
> Document wrong spelling in usage of MapredTestDriver tools.
> ---
>
> Key: MAPREDUCE-6896
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6896
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0-alpha3
>Reporter: LiXin Ge
>Assignee: LiXin Ge
> Labels: easyfix
> Attachments: MAPREDUCE-6896.001.patch
>
>
> When the user run performance test of MapredTestDriver, a spelling mistake
> seems exposed to user:
> {quote}
> ./hadoop org.apache.hadoop.test.MapredTestDriver
> An example program must be given as the first argument.
> Valid program names are:
> ..
> timelineperformance: A job that launches mappers to test *timline* service
> performance.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6676) NNBench should Throw IOException when rename and delete fails
[
https://issues.apache.org/jira/browse/MAPREDUCE-6676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brahma Reddy Battula updated MAPREDUCE-6676:
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.8.2
3.0.0-alpha4
2.9.0
Status: Resolved (was: Patch Available)
Committed to {{trunk}},{{branch-2}} and {{branch-2.8}}. [~ajisakaa] thanks a
lot for your review.
> NNBench should Throw IOException when rename and delete fails
> -
>
> Key: MAPREDUCE-6676
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6676
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Brahma Reddy Battula
>Assignee: Brahma Reddy Battula
> Fix For: 2.9.0, 3.0.0-alpha4, 2.8.2
>
> Attachments: MAPREDUCE-6676.patch
>
>
> Throw IOException when rename,delete fails, currently it's unknown to user
> when rename and delte fails..
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6676) NNBench should Throw IOException when rename and delete fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-6676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated MAPREDUCE-6676: Summary: NNBench should Throw IOException when rename and delete fails (was: [ NNBench ] Throw IOException when rename,delete fails) > NNBench should Throw IOException when rename and delete fails > - > > Key: MAPREDUCE-6676 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6676 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula > Attachments: MAPREDUCE-6676.patch > > > Throw IOException when rename,delete fails, currently it's unknown to user > when rename and delte fails.. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6676) [ NNBench ] Throw IOException when rename,delete fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-6676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040713#comment-16040713 ] Akira Ajisaka commented on MAPREDUCE-6676: -- LGTM, +1 > [ NNBench ] Throw IOException when rename,delete fails > -- > > Key: MAPREDUCE-6676 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6676 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Brahma Reddy Battula >Assignee: Brahma Reddy Battula > Attachments: MAPREDUCE-6676.patch > > > Throw IOException when rename,delete fails, currently it's unknown to user > when rename and delte fails.. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6896) Document wrong spelling in usage of MapredTestDriver tools.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040690#comment-16040690
]
Hadoop QA commented on MAPREDUCE-6896:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 14s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m 29s
{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
31s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 50s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
37s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 16s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
47s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 59s
{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common in
trunk has 1 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s
{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 13s
{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m 30s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 15s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 33s
{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 19m 13s
{color} | {color:green} hadoop-yarn-client in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 99m 15s
{color} | {color:green} hadoop-mapreduce-client-jobclient in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 167m 37s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12871785/MAPREDUCE-6896.001.patch
|
| JIRA Issue | MAPREDUCE-6896 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux d0a6b8b6dbff 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 5ec7163 |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| findbugs |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6989/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common-warnings.html
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6989/testReport/ |
| modules | C:
[jira] [Commented] (MAPREDUCE-6447) reduce shuffle throws "java.lang.OutOfMemoryError: Java heap space"
[ https://issues.apache.org/jira/browse/MAPREDUCE-6447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040517#comment-16040517 ] Adam Sotona commented on MAPREDUCE-6447: With relation to this issue I have manually performed a kind of configuration "bracketing", where I manually altered the job configuration of a still running application before another attempt. To do it manually is a bit complicated as you have to modify the job configuration in HDFS stagging folder of the application as well as on each data node in the local appcache//filecache. However what if there would be a mechanism to prepare such alternate configuration(s), which will be used for second and following attempts after a task fail? For example in job configuration I would set: mapreduce.reduce.shuffle.input.buffer.percent=0.70 mapreduce.reduce.shuffle.memory.limit.percent=0.4 mapreduce.reduce.shuffle.parallelcopies=30 attempt-2.mapreduce.reduce.shuffle.input.buffer.percent=0.70 attempt-2.mapreduce.reduce.shuffle.memory.limit.percent=0.2 attempt-2.mapreduce.reduce.shuffle.parallelcopies=4 attempt-3.mapreduce.reduce.shuffle.input.buffer.percent=0.20 attempt-3.mapreduce.reduce.shuffle.memory.limit.percent=0.2 attempt-3.mapreduce.reduce.shuffle.parallelcopies=2 Such mechanism should not be hard to implement and it would allow to have alternate fail-over configurations prepared in advance instead of long-term repeated bug hunting. Thanks > reduce shuffle throws "java.lang.OutOfMemoryError: Java heap space" > --- > > Key: MAPREDUCE-6447 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6447 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.5.0, 2.6.0, 2.5.1, 2.7.1 >Reporter: shuzhangyao >Assignee: shuzhangyao >Priority: Minor > > 2015-08-11 14:03:54,550 WARN [main] org.apache.hadoop.mapred.YarnChild: > Exception running child : > org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError: error in > shuffle in fetcher#10 > at org.apache.hadoop.mapreduce.task.reduce.Shuffle.run(Shuffle.java:134) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:376) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) > Caused by: java.lang.OutOfMemoryError: Java heap space > at > org.apache.hadoop.io.BoundedByteArrayOutputStream.(BoundedByteArrayOutputStream.java:56) > at > org.apache.hadoop.io.BoundedByteArrayOutputStream.(BoundedByteArrayOutputStream.java:46) > at > org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput.(InMemoryMapOutput.java:63) > at > org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl.unconditionalReserve(MergeManagerImpl.java:303) > at > org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl.reserve(MergeManagerImpl.java:293) > at > org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:511) > at > org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:329) > at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:193) -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6896) Document wrong spelling in usage of MapredTestDriver tools.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LiXin Ge updated MAPREDUCE-6896:
Status: Patch Available (was: Open)
> Document wrong spelling in usage of MapredTestDriver tools.
> ---
>
> Key: MAPREDUCE-6896
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6896
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0-alpha3
>Reporter: LiXin Ge
> Labels: easyfix
> Attachments: MAPREDUCE-6896.001.patch
>
>
> When the user run performance test of MapredTestDriver, a spelling mistake
> seems exposed to user:
> {quote}
> ./hadoop org.apache.hadoop.test.MapredTestDriver
> An example program must be given as the first argument.
> Valid program names are:
> ..
> timelineperformance: A job that launches mappers to test *timline* service
> performance.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6896) Document wrong spelling in usage of MapredTestDriver tools.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
LiXin Ge updated MAPREDUCE-6896:
Attachment: MAPREDUCE-6896.001.patch
> Document wrong spelling in usage of MapredTestDriver tools.
> ---
>
> Key: MAPREDUCE-6896
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6896
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: documentation
>Affects Versions: 3.0.0-alpha3
>Reporter: LiXin Ge
> Labels: easyfix
> Attachments: MAPREDUCE-6896.001.patch
>
>
> When the user run performance test of MapredTestDriver, a spelling mistake
> seems exposed to user:
> {quote}
> ./hadoop org.apache.hadoop.test.MapredTestDriver
> An example program must be given as the first argument.
> Valid program names are:
> ..
> timelineperformance: A job that launches mappers to test *timline* service
> performance.
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6896) Document wrong spelling in usage of MapredTestDriver tools.
LiXin Ge created MAPREDUCE-6896:
---
Summary: Document wrong spelling in usage of MapredTestDriver
tools.
Key: MAPREDUCE-6896
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6896
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: documentation
Affects Versions: 3.0.0-alpha3
Reporter: LiXin Ge
When the user run performance test of MapredTestDriver, a spelling mistake
seems exposed to user:
{quote}
./hadoop org.apache.hadoop.test.MapredTestDriver
An example program must be given as the first argument.
Valid program names are:
..
timelineperformance: A job that launches mappers to test *timline* service
performance.
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6447) reduce shuffle throws "java.lang.OutOfMemoryError: Java heap space"
[ https://issues.apache.org/jira/browse/MAPREDUCE-6447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040315#comment-16040315 ] Adam Sotona commented on MAPREDUCE-6447: Hi guys, I'm experiencing exactly the same issue, even I decreased the values following way: mapreduce.reduce.shuffle.input.buffer.percent=0.70 mapreduce.reduce.shuffle.memory.limit.percent=0.2 mapreduce.reduce.shuffle.parallelcopies=4 According to the above calculations 0.7 * 0.2 * 4 = 0.56 < 1 Could someone explain it? I'm running very long MapReduce process, where some of the reducers almost deterministically fail on OOME, repeated attempts still cause OOME and the whole restarted application with reduced shuffle parameters still fails (after several days of computation). Is there any reason why InMemoryMapOutput blindly allocates memory despite the actual situation? Could Fetcher.copyMapOutput recover from that situation by falling into an alternative MapOutput? Thanks > reduce shuffle throws "java.lang.OutOfMemoryError: Java heap space" > --- > > Key: MAPREDUCE-6447 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6447 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.5.0, 2.6.0, 2.5.1, 2.7.1 >Reporter: shuzhangyao >Assignee: shuzhangyao >Priority: Minor > > 2015-08-11 14:03:54,550 WARN [main] org.apache.hadoop.mapred.YarnChild: > Exception running child : > org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError: error in > shuffle in fetcher#10 > at org.apache.hadoop.mapreduce.task.reduce.Shuffle.run(Shuffle.java:134) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:376) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) > Caused by: java.lang.OutOfMemoryError: Java heap space > at > org.apache.hadoop.io.BoundedByteArrayOutputStream.(BoundedByteArrayOutputStream.java:56) > at > org.apache.hadoop.io.BoundedByteArrayOutputStream.(BoundedByteArrayOutputStream.java:46) > at > org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput.(InMemoryMapOutput.java:63) > at > org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl.unconditionalReserve(MergeManagerImpl.java:303) > at > org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl.reserve(MergeManagerImpl.java:293) > at > org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:511) > at > org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:329) > at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:193) -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6879) TestDFSIO#sequentialTest throws java.lang.NullPointerException due to uninitialized IOStream
[
https://issues.apache.org/jira/browse/MAPREDUCE-6879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040285#comment-16040285
]
ASF GitHub Bot commented on MAPREDUCE-6879:
---
Github user wenxinhe closed the pull request at:
https://github.com/apache/hadoop/pull/215
> TestDFSIO#sequentialTest throws java.lang.NullPointerException due to
> uninitialized IOStream
>
>
> Key: MAPREDUCE-6879
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6879
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 2.7.3, 3.0.0-alpha2
>Reporter: wenxin he
>Priority: Minor
>
> When I use {color:red} -seq {color} arg to write files,
> TestDFSIO#sequentialTest throws java.lang.NullPointerException due to
> uninitialized stream inherited from IOMapperBase:
> {code}
> [root@7b65290f2609 hadoop-3.0.0-alpha3-SNAPSHOT]# bin/hadoop jar
> share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.0.0-alpha3-SNAPSHOT-tests.jar
> TestDFSIO -seq -write
> 2017-04-20 15:50:43,505 INFO fs.TestDFSIO: TestDFSIO.1.8
> 2017-04-20 15:50:43,510 INFO fs.TestDFSIO: nrFiles = 1
> 2017-04-20 15:50:43,510 INFO fs.TestDFSIO: nrBytes (MB) = 1.0
> 2017-04-20 15:50:43,510 INFO fs.TestDFSIO: bufferSize = 100
> 2017-04-20 15:50:43,510 INFO fs.TestDFSIO: baseDir = /benchmarks/TestDFSIO
> 2017-04-20 15:50:43,655 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> java.lang.NullPointerException
> at org.apache.hadoop.fs.TestDFSIO$WriteMapper.doIO(TestDFSIO.java:427)
> at org.apache.hadoop.fs.TestDFSIO$WriteMapper.doIO(TestDFSIO.java:395)
> at org.apache.hadoop.fs.TestDFSIO.sequentialTest(TestDFSIO.java:722)
> at org.apache.hadoop.fs.TestDFSIO.run(TestDFSIO.java:846)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at org.apache.hadoop.fs.TestDFSIO.main(TestDFSIO.java:731)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
> at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144)
> at
> org.apache.hadoop.test.MapredTestDriver.run(MapredTestDriver.java:136)
> at
> org.apache.hadoop.test.MapredTestDriver.main(MapredTestDriver.java:144)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:239)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
