[jira] [Commented] (MAPREDUCE-4683) Create and distribute hadoop-mapreduce-client-core-tests.jar

2016-12-07 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-4683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15730887#comment-15730887
 ] 

Hudson commented on MAPREDUCE-4683:
---

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10966 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/10966/])
MAPREDUCE-4683. Create and distribute (aajisaka: rev 
f54afdb83b2567f63f0dc94f5f4e26ead2eefc5e)
* (edit) 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/pom.xml


> Create and distribute hadoop-mapreduce-client-core-tests.jar
> 
>
> Key: MAPREDUCE-4683
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4683
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: build
>Reporter: Arun C Murthy
>Assignee: Akira Ajisaka
>Priority: Critical
> Fix For: 3.0.0-alpha2
>
> Attachments: MAPREDUCE-4683.patch
>
>
> We need to fix our build to create/distribute 
> hadoop-mapreduce-client-core-tests.jar, need this before MAPREDUCE-4253



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-4253) Tests for mapreduce-client-core are lying under mapreduce-client-jobclient

2016-12-07 Thread Akira Ajisaka (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-4253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15730874#comment-15730874
 ] 

Akira Ajisaka commented on MAPREDUCE-4253:
--

After 2 years, MAPREDUCE-4683 was fixed. Now we can fix this problem.

> Tests for mapreduce-client-core are lying under mapreduce-client-jobclient
> --
>
> Key: MAPREDUCE-4253
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4253
> Project: Hadoop Map/Reduce
>  Issue Type: Test
>  Components: client
>Affects Versions: 2.0.0-alpha
>Reporter: Harsh J
>Assignee: Tsuyoshi Ozawa
> Attachments: MR-4253.1.patch, MR-4253.2.patch, 
> crossing_project_checker.rb, result.txt
>
>
> Many of the tests for client libs from mapreduce-client-core are lying under 
> mapreduce-client-jobclient.
> We should investigate if this is the right thing to do and if not, move the 
> tests back into client-core.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Updated] (MAPREDUCE-4683) Create and distribute hadoop-mapreduce-client-core-tests.jar

2016-12-07 Thread Akira Ajisaka (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-4683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira Ajisaka updated MAPREDUCE-4683:
-
   Resolution: Fixed
Fix Version/s: 3.0.0-alpha2
   Status: Resolved  (was: Patch Available)

Committed this to trunk. Thanks [~templedf] for the review!

> Create and distribute hadoop-mapreduce-client-core-tests.jar
> 
>
> Key: MAPREDUCE-4683
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4683
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: build
>Reporter: Arun C Murthy
>Assignee: Akira Ajisaka
>Priority: Critical
> Fix For: 3.0.0-alpha2
>
> Attachments: MAPREDUCE-4683.patch
>
>
> We need to fix our build to create/distribute 
> hadoop-mapreduce-client-core-tests.jar, need this before MAPREDUCE-4253



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Updated] (MAPREDUCE-4683) Create and distribute hadoop-mapreduce-client-core-tests.jar

2016-12-07 Thread Akira Ajisaka (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-4683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira Ajisaka updated MAPREDUCE-4683:
-
Hadoop Flags: Incompatible change
Release Note: hadoop-mapreduce-client-core module now creates and 
distributes test jar.
 Summary: Create and distribute hadoop-mapreduce-client-core-tests.jar  
(was: We need to fix our build to create/distribute 
hadoop-mapreduce-client-core-tests.jar)

> Create and distribute hadoop-mapreduce-client-core-tests.jar
> 
>
> Key: MAPREDUCE-4683
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4683
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: build
>Reporter: Arun C Murthy
>Assignee: Akira Ajisaka
>Priority: Critical
> Attachments: MAPREDUCE-4683.patch
>
>
> We need to fix our build to create/distribute 
> hadoop-mapreduce-client-core-tests.jar, need this before MAPREDUCE-4253



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Updated] (MAPREDUCE-6734) Add option to distcp to preserve file path structure of source files at the destination

2016-12-07 Thread Frederick Tucker (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Frederick Tucker updated MAPREDUCE-6734:

Priority: Major  (was: Critical)

> Add option to distcp to preserve file path structure of source files at the 
> destination
> ---
>
> Key: MAPREDUCE-6734
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6734
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: distcp
>Affects Versions: 3.0.0-alpha2
> Environment: Software platform
>Reporter: Frederick Tucker
>  Labels: distcp, newbie, patch
> Fix For: 3.0.0-alpha2
>
> Attachments: MAPREDUCE-6734.3.0.0-alpha2.patch, 
> MAPREDUCE-6734.3.0.0-alpha2.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> When copying files using distcp with globbed source files, all the matched 
> files in the glob are copied in a single flat directory.  This causes 
> problems when the file structure at the source is important.  It also is an 
> issue when there are two files matched in the glob with the same name because 
> it causes a duplicate file error at the target.  I'd like to have an option 
> to preserve the file structure of the source files when globbing inputs.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Comment Edited] (MAPREDUCE-6734) Add option to distcp to preserve file path structure of source files at the destination

2016-12-07 Thread Daniel Templeton (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15730367#comment-15730367
 ] 

Daniel Templeton edited comment on MAPREDUCE-6734 at 12/8/16 12:24 AM:
---

I meant the priority.

Oh, whoops.  I meant bump the priority down to Major.


was (Author: templedf):
I meant the priority.

> Add option to distcp to preserve file path structure of source files at the 
> destination
> ---
>
> Key: MAPREDUCE-6734
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6734
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: distcp
>Affects Versions: 3.0.0-alpha2
> Environment: Software platform
>Reporter: Frederick Tucker
>Priority: Critical
>  Labels: distcp, newbie, patch
> Fix For: 3.0.0-alpha2
>
> Attachments: MAPREDUCE-6734.3.0.0-alpha2.patch, 
> MAPREDUCE-6734.3.0.0-alpha2.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> When copying files using distcp with globbed source files, all the matched 
> files in the glob are copied in a single flat directory.  This causes 
> problems when the file structure at the source is important.  It also is an 
> issue when there are two files matched in the glob with the same name because 
> it causes a duplicate file error at the target.  I'd like to have an option 
> to preserve the file structure of the source files when globbing inputs.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-6734) Add option to distcp to preserve file path structure of source files at the destination

2016-12-07 Thread Daniel Templeton (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15730406#comment-15730406
 ] 

Daniel Templeton commented on MAPREDUCE-6734:
-

Comments:

* In {{CopyListing}}, I wouldn't make the call to {{adjustPath()}} an 
assignment.  You're modifying the {{Text}} in the method, so the assignment 
just looks suspicious.  Same in {{CopyMapper.map()}}.
* In {{DistCpUtils.adjustPath()}}, the javadoc probably shouldn't talk about 
keys and values since it's used in a broader context, e.g. {{CopyListing}}
* {{adjustPath()}} doesn't tolerate leaving out the leading slash.  Seems a 
logical thing for a user to want to do.
* In your tests, please add assert messages, especially for 
{{assertTrue()}}/{{assertFalse()}}
* In the fail message, if it includes an exception, please use {{toString()}} 
instead of {{getMessage()}}
* There's a lot of overlap between the tests in {{TestCopyMapper}}.  Think you 
can extract the common logic into a shared method?  Also, it would be nice to 
add some cleanup for the created files.  I don't see any other methods doing 
that, but maybe you'll start a trend!
* In the {{TestCopyListing}} test, you don't need the first 
{{TestDistCpUtils.delete(fs, "/tmp");}}.  I'd also make the 
{{InvalidInputException}} expected instead of catching and ignoring
* In the {{TestDistCpUtils}} test, I think you can move that repeated logic 
into a separate method and call that four times

> Add option to distcp to preserve file path structure of source files at the 
> destination
> ---
>
> Key: MAPREDUCE-6734
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6734
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: distcp
>Affects Versions: 3.0.0-alpha2
> Environment: Software platform
>Reporter: Frederick Tucker
>Priority: Critical
>  Labels: distcp, newbie, patch
> Fix For: 3.0.0-alpha2
>
> Attachments: MAPREDUCE-6734.3.0.0-alpha2.patch, 
> MAPREDUCE-6734.3.0.0-alpha2.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> When copying files using distcp with globbed source files, all the matched 
> files in the glob are copied in a single flat directory.  This causes 
> problems when the file structure at the source is important.  It also is an 
> issue when there are two files matched in the glob with the same name because 
> it causes a duplicate file error at the target.  I'd like to have an option 
> to preserve the file structure of the source files when globbing inputs.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-6734) Add option to distcp to preserve file path structure of source files at the destination

2016-12-07 Thread Daniel Templeton (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15730367#comment-15730367
 ] 

Daniel Templeton commented on MAPREDUCE-6734:
-

I meant the priority.

> Add option to distcp to preserve file path structure of source files at the 
> destination
> ---
>
> Key: MAPREDUCE-6734
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6734
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: distcp
>Affects Versions: 3.0.0-alpha2
> Environment: Software platform
>Reporter: Frederick Tucker
>Priority: Critical
>  Labels: distcp, newbie, patch
> Fix For: 3.0.0-alpha2
>
> Attachments: MAPREDUCE-6734.3.0.0-alpha2.patch, 
> MAPREDUCE-6734.3.0.0-alpha2.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> When copying files using distcp with globbed source files, all the matched 
> files in the glob are copied in a single flat directory.  This causes 
> problems when the file structure at the source is important.  It also is an 
> issue when there are two files matched in the glob with the same name because 
> it causes a duplicate file error at the target.  I'd like to have an option 
> to preserve the file structure of the source files when globbing inputs.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-4683) We need to fix our build to create/distribute hadoop-mapreduce-client-core-tests.jar

2016-12-07 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-4683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15730356#comment-15730356
 ] 

Hadoop QA commented on MAPREDUCE-4683:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s 
{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m 
42s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 27s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 32s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 22s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 27s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m 1s 
{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 1s 
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed. 
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
15s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 6s {color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:a9ad5d6 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12669891/MAPREDUCE-4683.patch |
| JIRA Issue | MAPREDUCE-4683 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  xml  |
| uname | Linux f3afc556a990 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / 72fe546 |
| Default Java | 1.8.0_111 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6836/testReport/ |
| modules | C: 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core 
U: 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core |
| Console output | 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6836/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> We need to fix our build to create/distribute 
> hadoop-mapreduce-client-core-tests.jar
> 
>
> Key: MAPREDUCE-4683
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4683
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: build
>Reporter: Arun C Murthy
>Assignee: Akira Ajisaka
>Priority: Critical
> Attachments: MAPREDUCE-4683.patch
>
>
> We need to fix our build to create/distribute 
> hadoop-mapreduce-client-core-tests.jar, need this before MAPREDUCE-4253



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To u

[jira] [Commented] (MAPREDUCE-6734) Add option to distcp to preserve file path structure of source files at the destination

2016-12-07 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15730344#comment-15730344
 ] 

Hadoop QA commented on MAPREDUCE-6734:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s {color} 
| {color:red} MAPREDUCE-6734 does not apply to trunk. Rebase required? Wrong 
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12818705/MAPREDUCE-6734.3.0.0-alpha2.patch
 |
| JIRA Issue | MAPREDUCE-6734 |
| Console output | 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6837/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Add option to distcp to preserve file path structure of source files at the 
> destination
> ---
>
> Key: MAPREDUCE-6734
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6734
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: distcp
>Affects Versions: 3.0.0-alpha2
> Environment: Software platform
>Reporter: Frederick Tucker
>Priority: Critical
>  Labels: distcp, newbie, patch
> Fix For: 3.0.0-alpha2
>
> Attachments: MAPREDUCE-6734.3.0.0-alpha2.patch, 
> MAPREDUCE-6734.3.0.0-alpha2.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> When copying files using distcp with globbed source files, all the matched 
> files in the glob are copied in a single flat directory.  This causes 
> problems when the file structure at the source is important.  It also is an 
> issue when there are two files matched in the glob with the same name because 
> it causes a duplicate file error at the target.  I'd like to have an option 
> to preserve the file structure of the source files when globbing inputs.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-6734) Add option to distcp to preserve file path structure of source files at the destination

2016-12-07 Thread Frederick Tucker (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15730343#comment-15730343
 ] 

Frederick Tucker commented on MAPREDUCE-6734:
-

[~templedf] Thanks a lot for taking a look.  Sure thing we can change it to 
important - do I just check Important under 'Flags'?

> Add option to distcp to preserve file path structure of source files at the 
> destination
> ---
>
> Key: MAPREDUCE-6734
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6734
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: distcp
>Affects Versions: 3.0.0-alpha2
> Environment: Software platform
>Reporter: Frederick Tucker
>Priority: Critical
>  Labels: distcp, newbie, patch
> Fix For: 3.0.0-alpha2
>
> Attachments: MAPREDUCE-6734.3.0.0-alpha2.patch, 
> MAPREDUCE-6734.3.0.0-alpha2.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> When copying files using distcp with globbed source files, all the matched 
> files in the glob are copied in a single flat directory.  This causes 
> problems when the file structure at the source is important.  It also is an 
> issue when there are two files matched in the glob with the same name because 
> it causes a duplicate file error at the target.  I'd like to have an option 
> to preserve the file structure of the source files when globbing inputs.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-6734) Add option to distcp to preserve file path structure of source files at the destination

2016-12-07 Thread Daniel Templeton (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15730316#comment-15730316
 ] 

Daniel Templeton commented on MAPREDUCE-6734:
-

Thanks, [~fctucker], for the patch.  I'm taking a look at it now.  In the 
meantime, can we drop this to Important?

> Add option to distcp to preserve file path structure of source files at the 
> destination
> ---
>
> Key: MAPREDUCE-6734
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6734
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: distcp
>Affects Versions: 3.0.0-alpha2
> Environment: Software platform
>Reporter: Frederick Tucker
>Priority: Critical
>  Labels: distcp, newbie, patch
> Fix For: 3.0.0-alpha2
>
> Attachments: MAPREDUCE-6734.3.0.0-alpha2.patch, 
> MAPREDUCE-6734.3.0.0-alpha2.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> When copying files using distcp with globbed source files, all the matched 
> files in the glob are copied in a single flat directory.  This causes 
> problems when the file structure at the source is important.  It also is an 
> issue when there are two files matched in the glob with the same name because 
> it causes a duplicate file error at the target.  I'd like to have an option 
> to preserve the file structure of the source files when globbing inputs.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-4683) We need to fix our build to create/distribute hadoop-mapreduce-client-core-tests.jar

2016-12-07 Thread Daniel Templeton (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-4683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15730298#comment-15730298
 ] 

Daniel Templeton commented on MAPREDUCE-4683:
-

Seems fine to me.  +1  Shall we get this in?

> We need to fix our build to create/distribute 
> hadoop-mapreduce-client-core-tests.jar
> 
>
> Key: MAPREDUCE-4683
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4683
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: build
>Reporter: Arun C Murthy
>Assignee: Akira Ajisaka
>Priority: Critical
> Attachments: MAPREDUCE-4683.patch
>
>
> We need to fix our build to create/distribute 
> hadoop-mapreduce-client-core-tests.jar, need this before MAPREDUCE-4253



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-6704) Container fail to launch for mapred application

2016-12-07 Thread Daniel Templeton (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15730181#comment-15730181
 ] 

Daniel Templeton commented on MAPREDUCE-6704:
-

Works for me.

> Container fail to launch for mapred application
> ---
>
> Key: MAPREDUCE-6704
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6704
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Blocker
> Attachments: 0001-MAPREDUCE-6704.patch, 0001-YARN-5026.patch, 
> ClusterSetup.html, MAPREDUCE-6704.0002.patch, MR-6704-branch2.8.tar.gz, 
> MR-6704-trunk-tempPatch.tar.gz, MR-6704-trunk.tar.gz, SingleCluster.html, 
> container-whitelist-env-wip.patch, temp.patch
>
>
> Container fail to launch for mapred application.
> As part for launch script {{HADOOP_MAPRED_HOME}} default value is not set 
> .After 
> https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c
>{{HADOOP_MAPRED_HOME}} is not able to get from {{builder.environment()}} 
> since {{DefaultContainerExecutor#buildCommandExecutor}} sets inherit to false.
> {noformat}
> 16/05/02 09:16:05 INFO mapreduce.Job: Job job_1462155939310_0004 failed with 
> state FAILED due to: Application application_1462155939310_0004 failed 2 
> times due to AM Container for appattempt_1462155939310_0004_02 exited 
> with  exitCode: 1
> Failing this attempt.Diagnostics: Exception from container-launch.
> Container id: container_1462155939310_0004_02_01
> Exit code: 1
> Stack trace: ExitCodeException exitCode=1:
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:946)
> at org.apache.hadoop.util.Shell.run(Shell.java:850)
> at 
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:1144)
> at 
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:227)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.launchContainer(ContainerLaunch.java:385)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:281)
> at 
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:89)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier; 
> support was removed in 8.0
> Error: Could not find or load main class 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster
> Container exited with a non-zero exit code 1. Last 4096 bytes of stderr :
> Java HotSpot(TM) 64-Bit Server VM warning: ignoring option UseSplitVerifier; 
> support was removed in 8.0
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Updated] (MAPREDUCE-6506) Make the reducer-preemption configs consistent in how they handle defaults

2016-12-07 Thread Haibo Chen (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haibo Chen updated MAPREDUCE-6506:
--
Assignee: (was: Haibo Chen)

> Make the reducer-preemption configs consistent in how they handle defaults
> --
>
> Key: MAPREDUCE-6506
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6506
> Project: Hadoop Map/Reduce
>  Issue Type: Task
>  Components: applicationmaster
>Affects Versions: 2.8.0
>Reporter: Karthik Kambatla
>Priority: Critical
>
> {{mapreduce.job.reducer.preempt.delay.sec}} and 
> {{mapreduce.job.reducer.unconditional-preempt.delay.sec}} are two configs 
> related to reducer preemption. These configs are not consistent in how they 
> handle non-positive values. Also, the way to disable them is different. 
> It would be nice to make them consistent somehow, and change the behavior of 
> former if need be. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-6817) The format of job start time in JHS is different from those of submit and finish time

2016-12-07 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15730056#comment-15730056
 ] 

Hudson commented on MAPREDUCE-6817:
---

SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10962 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/10962/])
MAPREDUCE-6817. The format of job start time in JHS is different from (kasha: 
rev a793cec2c9a7ad80aaf67be9dec2245e8ecce63c)
* (edit) 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/webapp/dao/TestJobInfo.java
* (edit) 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/HsJobsBlock.java
* (edit) 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/webapp/dao/JobInfo.java


> The format of job start time in JHS is different from those of submit and 
> finish time
> -
>
> Key: MAPREDUCE-6817
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6817
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: jobhistoryserver
>Affects Versions: 3.0.0-alpha1
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Fix For: 2.9.0, 3.0.0-alpha2
>
> Attachments: MAPREDUCE-6817.01.patch, MAPREDUCE-6817.02.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Updated] (MAPREDUCE-6817) The format of job start time in JHS is different from those of submit and finish time

2016-12-07 Thread Karthik Kambatla (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Karthik Kambatla updated MAPREDUCE-6817:

   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 3.0.0-alpha2
   2.9.0
   Status: Resolved  (was: Patch Available)

> The format of job start time in JHS is different from those of submit and 
> finish time
> -
>
> Key: MAPREDUCE-6817
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6817
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: jobhistoryserver
>Affects Versions: 3.0.0-alpha1
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Fix For: 2.9.0, 3.0.0-alpha2
>
> Attachments: MAPREDUCE-6817.01.patch, MAPREDUCE-6817.02.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-6817) The format of job start time in JHS is different from those of submit and finish time

2016-12-07 Thread Karthik Kambatla (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15729994#comment-15729994
 ] 

Karthik Kambatla commented on MAPREDUCE-6817:
-

Thanks for the fix, [~haibochen]. Just committed this to trunk and branch-2. 

> The format of job start time in JHS is different from those of submit and 
> finish time
> -
>
> Key: MAPREDUCE-6817
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6817
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: jobhistoryserver
>Affects Versions: 3.0.0-alpha1
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Attachments: MAPREDUCE-6817.01.patch, MAPREDUCE-6817.02.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-6817) The format of job start time in JHS is different from those of submit and finish time

2016-12-07 Thread Karthik Kambatla (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15729987#comment-15729987
 ] 

Karthik Kambatla commented on MAPREDUCE-6817:
-

+1

> The format of job start time in JHS is different from those of submit and 
> finish time
> -
>
> Key: MAPREDUCE-6817
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6817
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: jobhistoryserver
>Affects Versions: 3.0.0-alpha1
>Reporter: Haibo Chen
>Assignee: Haibo Chen
> Attachments: MAPREDUCE-6817.01.patch, MAPREDUCE-6817.02.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-5155) Race condition in test case TestFetchFailure cause it to fail

2016-12-07 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15729944#comment-15729944
 ] 

Hadoop QA commented on MAPREDUCE-5155:
--

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 19s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 
0s {color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m 
52s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
16s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 30s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 
37s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 
22s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 21s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 21s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 26s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 55s 
{color} | {color:green} hadoop-mapreduce-client-app in the patch passed. 
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
16s {color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 44s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:a9ad5d6 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12842209/MAPREDUCE-5155.01.patch
 |
| JIRA Issue | MAPREDUCE-5155 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux 2b499480ce91 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 
17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / 9f5d2c4 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6835/testReport/ |
| modules | C: 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app |
| Console output | 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6835/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Race condition in test case TestFetchFailure cause it to fail
> -
>
> Key: MAPREDUCE-5155
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5155
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.0.3-alpha
> Environment: Suse x86_64 GNU/Linux
> Java(TM) SE Runtime Environment (build 1.6.0_32-b05
>Reporter: Nemon Lou

[jira] [Updated] (MAPREDUCE-5155) Race condition in test case TestFetchFailure cause it to fail

2016-12-07 Thread Haibo Chen (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haibo Chen updated MAPREDUCE-5155:
--
Status: Patch Available  (was: Open)

> Race condition in test case TestFetchFailure cause it to fail
> -
>
> Key: MAPREDUCE-5155
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5155
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.0.3-alpha
> Environment: Suse x86_64 GNU/Linux
> Java(TM) SE Runtime Environment (build 1.6.0_32-b05
>Reporter: Nemon Lou
>Assignee: Haibo Chen
>Priority: Minor
> Attachments: MAPREDUCE-5155.01.patch, 
> org.apache.hadoop.mapreduce.v2.app.TestFetchFailure-output.txt, 
> org.apache.hadoop.mapreduce.v2.app.TestFetchFailure.txt
>
>
> I run into this once: 
> testFetchFailureWithRecovery(org.apache.hadoop.mapreduce.v2.app.TestFetchFailure):
>  Num completion events not correct expected:<1> but was:<0>
> There is a race condition between job.getTaskAttemptCompletionEvents and 
> dealing with JOB_TASK_ATTEMPT_COMPLETED event.
> If job.getTaskAttemptCompletionEvents invoked because of task in SUCCEEDED 
> state ,but before JOB_TASK_ATTEMPT_COMPLETED event scheduled,the test case 
> will fail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Updated] (MAPREDUCE-5155) Race condition in test case TestFetchFailure cause it to fail

2016-12-07 Thread Haibo Chen (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haibo Chen updated MAPREDUCE-5155:
--
Attachment: MAPREDUCE-5155.01.patch

Upload a patch that contains the same fix in in MAPREDUCE-4785

> Race condition in test case TestFetchFailure cause it to fail
> -
>
> Key: MAPREDUCE-5155
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5155
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.0.3-alpha
> Environment: Suse x86_64 GNU/Linux
> Java(TM) SE Runtime Environment (build 1.6.0_32-b05
>Reporter: Nemon Lou
>Assignee: Haibo Chen
>Priority: Minor
> Attachments: MAPREDUCE-5155.01.patch, 
> org.apache.hadoop.mapreduce.v2.app.TestFetchFailure-output.txt, 
> org.apache.hadoop.mapreduce.v2.app.TestFetchFailure.txt
>
>
> I run into this once: 
> testFetchFailureWithRecovery(org.apache.hadoop.mapreduce.v2.app.TestFetchFailure):
>  Num completion events not correct expected:<1> but was:<0>
> There is a race condition between job.getTaskAttemptCompletionEvents and 
> dealing with JOB_TASK_ATTEMPT_COMPLETED event.
> If job.getTaskAttemptCompletionEvents invoked because of task in SUCCEEDED 
> state ,but before JOB_TASK_ATTEMPT_COMPLETED event scheduled,the test case 
> will fail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Commented] (MAPREDUCE-5155) Race condition in test case TestFetchFailure cause it to fail

2016-12-07 Thread Haibo Chen (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15729837#comment-15729837
 ] 

Haibo Chen commented on MAPREDUCE-5155:
---

TestMRApp.testUpdatedNodes() has been fixed in MAPREDUCE-4785

> Race condition in test case TestFetchFailure cause it to fail
> -
>
> Key: MAPREDUCE-5155
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5155
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.0.3-alpha
> Environment: Suse x86_64 GNU/Linux
> Java(TM) SE Runtime Environment (build 1.6.0_32-b05
>Reporter: Nemon Lou
>Assignee: Haibo Chen
>Priority: Minor
> Attachments: 
> org.apache.hadoop.mapreduce.v2.app.TestFetchFailure-output.txt, 
> org.apache.hadoop.mapreduce.v2.app.TestFetchFailure.txt
>
>
> I run into this once: 
> testFetchFailureWithRecovery(org.apache.hadoop.mapreduce.v2.app.TestFetchFailure):
>  Num completion events not correct expected:<1> but was:<0>
> There is a race condition between job.getTaskAttemptCompletionEvents and 
> dealing with JOB_TASK_ATTEMPT_COMPLETED event.
> If job.getTaskAttemptCompletionEvents invoked because of task in SUCCEEDED 
> state ,but before JOB_TASK_ATTEMPT_COMPLETED event scheduled,the test case 
> will fail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org



[jira] [Assigned] (MAPREDUCE-5155) Race condition in test case TestFetchFailure cause it to fail

2016-12-07 Thread Haibo Chen (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Haibo Chen reassigned MAPREDUCE-5155:
-

Assignee: Haibo Chen

> Race condition in test case TestFetchFailure cause it to fail
> -
>
> Key: MAPREDUCE-5155
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5155
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.0.3-alpha
> Environment: Suse x86_64 GNU/Linux
> Java(TM) SE Runtime Environment (build 1.6.0_32-b05
>Reporter: Nemon Lou
>Assignee: Haibo Chen
>Priority: Minor
> Attachments: 
> org.apache.hadoop.mapreduce.v2.app.TestFetchFailure-output.txt, 
> org.apache.hadoop.mapreduce.v2.app.TestFetchFailure.txt
>
>
> I run into this once: 
> testFetchFailureWithRecovery(org.apache.hadoop.mapreduce.v2.app.TestFetchFailure):
>  Num completion events not correct expected:<1> but was:<0>
> There is a race condition between job.getTaskAttemptCompletionEvents and 
> dealing with JOB_TASK_ATTEMPT_COMPLETED event.
> If job.getTaskAttemptCompletionEvents invoked because of task in SUCCEEDED 
> state ,but before JOB_TASK_ATTEMPT_COMPLETED event scheduled,the test case 
> will fail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org