[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16434128#comment-16434128
 ] 

Hadoop QA commented on MAPREDUCE-6826:
--------------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m 
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 26m 
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 18m 
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
29m 53s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color} 
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m  
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  4m 
57s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m 
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 25m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 25m 
42s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 19m  
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}  
9m 19s{color} | {color:green} patch has no errors when building and testing our 
client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Skipped patched modules with no Java source: . {color} 
|
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  5m 
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}127m 43s{color} 
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
36s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}299m 18s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.web.TestWebHdfsTimeouts |
|   | hadoop.hdfs.TestEncryptionZonesWithKMS |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b |
| JIRA Issue | MAPREDUCE-6826 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12914665/MAPREDUCE-6826-001.patch
 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  shadedclient  findbugs  checkstyle  |
| uname | Linux 9ec1abd78009 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 7eb783e |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_162 |
| unit | 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7385/artifact/out/patch-unit-root.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7385/testReport/ |
| Max. process+thread count | 4796 (vs. ulimit of 10000) |
| modules | C: . U: . |
| Console output | 
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7385/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Job fails with InvalidStateTransitonException: Invalid event: 
> JOB_TASK_COMPLETED at SUCCEEDED
> ---------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6826
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6826
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.7.2
>            Reporter: Varun Saxena
>            Assignee: Bilwa S T
>            Priority: Major
>         Attachments: MAPREDUCE-6826-001.patch
>
>
> This happens if a container is preempted by scheduler after job starts 
> committing.
> And this exception in turn leads to application being marked as FAILED in 
> YARN.
> I think we can probably ignore JOB_TASK_COMPLETED event while JobImpl state 
> is COMMITTING or SUCCEEDED as job is in the process of finishing.
> Also is there any point in attempting to scheduler another task attempt if 
> job is already in COMMITTING or SUCCEEDED state.
> {noformat}
> 2016-12-23 09:10:38,642 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: 
> task_1482404625971_23910_m_000004 Task Transitioned from RUNNING to SUCCEEDED
> 2016-12-23 09:10:38,642 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 5
> 2016-12-23 09:10:38,643 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: 
> job_1482404625971_23910Job Transitioned from RUNNING to COMMITTING
> 2016-12-23 09:10:38,644 INFO [ContainerLauncher #5] 
> org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing 
> the event EventType: CONTAINER_REMOTE_CLEANUP for container 
> container_e55_1482404625971_23910_01_000010 taskAttempt 
> attempt_1482404625971_23910_m_000004_1
> 2016-12-23 09:10:38,644 INFO [ContainerLauncher #5] 
> org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING 
> attempt_1482404625971_23910_m_000004_1
> 2016-12-23 09:10:38,644 INFO [ContainerLauncher #5] 
> org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: 
> Opening proxy : linux-19:26009
> 2016-12-23 09:10:38,644 INFO [CommitterEvent Processor #4] 
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing 
> the event EventType: JOB_COMMIT
> 2016-12-23 09:10:38,724 INFO [IPC Server handler 0 on 27113] 
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : 
> jvm_1482404625971_23910_m_60473139527690 asked for a task
> 2016-12-23 09:10:38,724 INFO [IPC Server handler 0 on 27113] 
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: 
> jvm_1482404625971_23910_m_60473139527690 is invalid and will be killed.
> 2016-12-23 09:10:38,797 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Calling handler for 
> JobFinishedEvent 
> 2016-12-23 09:10:38,797 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: 
> job_1482404625971_23910Job Transitioned from COMMITTING to SUCCEEDED
> 2016-12-23 09:10:38,798 INFO [Thread-93] 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Job finished cleanly, 
> recording last MRAppMaster retry
> 2016-12-23 09:10:38,798 INFO [Thread-93] 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator 
> isAMLastRetry: true
> 2016-12-23 09:10:38,798 INFO [Thread-93] 
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: RMCommunicator notified 
> that shouldUnregistered is: true
> 2016-12-23 09:10:38,799 INFO [Thread-93] 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: 
> true
> 2016-12-23 09:10:38,799 INFO [Thread-93] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: 
> JobHistoryEventHandler notified that forceJobCompletion is true
> 2016-12-23 09:10:38,799 INFO [Thread-93] 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the 
> services
> 2016-12-23 09:10:38,800 INFO [Thread-93] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping 
> JobHistoryEventHandler. Size of the outstanding queue size is 1
> 2016-12-23 09:10:38,989 INFO [RMCommunicator Allocator] 
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before 
> Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 
> AssignedReds:0 CompletedMaps:5 CompletedReds:0 ContAlloc:8 ContRel:0 
> HostLocal:0 RackLocal:0
> 2016-12-23 09:10:38,993 INFO [RMCommunicator Allocator] 
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received 
> completed container container_e55_1482404625971_23910_01_000010
> 2016-12-23 09:10:38,993 INFO [RMCommunicator Allocator] 
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: 
> PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 
> CompletedMaps:5 CompletedReds:0 ContAlloc:8 ContRel:0 HostLocal:0 RackLocal:0
> 2016-12-23 09:10:38,993 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics 
> report from attempt_1482404625971_23910_m_000004_1: Container preempted by 
> scheduler
> 2016-12-23 09:10:38,994 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: 
> attempt_1482404625971_23910_m_000004_1 TaskAttempt Transitioned from 
> SUCCEEDED to KILLED
> 2016-12-23 09:10:38,995 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: 
> task_1482404625971_23910_m_000004 Task Transitioned from SUCCEEDED to 
> SCHEDULED
> 2016-12-23 09:10:38,996 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: 
> attempt_1482404625971_23910_m_000004_2 TaskAttempt Transitioned from NEW to 
> UNASSIGNED
> 2016-12-23 09:10:39,044 INFO [RMCommunicator Allocator] 
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before 
> Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:0 
> AssignedReds:0 CompletedMaps:5 CompletedReds:0 ContAlloc:8 ContRel:0 
> HostLocal:0 RackLocal:0
> 2016-12-23 09:10:39,050 INFO [RMCommunicator Allocator] 
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() 
> for application_1482404625971_23910: ask=1 release= 0 newContainers=0 
> finishedContainers=0 resourcelimit=<memory:0, vCores:0> knownNMs=11
> 2016-12-23 09:10:40,053 INFO [RMCommunicator Allocator] 
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated 
> containers 1
> 2016-12-23 09:10:40,055 INFO [RMCommunicator Allocator] 
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned 
> container container_e55_1482404625971_23910_01_000011 to 
> attempt_1482404625971_23910_m_000004_2
> 2016-12-23 09:10:40,055 INFO [RMCommunicator Allocator] 
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: 
> PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:0 
> CompletedMaps:5 CompletedReds:0 ContAlloc:9 ContRel:0 HostLocal:0 RackLocal:0
> 2016-12-23 09:10:40,056 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: 
> attempt_1482404625971_23910_m_000004_2 TaskAttempt Transitioned from 
> UNASSIGNED to ASSIGNED
> 2016-12-23 09:10:40,056 INFO [ContainerLauncher #6] 
> org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing 
> the event EventType: CONTAINER_REMOTE_LAUNCH for container 
> container_e55_1482404625971_23910_01_000011 taskAttempt 
> attempt_1482404625971_23910_m_000004_2
> 2016-12-23 09:10:40,056 INFO [ContainerLauncher #6] 
> org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching 
> attempt_1482404625971_23910_m_000004_2
> 2016-12-23 09:10:40,056 INFO [ContainerLauncher #6] 
> org.apache.hadoop.yarn.client.api.impl.ContainerManagementProtocolProxy: 
> Opening proxy : linux-17:26009
> 2016-12-23 09:10:40,068 INFO [ContainerLauncher #6] 
> org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle 
> port returned by ContainerManager for attempt_1482404625971_23910_m_000004_2 
> : 26008
> 2016-12-23 09:10:40,074 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: 
> [attempt_1482404625971_23910_m_000004_2] using containerId: 
> [container_e55_1482404625971_23910_01_000011 on NM: [linux-17:26009]
> 2016-12-23 09:10:40,074 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: 
> attempt_1482404625971_23910_m_000004_2 TaskAttempt Transitioned from ASSIGNED 
> to RUNNING
> 2016-12-23 09:10:40,075 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: 
> task_1482404625971_23910_m_000004 Task Transitioned from SCHEDULED to RUNNING
> 2016-12-23 09:10:40,108 INFO [RMCommunicator Allocator] 
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: getResources() 
> for application_1482404625971_23910: ask=1 release= 0 newContainers=0 
> finishedContainers=0 resourcelimit=<memory:0, vCores:0> knownNMs=11
> 2016-12-23 09:10:42,949 INFO [Socket Reader #1 for port 27113] 
> SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for 
> job_1482404625971_23910 (auth:SIMPLE)
> 2016-12-23 09:10:42,995 INFO [IPC Server handler 6 on 27113] 
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : 
> jvm_1482404625971_23910_m_60473139527691 asked for a task
> 2016-12-23 09:10:42,995 INFO [IPC Server handler 6 on 27113] 
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: 
> jvm_1482404625971_23910_m_60473139527691 given task: 
> attempt_1482404625971_23910_m_000004_2
> 2016-12-23 09:10:46,163 INFO [IPC Server handler 29 on 27113] 
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update 
> from attempt_1482404625971_23910_m_000004_2
> 2016-12-23 09:10:46,165 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: 
> attempt_1482404625971_23910_m_000004_2 TaskAttempt Transitioned from RUNNING 
> to COMMIT_PENDING
> 2016-12-23 09:10:46,165 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: 
> attempt_1482404625971_23910_m_000004_2 given a go for committing the task 
> output.
> 2016-12-23 09:10:46,166 INFO [IPC Server handler 9 on 27113] 
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request 
> from attempt_1482404625971_23910_m_000004_2
> 2016-12-23 09:10:46,167 INFO [IPC Server handler 9 on 27113] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for 
> attempt_1482404625971_23910_m_000004_2:true
> 2016-12-23 09:10:46,265 INFO [IPC Server handler 20 on 27113] 
> org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgement from 
> attempt_1482404625971_23910_m_000004_2
> 2016-12-23 09:10:46,267 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: 
> attempt_1482404625971_23910_m_000004_2 TaskAttempt Transitioned from 
> COMMIT_PENDING to SUCCEEDED
> 2016-12-23 09:10:46,267 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with 
> attempt attempt_1482404625971_23910_m_000004_2
> 2016-12-23 09:10:46,267 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: 
> task_1482404625971_23910_m_000004 Task Transitioned from RUNNING to SUCCEEDED
> 2016-12-23 09:10:46,267 ERROR [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Can't handle this event 
> at current state
> org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: 
> JOB_TASK_COMPLETED at SUCCEEDED
>       at 
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
>       at 
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
>       at 
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
>       at 
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:997)
>       at 
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl.handle(JobImpl.java:139)
>       at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1399)
>       at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher.handle(MRAppMaster.java:1395)
>       at 
> org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:192)
>       at 
> org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:118)
>       at java.lang.Thread.run(Thread.java:745)
> 2016-12-23 09:10:46,270 INFO [AsyncDispatcher event handler] 
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: 
> job_1482404625971_23910Job Transitioned from SUCCEEDED to ERROR
> 2016-12-23 09:10:46,271 INFO [Thread-97] 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator 
> isAMLastRetry: true
> 2016-12-23 09:10:46,271 INFO [Thread-97] 
> org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: RMCommunicator notified 
> that shouldUnregistered is: true
> 2016-12-23 09:10:46,271 INFO [Thread-97] 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: 
> true
> 2016-12-23 09:10:46,271 INFO [Thread-97] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: 
> JobHistoryEventHandler notified that forceJobCompletion is true
> 2016-12-23 09:10:46,271 INFO [Thread-97] 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the 
> services
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org

Reply via email to