[
https://issues.apache.org/jira/browse/MAPREDUCE-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13169721#comment-13169721
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-3531:
----------------------------------------------------
+! for the fix as is. Atleast we could use that brute-force test to reproduce
it sometimes, validating the fix is mostly the correct one.
> Sometimes java.lang.IllegalArgumentException: Invalid key to HMAC computation
> in NODE_UPDATE also causing RM to stop scheduling
> --------------------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-3531
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3531
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2, resourcemanager, scheduler
> Affects Versions: 0.23.1
> Reporter: Karam Singh
> Assignee: Robert Joseph Evans
> Priority: Blocker
> Fix For: 0.23.1, 0.24.0
>
> Attachments: MR-3531.txt, MR-3531.txt
>
>
> Filling this Jira a bit late
> Started 350 cluster
> sbummited large sleep job.
> Foud that job was not running as RM has not allocated resouces to it.
> {code}
> 2011-12-01 11:56:25,200 INFO
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler:
> nodeUpdate: <NMHost>:48490 clusterResources: memory: 3225600
> 2011-12-01 11:56:25,202 ERROR
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in
> handling event
> type NODE_UPDATE to the scheduler
> java.lang.IllegalArgumentException: Invalid key to HMAC computation
> at
> org.apache.hadoop.security.token.SecretManager.createPassword(SecretManager.java:141)
> at
> org.apache.hadoop.yarn.server.security.ContainerTokenSecretManager.createPassword(ContainerTokenSecretManager.java:61)
>
> atorg.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.createContainer(LeafQueue.java:1108)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.getContainer(LeafQueue.java:1091)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.assignContainer(LeafQueue.java:1137)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.assignNodeLocalContainers(LeafQueue.java:1001)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.assignContainersOnNode(LeafQueue.java:973)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.assignContainers(LeafQueue.java:760)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainersToChildQueues(ParentQueue.java:583)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.assignContainers(ParentQueue.java:513)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:569)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:611)
> at
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
> at java.lang.Thread.run(Thread.java:619)
> Caused by: java.security.InvalidKeyException: Secret key expected
> at com.sun.crypto.provider.HmacCore.a(DashoA13*..)
> at com.sun.crypto.provider.HmacSHA1.engineInit(DashoA13*..)
> at javax.crypto.Mac.init(DashoA13*..)
> at
> org.apache.hadoop.security.token.SecretManager.createPassword(SecretManager.java:139)
> ... 14 more
> {code}
> As this stack is from 30 Nov checkou line number may be different
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
