András Győri created YARN-11147:
-----------------------------------
Summary: ResourceUsage and QueueCapacities classes provide node
label iterators that are not thread safe
Key: YARN-11147
URL: https://issues.apache.org/jira/browse/YARN-11147
Project: Hadoop YARN
Issue Type: Bug
Components: capacity scheduler
Reporter: András Győri
Assignee: András Győri
AbstractResourceUsage#getNodePartitionsSet and
QueueCapacities#getNodePartitionsSet provide keySet, a mutable view on the
HashMap's keys, that is subject to change. Iterating through an iterator that
is modified by an other thread at the same time results in a
ConcurrentModificationException as the following stacktrace shows:
{code:java}
2022-04-28 13:21:53,692 FATAL org.apache.hadoop.yarn.event.EventDispatcher:
Error in handling event type NODE_LABELS_UPDATE to the Event Dispatcher
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1445)
at java.util.HashMap$KeyIterator.next(HashMap.java:1469)
at com.google.common.collect.Sets$1$1.computeNext(Sets.java:758)
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:141)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:136)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CSQueueUtils.updateQueueStatistics(CSQueueUtils.java:236)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue.updateClusterResource(ParentQueue.java:1281)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.updateNodeLabelsAndQueueResource(CapacityScheduler.java:2115)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:1900)
at
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:169)
at
org.apache.hadoop.yarn.event.EventDispatcher$EventProcessor.run(EventDispatcher.java:66)
at java.lang.Thread.run(Thread.java:748)
{code}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
