[
https://issues.apache.org/jira/browse/HADOOP-5214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12674179#action_12674179
]
rahul k singh commented on HADOOP-5214:
---------------------------------------
Had thought about following approaches in mind.
1. Generate the local copy of taskTrackers and go over them to find out total
slots.
2. Synchronizing the access points of taskTrackers().values esp making sure
iterators are in synchronizing block.
3. Using ClusterStatus to fetch the required information , ClusterStatus
already calculates the MaxMapTasks and MaxReduceTasks at cluster level (
cluster's maximum capacity for running map or reduce tasks).
took the 3rd approach.
changed the synchronzation of update() method to more granular level , this
allows us to directly get lock over JobTracker to fetch the clusterStatus,
instead of taking JobTracker lock and then scheduler lock.
Currently update() method is like below
{code}
protected void update() {
ClusterStatus clusterStatus = taskTrackerManager.getClusterStatus();
synchronized(this){
{code}
getTotalSlots now takes additional parameter clusterStatus.
PS : We have made getTotalSlots private , as we couldnt find any usage of this
method outside the class and reason why we would need to make this method
public.
updateFairShare() method also takes clusterStatus as parameter.
PS : Matei can you look into the patch.
> ConcurrentModificationException in FairScheduler.getTotalSlots
> --------------------------------------------------------------
>
> Key: HADOOP-5214
> URL: https://issues.apache.org/jira/browse/HADOOP-5214
> Project: Hadoop Core
> Issue Type: Bug
> Components: contrib/fair-share
> Reporter: Vinod K V
> Assignee: rahul k singh
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
