[
https://issues.apache.org/jira/browse/MAPREDUCE-1695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12858969#action_12858969
]
Hong Tang commented on MAPREDUCE-1695:
--------------------------------------
Hemanth, thanks for reviewing the patch!
bq. The synchronization added around obtainNewTask seems unnecessary as all
code paths are locked on the scheduler.
Possibly true. My changes were only motivated by the findbugs report which
could produce false positives.
bq. getClusterCapacity seems unused. Should we remove it ?
It seems to be an interface method and could be usable to support adaptive
scheduling decisions based on load pressure in the future. But I am not
familiar with Capacity Scheduler enough, so either way is fine with me.
bq. There seem to be some constructor variables that are not needed. They are
named 'ignored'. I suppose we can remove them ?
bq. Similarly, I suppose we can also remove the MapTaskDataView and
ReduceTaskDataView constructors ?
Sure. Only reason I kept it there is in case these classes are constructed
through reflection and removing a parameter would lead to constructor-not-found
exception.
bq. This is not really a comment on this patch, but I am noting it here to get
some comments from other folks: There seems to be a bug in the user limit
handling code that is fixed by this patch. Before this patch,
TaskSchedulingContext.updateNoOfSlotsOccupiedByUser on a container queue was
only adding entries when the user's name was already added to its data
structure. However, since this map begins empty, it would never be updated with
a user's statistics from a child queue. While the bug is being fixed in this
patch, I think this is a serious enough bug to warrant a test case. If my
understanding is correct, we should file a JIRA to add a test case to verify
this.
Yes, I talked with Arun about this, and I was very puzzled by this code that
seems to be doing nothing useful. I will remove the fix in this patch (the else
statement) and regenerate the patch.
> capacity scheduler is not included in findbugs/javadoc targets
> --------------------------------------------------------------
>
> Key: MAPREDUCE-1695
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1695
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: contrib/capacity-sched
> Reporter: Hong Tang
> Assignee: Hong Tang
> Attachments: MAPREDUCE-1695-2.patch, MAPREDUCE-1695-3.patch,
> MAPREDUCE-1695.patch, mr1695-hadoop-findbugs-report-1.html,
> mr1695-hadoop-findbugs-report-2.html
>
>
> Capacity Scheduler is not included in findbugs/javadoc targets.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
