[
https://issues.apache.org/jira/browse/MAPREDUCE-7169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17006240#comment-17006240
]
Ahmed Hussein commented on MAPREDUCE-7169:
------------------------------------------
[~BilwaST], can you also add a configuration to enable/disable your code
changes?
My intuition is that changing the policy to pick the node for the speculative
task will inherently change the efficiency of the speculation.
For example, picking a different node may increase the startup time of the
speculative task. This implies change of the speculation efficiency compared to
the legacy behavior. Thus, I suggest to give the option for the user to
enable/disable the new policy in case she prefers to evaluate the new behavior
and revert back to the legacy one if necessary.
> Speculative attempts should not run on the same node
> ----------------------------------------------------
>
> Key: MAPREDUCE-7169
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7169
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: yarn
> Affects Versions: 2.7.2
> Reporter: Lee chen
> Assignee: Bilwa S T
> Priority: Major
> Attachments: MAPREDUCE-7169-001.patch, MAPREDUCE-7169-002.patch,
> MAPREDUCE-7169-003.patch, image-2018-12-03-09-54-07-859.png
>
>
> I found in all versions of yarn, Speculative Execution may set the
> speculative task to the node of original task.What i have read is only it
> will try to have one more task attempt. haven't seen any place mentioning not
> on same node.It is unreasonable.If the node have some problems lead to tasks
> execution will be very slow. and then placement the speculative task to same
> node cannot help the problematic task.
> In our cluster （version 2.7.2，2700 nodes），this phenomenon appear
> almost everyday.
> !image-2018-12-03-09-54-07-859.png!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
