Github user squito commented on the issue:
https://github.com/apache/spark/pull/17238
I don't think you're missing anything @tgravescs , it sounds like this is
just a misconfiguration and we shouldn't be doing anything special for it
(since it could hurt correct configurations). I wanted to see if this was a
particularly common / easy misconfiguration in yarn, but you both have
convinced me its not.
my point about SPARK-18886 was to think through how bad the effect from the
misconfiguration is. In the originally described scenario, if a few nodes are
accidentally labelled as belonging to their own rack, then sometimes task will
get preferentially assigned to this false-rack at first. If tasks are long,
its not that big a deal -- after a short delay, you'll then assign the tasks to
the rest of the cluster. But if tasks are short, b/c of SPARK-18886, you may
just keep assigning to the nodes in your false-rack, and leave the rest of the
cluster idle.
Nonetheless, I think its still just a misconfiguration. sounds like this
issue is a "won't fix", unless @morenn520 makes the case otherwise.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
