Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/6652#issuecomment-109418206
> When there is no skew, are there situations where this would lead to
worse performance? E.g. will it make tasks bunch up on nodes more than before
and / or result in scheduling delays?
I forget how this corner of the scheduler works, but in this case I think
that we're just introducing preferences where there weren't any before, so as
long as our locality delay isn't too long then I don't know that this will lead
to a big queue for a particular node.
@shivaram, is there an easy way to feature-flag this? Might be nice for
being able to run benchmarks or to turn this off if it's bad for certain
workloads.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
