Github user pwendell commented on the pull request:
https://github.com/apache/spark/pull/1486#issuecomment-56484097
@mridulm the proposal here was to avoid proposing a generalized/public API
for these and instead do something simple/internal for the case of Hadoop RDD.
The underscore is not a valid character in a hostname, so we can use it safely
and continue to support it going forward at low cost. This just piggy-backs on
the existing support we already have for in-memory input blocks.
I'd like to see ups adding a publicly documented complete interface for
specifying task locality levels like you said and supporting them in a general
way in the TaskSetManager. URI's could be good for this, or some other
structured format. But that is a much more complicated proposition, and one
that requires some design discussion. The purpose of this patch is to do
something more surgical in the short term.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
