Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/5400#issuecomment-118970970
so I haven't looked at all the network layers yet but this approach seems
reasonable to me. If we end up wanting to do more logic at higher layer this
doesn't prevent that. It would become an extra unneeded abstraction but at
that point you could remove it if it makes sense. Based on the previous pull
request doing it this way seems fairly non-intrusive.
@rxin I think you had commented on previous pr or jira about the 2 options.
Did you have specific concerns that haven't been addressed?
@squito Assuming you have patches for caching, network transfer, and
shuffle based on this have you done any performance testing?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
