Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/17723
There are multiple aspects of concern here :
1. Dependency on yarn api from core.
2. Exposure of yarn api in core interfaces.
3. Introduction of yarn specific implementation details in core.
4. Backward compatibility for existing exposed interfaces.
Based on your comments @vanzin, for (1) core is transitively already
depending on yarn - now we will be explicitly depending on it.
(2) is a blocker - we cannot have yarn specific api being exposed in core
interfaces.
(3) is an issue for future evolution of the project - we could either fix
it now, or incur tech debt and revisit it in future if it becomes a problem.
We also have to ensure that changes made to accommodate above should not
cause, as much as possible, changes to existing implementations (4) - which are
based on yarn resource manager api.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
