Oleksandr Shevchenko created YARN-9023:
------------------------------------------
Summary: Pluggable YARN Auxiliary Services
Key: YARN-9023
URL: https://issues.apache.org/jira/browse/YARN-9023
Project: Hadoop YARN
Issue Type: New Feature
Reporter: Oleksandr Shevchenko
Assignee: Oleksandr Shevchenko
Attachments: Pluggable YARN Auxiliary Services Doc v1.pdf
The propose is to create customizable auxiliary services infrastructure for all
YARN services to provide abilities to extends or change functionality for all
YARN demons.
Currently, YARN has the ability to add auxiliary services only for NodeManager.
There are some cases when we need to add auxiliary services for some other YARN
demons. For example, in YARN-2267 proposed to add Auxiliary Service support in
RM for implementation monitor plugins or some alarm services. We faced with one
more case, we need to create/setup some storage for YARN demon data (for
example for RM state store or AM intermediate data) before demon started. Also,
will be good if we can use some auxiliary service in different YARN demons.
The end users can use YARN Auxiliary Services (YAS) to customize or extend the
functionality of all YARN demons. We can provide an ability to create aux
service both for one and for all demons. This will make easy creating of aux
service for shuffle or just to up some service within another YARN daemon. We
will get a unified and more extensible architecture for all aux services which
may need to create in the future.
Please look at the attached design doc for the details. [^Pluggable YARN
Auxiliary Services Doc v1.pdf]
Also, you can add your comments to this doc:
https://docs.google.com/document/d/1Pxs8eeWZwnlwryPj7gyN8Ej1O-FRBSX_C0ispV9ORT8/edit?usp=sharing
Thanks a lot for any comments and attention to this feature!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
