[
https://issues.apache.org/jira/browse/YARN-5673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Varun Vasudev updated YARN-5673:
--------------------------------
Attachment: container-executor Re-write Design Document.pdf
Attached a design document with some thoughts on a new container-executor.
[~chris.douglas], [~aw] - can you please review?
> [Umbrella] Re-write container-executor to improve security, extensibility,
> and portability
> ------------------------------------------------------------------------------------------
>
> Key: YARN-5673
> URL: https://issues.apache.org/jira/browse/YARN-5673
> Project: Hadoop YARN
> Issue Type: New Feature
> Components: nodemanager
> Reporter: Varun Vasudev
> Assignee: Varun Vasudev
> Attachments: container-executor Re-write Design Document.pdf
>
>
> As YARN adds support for new features that require administrator
> privileges(such as support for network throttling and docker), we’ve had to
> add new capabilities to the container-executor. This has led to a recognition
> that the current container-executor security features as well as the code
> could be improved. The current code is fragile and it’s hard to add new
> features without causing regressions. Some of the improvements that need to
> be made are -
> *Security*
> Currently the container-executor has limited security features. It relies
> primarily on the permissions set on the binary but does little additional
> security beyond that. There are few outstanding issues today -
> - No audit log
> - No way to disable features - network throttling and docker support are
> built in and there’s no way to turn them off at a container-executor level
> - Code can be improved - a lot of the code switches users back and forth in
> an arbitrary manner
> - No input validation - the paths, and files provided at invocation are not
> validated or required to be in some specific location
> - No signing functionality - there is no way to enforce that the binary was
> invoked by the NM and not by any other process
> *Code Issues*
> The code layout and implementation themselves can be improved. Some issues
> there are -
> - No support for log levels - everything is logged and this can’t be turned
> on or off
> - Extremely long set of invocation parameters(specifically during container
> launch) which makes turning features on or off complicated
> - Poor test coverage - it’s easy to introduce regressions today due to the
> lack of a proper test setup
> - Duplicate functionality - there is some amount of code duplication
> - Hard to make improvements or add new features due to the issues raised above
> *Portability*
> - The container-executor mixes platform dependent APIs with platform
> independent APIs making it hard to run it on multiple platforms. Allowing it
> to run on multiple platforms also improves the overall code structure .
> One option is to improve the existing container-executor, however it might be
> easier to start from scratch. That allows existing functionality to be
> supported until we are ready to switch to the new code.
> This umbrella JIRA is to capture all the work required for the new code. I'm
> going to work on a design doc for the changes - any suggestions or
> improvements are welcome.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
