Varun Vasudev created YARN-5673:
-----------------------------------
Summary: [Umbrella] Re-write container-executor to improve
security, extensibility, and portability
Key: YARN-5673
URL: https://issues.apache.org/jira/browse/YARN-5673
Project: Hadoop YARN
Issue Type: New Feature
Components: nodemanager
Reporter: Varun Vasudev
Assignee: Varun Vasudev
As YARN adds support for new features that require administrator
privileges(such as support for network throttling and docker), we’ve had to add
new capabilities to the container-executor. This has led to a recognition that
the current container-executor security features as well as the code could be
improved. The current code is fragile and it’s hard to add new features without
causing regressions. Some of the improvements that need to be made are -
*Security*
Currently the container-executor has limited security features. It relies
primarily on the permissions set on the binary but does little additional
security beyond that. There are few outstanding issues today -
- No audit log
- No way to disable features - network throttling and docker support are built
in and there’s no way to turn them off at a container-executor level
- Code can be improved - a lot of the code switches users back and forth in an
arbitrary manner
- No input validation - the paths, and files provided at invocation are not
validated or required to be in some specific location
- No signing functionality - there is no way to enforce that the binary was
invoked by the NM and not by any other process
*Code Issues*
The code layout and implementation themselves can be improved. Some issues
there are -
- No support for log levels - everything is logged and this can’t be turned on
or off
- Extremely long set of invocation parameters(specifically during container
launch) which makes turning features on or off complicated
- Poor test coverage - it’s easy to introduce regressions today due to the lack
of a proper test setup
- Duplicate functionality - there is some amount of code duplication
- Hard to make improvements or add new features due to the issues raised above
*Portability*
- The container-executor mixes platform dependent APIs with platform
independent APIs making it hard to run it on multiple platforms. Allowing it to
run on multiple platforms also improves the overall code structure .
One option is to improve the existing container-executor, however it might be
easier to start from scratch. That allows existing functionality to be
supported until we are ready to switch to the new code.
This umbrella JIRA is to capture all the work required for the new code. I'm
going to work on a design doc for the changes - any suggestions or improvements
are welcome.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
