[
https://issues.apache.org/jira/browse/YARN-1972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14129854#comment-14129854
]
Hadoop QA commented on YARN-1972:
---------------------------------
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12667974/YARN-1972.trunk.4.patch
against trunk revision 4be9517.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-common-project/hadoop-common
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
org.apache.hadoop.ha.TestZKFailoverControllerStress
org.apache.hadoop.yarn.server.nodemanager.TestLinuxContainerExecutorWithMocks
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/4886//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/4886//console
This message is automatically generated.
> Implement secure Windows Container Executor
> -------------------------------------------
>
> Key: YARN-1972
> URL: https://issues.apache.org/jira/browse/YARN-1972
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: nodemanager
> Reporter: Remus Rusanu
> Assignee: Remus Rusanu
> Labels: security, windows
> Attachments: YARN-1972.1.patch, YARN-1972.2.patch, YARN-1972.3.patch,
> YARN-1972.delta.4.patch, YARN-1972.trunk.4.patch
>
>
> h1. Windows Secure Container Executor (WCE)
> YARN-1063 adds the necessary infrasturcture to launch a process as a domain
> user as a solution for the problem of having a security boundary between
> processes executed in YARN containers and the Hadoop services. The WCE is a
> container executor that leverages the winutils capabilities introduced in
> YARN-1063 and launches containers as an OS process running as the job
> submitter user. A description of the S4U infrastructure used by YARN-1063
> alternatives considered can be read on that JIRA.
> The WCE is based on the DefaultContainerExecutor. It relies on the DCE to
> drive the flow of execution, but it overwrrides some emthods to the effect of:
> * change the DCE created user cache directories to be owned by the job user
> and by the nodemanager group.
> * changes the actual container run command to use the 'createAsUser' command
> of winutils task instead of 'create'
> * runs the localization as standalone process instead of an in-process Java
> method call. This in turn relies on the winutil createAsUser feature to run
> the localization as the job user.
>
> When compared to LinuxContainerExecutor (LCE), the WCE has some minor
> differences:
> * it does no delegate the creation of the user cache directories to the
> native implementation.
> * it does no require special handling to be able to delete user files
> The approach on the WCE came from a practical trial-and-error approach. I had
> to iron out some issues around the Windows script shell limitations (command
> line length) to get it to work, the biggest issue being the huge CLASSPATH
> that is commonplace in Hadoop environment container executions. The job
> container itself is already dealing with this via a so called 'classpath
> jar', see HADOOP-8899 and YARN-316 for details. For the WCE localizer launch
> as a separate container the same issue had to be resolved and I used the same
> 'classpath jar' approach.
> h2. Deployment Requirements
> To use the WCE one needs to set the
> `yarn.nodemanager.container-executor.class` to
> `org.apache.hadoop.yarn.server.nodemanager.WindowsSecureContainerExecutor`
> and set the `yarn.nodemanager.windows-secure-container-executor.group` to a
> Windows security group name that is the nodemanager service principal is a
> member of (equivalent of LCE
> `yarn.nodemanager.linux-container-executor.group`). Unlike the LCE the WCE
> does not require any configuration outside of the Hadoop own's yar-site.xml.
> For WCE to work the nodemanager must run as a service principal that is
> member of the local Administrators group or LocalSystem. this is derived from
> the need to invoke LoadUserProfile API which mention these requirements in
> the specifications. This is in addition to the SE_TCB privilege mentioned in
> YARN-1063, but this requirement will automatically imply that the SE_TCB
> privilege is held by the nodemanager. For the Linux speakers in the audience,
> the requirement is basically to run NM as root.
> h2. Dedicated high privilege Service
> Due to the high privilege required by the WCE we had discussed the need to
> isolate the high privilege operations into a separate process, an 'executor'
> service that is solely responsible to start the containers (incloding the
> localizer). The NM would have to authenticate, authorize and communicate with
> this service via an IPC mechanism and use this service to launch the
> containers. I still believe we'll end up deploying such a service, but the
> effort to onboard such a new platfrom specific new service on the project are
> not trivial.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
