[
https://issues.apache.org/jira/browse/YARN-7654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16445990#comment-16445990
]
Eric Yang edited comment on YARN-7654 at 4/20/18 8:13 PM:
----------------------------------------------------------
[~jlowe] {quote}This is not a pro, it's a con. Users will pass secrets to the
container via environment variables, and anyone on the system could casually
observe them via the ps command.{quote}
This is not true, you can combine YARN-8097 to hide secrets, and they will not
be revealed via ps command.
{quote}This is not specific to proposal 1. System administrator variables can
be (and already are) overridden by the nodemanager before the environment
variables are passed down to the container executor.{quote}
Again, you misunderstand my definition of system administrator in this case. I
am not referring to node manager system environment variables. They are
omitted by ENTRY_POINT support. The user who launch privileged container to
run a system wide Hive service. This user is a system administrator, and he
should have the capability to setup in site specific environment variables
files. Software vendor can provide a set of yarnfile template. This allows
separation to determine which variable is provided by user who ran the job, or
software vendor.
{quote}There are no filename clashes. The env file is being written to the
container private directory, the same place tokens are written by the
nodemanager today. This is not the container working directory where user files
are located, so there's no possibility of a collision.{quote}
The problem associated with having the environment variable file in node
manager private directory, will require docker to run as root. In your reply,
you did not reply if you will accept all consequence to hard code
container-executor to always launch docker as root user. If this question
remains unanswered, then filename collision will happen in localizer directory.
There is a settle difference between container-executor having access to node
manager private directory, vs docker have access to node manager private
directory. This issue must be resolved first for no filename collision
assumption to be valid.
{quote}That means we need to go the env-file route. If we have to go the
env-file route for security reasons then I don't understand why we would need
yet another way to set environment variables.{quote}
The world is usually more complex, and there are people delivering the software
to customer, and the administrator that maintains the system. This is similar
to Hadoop have core-default.xml, why do we have core-site.xml. There are good
reasons to have those separations. I am only looking for permission to hard
code docker to always be started as root. In the current feature set, this is
not true because we want to support
yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user. Proposal
2 will prevent future support of
yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user for docker.
This requires community's consensus to drop support for launch docker other
than root user. [~jlowe] I will accept your view point as drop support for
yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user to work
with docker. However, I know that majority of the enterprises do not like
having application to run as root. This is the reason that I like everyone to
form a opinion on this issue, and we decide as a community if we are going to
make that hard code. Please be patient and let involved people to provide
their feedback. Thank you for your patience.
was (Author: eyang):
[~jlowe] {quote}This is not a pro, it's a con. Users will pass secrets to the
container via environment variables, and anyone on the system could casually
observe them via the ps command.{quote}
This is not true, you can combine YARN-8079 to hide secrets, and they will not
be revealed via ps command.
{quote}This is not specific to proposal 1. System administrator variables can
be (and already are) overridden by the nodemanager before the environment
variables are passed down to the container executor.{quote}
Again, you misunderstand my definition of system administrator in this case. I
am not referring to node manager system environment variables. They are
omitted by ENTRY_POINT support. The user who launch privileged container to
run a system wide Hive service. This user is a system administrator, and he
should have the capability to setup in site specific environment variables
files. Software vendor can provide a set of yarnfile template. This allows
separation to determine which variable is provided by user who ran the job, or
software vendor.
{quote}There are no filename clashes. The env file is being written to the
container private directory, the same place tokens are written by the
nodemanager today. This is not the container working directory where user files
are located, so there's no possibility of a collision.{quote}
The problem associated with having the environment variable file in node
manager private directory, will require docker to run as root. In your reply,
you did not reply if you will accept all consequence to hard code
container-executor to always launch docker as root user. If this question
remains unanswered, then filename collision will happen in localizer directory.
There is a settle difference between container-executor having access to node
manager private directory, vs docker have access to node manager private
directory. This issue must be resolved first for no filename collision
assumption to be valid.
{quote}That means we need to go the env-file route. If we have to go the
env-file route for security reasons then I don't understand why we would need
yet another way to set environment variables.{quote}
The world is usually more complex, and there are people delivering the software
to customer, and the administrator that maintains the system. This is similar
to Hadoop have core-default.xml, why do we have core-site.xml. There are good
reasons to have those separations. I am only looking for permission to hard
code docker to always be started as root. In the current feature set, this is
not true because we want to support
yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user. Proposal
2 will prevent future support of
yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user for docker.
This requires community's consensus to drop support for launch docker other
than root user. [~jlowe] I will accept your view point as drop support for
yarn.nodemanager.linux-container-executor.nonsecure-mode.local-user to work
with docker. However, I know that majority of the enterprises do not like
having application to run as root. This is the reason that I like everyone to
form a opinion on this issue, and we decide as a community if we are going to
make that hard code. Please be patient and let involved people to provide
their feedback. Thank you for your patience.
> Support ENTRY_POINT for docker container
> ----------------------------------------
>
> Key: YARN-7654
> URL: https://issues.apache.org/jira/browse/YARN-7654
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
> Affects Versions: 3.1.0
> Reporter: Eric Yang
> Assignee: Eric Yang
> Priority: Blocker
> Attachments: YARN-7654.001.patch, YARN-7654.002.patch,
> YARN-7654.003.patch, YARN-7654.004.patch, YARN-7654.005.patch,
> YARN-7654.006.patch, YARN-7654.007.patch, YARN-7654.008.patch,
> YARN-7654.009.patch, YARN-7654.010.patch, YARN-7654.011.patch,
> YARN-7654.012.patch, YARN-7654.013.patch, YARN-7654.014.patch,
> YARN-7654.015.patch
>
>
> Docker image may have ENTRY_POINT predefined, but this is not supported in
> the current implementation. It would be nice if we can detect existence of
> {{launch_command}} and base on this variable launch docker container in
> different ways:
> h3. Launch command exists
> {code}
> docker run [image]:[version]
> docker exec [container_id] [launch_command]
> {code}
> h3. Use ENTRY_POINT
> {code}
> docker run [image]:[version]
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
