[jira] [Commented] (HADOOP-15162) UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
[
https://issues.apache.org/jira/browse/HADOOP-15162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318749#comment-16318749
]
Eric Yang commented on HADOOP-15162:
[~daryn] {quote}
Are you writing your own custom http server and authentication filter?
{quote}
No. This JIRA serves the purpose to provide information for less experienced
developer to understand proxy ACL must be verified to enable perimeter
security. Code written as:
{code}
proxyUser = UserGroupInformation.getLoginUser();
ugi = UserGroupInformation
.createProxyUser(remoteUser, proxyUser);
{code}
Without using UGI.createRemoteUser(remoteUser) is equally good. There is no
need of isSecurityEnabled() check, and there is no need of explicitly call
UGI.createRemoteUser(remoteUser). User only get to shoot themselves in the
foot, if {{hadoop.http.authentication.simple.anonymous.allowed}} is
misconfigured which allow anyone to impersonate as someone else. I would
propose to deprecate createRemoteUser(remoteUser) API because it creates
confusion on how code should be written.
> UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
> --
>
> Key: HADOOP-15162
> URL: https://issues.apache.org/jira/browse/HADOOP-15162
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Eric Yang
>
> {{UserGroupInformation.createRemoteUser(String user)}} is hard coded
> Authentication method to SIMPLE by HADOOP-10683. This by passed proxyuser
> ACL check, isSecurityEnabled check, and allow caller to impersonate as
> anyone. This method could be abused in the main code base, which can cause
> part of Hadoop to become insecure without proxyuser check for both SIMPLE or
> Kerberos enabled environment.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15162) UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
[
https://issues.apache.org/jira/browse/HADOOP-15162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16318599#comment-16318599
]
Daryn Sharp commented on HADOOP-15162:
--
bq. Proxy user credential should be verified if it can impersonate.
_There are no credentials_ with security disabled but a proxy user is verified
if the client reported it's a proxy user – for http rest services via the doAs
parameter.
bq. In my usage, I am writing a component for YARN, and end user credential is
verified in http request.
It is verified and you have nothing to do if you use the standard HttpServer
and authentication filters.
bq. If code is written as UGI.createRemoteUser(remoteUser), should there be a
check to determine if the current service user can proxy? Some Hadoop PMC told
me no because they assumed isSecurityEnabled == false, there should be no proxy
ACL check.
Of course it should be verified and as I keep stressing it is verified. I
think the PMC gave you bad advice and/or didn't understand the context.
bq. If this type of assumption is applied, then we will have components talking
to other components without honoring proxy user ACL, and leading to part of
Hadoop being completely insecure.
This boggles me. You are arguing: "oh no! my insecure server is completely
insecure!"
bq. The server should decide which authentication method to use, setup
authentication method and verify proxy ACL explicitly.
It already does. What am I missing? Are you writing your own custom http
server and authentication filter?
Let's conclude this discussion. Specifically, what existing code are you
proposing be changed and how? Post a patch.
> UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
> --
>
> Key: HADOOP-15162
> URL: https://issues.apache.org/jira/browse/HADOOP-15162
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Eric Yang
>
> {{UserGroupInformation.createRemoteUser(String user)}} is hard coded
> Authentication method to SIMPLE by HADOOP-10683. This by passed proxyuser
> ACL check, isSecurityEnabled check, and allow caller to impersonate as
> anyone. This method could be abused in the main code base, which can cause
> part of Hadoop to become insecure without proxyuser check for both SIMPLE or
> Kerberos enabled environment.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15162) UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
[
https://issues.apache.org/jira/browse/HADOOP-15162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317583#comment-16317583
]
Eric Yang commented on HADOOP-15162:
[~daryn] Thank you for your reply.
{quote}
Based on the snippets of code that conclude with "if authentication are in
place, server side code can be simplified to [...]
UserGroupInformation.createRemoteUser(remoteUser);", I think you are suggesting
that createRemote should auto-magically create a proxy user with the login
user? If you say yes, I'll provide a litany of reasons why that'd be completely
broken. If no, please more concisely state your use case.{quote}
Proxy user credential should be verified if it can impersonate. In my usage, I
am writing a component for YARN, and end user credential is verified in http
request. If code is written as UGI.createRemoteUser(remoteUser), should there
be a check to determine if the current service user can proxy? Some Hadoop PMC
told me no because they assumed isSecurityEnabled == false, there should be no
proxy ACL check. If this type of assumption is applied, then we will have
components talking to other components without honoring proxy user ACL, and
leading to part of Hadoop being completely insecure. This is the reason that I
think createRemoteUser default authentication method to SIMPLE is a bad
practice. The server should decide which authentication method to use, setup
authentication method and verify proxy ACL explicitly.
> UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
> --
>
> Key: HADOOP-15162
> URL: https://issues.apache.org/jira/browse/HADOOP-15162
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Eric Yang
>
> {{UserGroupInformation.createRemoteUser(String user)}} is hard coded
> Authentication method to SIMPLE by HADOOP-10683. This by passed proxyuser
> ACL check, isSecurityEnabled check, and allow caller to impersonate as
> anyone. This method could be abused in the main code base, which can cause
> part of Hadoop to become insecure without proxyuser check for both SIMPLE or
> Kerberos enabled environment.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15162) UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
[
https://issues.apache.org/jira/browse/HADOOP-15162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317414#comment-16317414
]
Daryn Sharp commented on HADOOP-15162:
--
bq. In summary, proxy user ACL should be checked for simple security instead of
reliance on isSecurityEnabled().
As stated earlier, proxy privs are always checked for non-token connections.
bq. isSecurityEnabled( gives a false sense that proxy user ACL shouldn't be
checked which leading to use of
UserGroupInformation.createRemoteUser(remoteUser) in server code, which is a
bad practice for not verifying the credential of current server user.
It's not bad practice for a server to use createRemoteUser – that's why it
exists. What does "verifying the credential of current server user" mean when
security is disabled and there are no credentials?
> UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
> --
>
> Key: HADOOP-15162
> URL: https://issues.apache.org/jira/browse/HADOOP-15162
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Eric Yang
>
> {{UserGroupInformation.createRemoteUser(String user)}} is hard coded
> Authentication method to SIMPLE by HADOOP-10683. This by passed proxyuser
> ACL check, isSecurityEnabled check, and allow caller to impersonate as
> anyone. This method could be abused in the main code base, which can cause
> part of Hadoop to become insecure without proxyuser check for both SIMPLE or
> Kerberos enabled environment.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15162) UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
[
https://issues.apache.org/jira/browse/HADOOP-15162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317395#comment-16317395
]
Daryn Sharp commented on HADOOP-15162:
--
bq. This has been known issues with SIMPLE security since Hadoop 0.20s release.
The words simple and security don't belong in the same sentence, but let's
discuss anyway.
bq. createRemoteUser should be intelligent to know the preference of the
authentication to apply to avoid security holes
No. The createRemoteUser method works as designed. App-specific control logic
does not belong in the ugi. A new ugi has only a presumed identity. It's up
to the app code to either fill it with credentials or attach a real user (via
createProxyUser) that has credentials and proxy privs.
Based on the snippets of code that conclude with "if authentication are in
place, server side code can be simplified to \[...\]
UserGroupInformation.createRemoteUser(remoteUser);", _I think_ you are
suggesting that createRemote should auto-magically create a proxy user with the
login user? If you say yes, I'll provide a litany of reasons why that'd be
completely broken. If no, please more concisely state your use case.
bq. Client code can assign any arbitrary user, and trigger authentication
challenge to occur when communicate with the server side. This is happening
when Kerberos security is enabled. It would be nice if the same practice can
apply to SIMPLE security without open up security holes regardless Kerberos
security is enabled or not.
If security is disabled, how can the phrases "trigger authentication" and
"without open security holes" have any meaning?
bq. This left SIMPLE security to be completely open, no security and no proxy
check.
Based on the code snippets, I think you are alarmed that an insecure client can
decide to not create a proxy user, thus bypassing an insecure server's proxy
checks for originating user/host. While one of my long held pet peeves is
client code should never be conditionalized for "security enabled", it's
impossible for an insecure server to know whether an insecure client is
honestly reporting whether it's a proxy user.
> UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
> --
>
> Key: HADOOP-15162
> URL: https://issues.apache.org/jira/browse/HADOOP-15162
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Eric Yang
>
> {{UserGroupInformation.createRemoteUser(String user)}} is hard coded
> Authentication method to SIMPLE by HADOOP-10683. This by passed proxyuser
> ACL check, isSecurityEnabled check, and allow caller to impersonate as
> anyone. This method could be abused in the main code base, which can cause
> part of Hadoop to become insecure without proxyuser check for both SIMPLE or
> Kerberos enabled environment.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15162) UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
[
https://issues.apache.org/jira/browse/HADOOP-15162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16317386#comment-16317386
]
Eric Yang commented on HADOOP-15162:
In summary, proxy user ACL should be checked for simple security instead of
reliance on isSecurityEnabled(). {{isSecurityEnabled()}} gives a false sense
that proxy user ACL shouldn't be checked which leading to use of
UserGroupInformation.createRemoteUser(remoteUser) in server code, which is a
bad practice for not verifying the credential of current server user. Is this
something that need to be improved or we mark this as won't fix and make sure
people always use proper proxy user directive for server side code?
{code}
proxyUser = UserGroupInformation.getLoginUser();
ugi = UserGroupInformation
.createProxyUser(remoteUser, proxyUser);
{code}
> UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
> --
>
> Key: HADOOP-15162
> URL: https://issues.apache.org/jira/browse/HADOOP-15162
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Eric Yang
>
> {{UserGroupInformation.createRemoteUser(String user)}} is hard coded
> Authentication method to SIMPLE by HADOOP-10683. This by passed proxyuser
> ACL check, isSecurityEnabled check, and allow caller to impersonate as
> anyone. This method could be abused in the main code base, which can cause
> part of Hadoop to become insecure without proxyuser check for both SIMPLE or
> Kerberos enabled environment.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15162) UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
[
https://issues.apache.org/jira/browse/HADOOP-15162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316758#comment-16316758
]
Eric Yang commented on HADOOP-15162:
Hi [~daryn],
{quote}
If you have a specific risk case, please take it up on the security list. Don't
irresponsibly post publicly.
{quote}
There is no security hole yet if the cluster is deployed with Kerberos or
isSecurityEnabled==true. If I am disclosing a real security hole, then it will
definitely have been sent to security mailing list first. I do not think this
issue is worthy of sounding the bell yet. This has been known issues with
SIMPLE security since Hadoop 0.20s release. I am only observing code changes
over the past couple years and some security holes are about to be opened up
due to inexperience developers following incorrect discipline. Without the
proper information to educate the public, fear will only cause panic and
prevent progress. I hope you understand my intention is to mitigate the risk
by disclosing information to lead to progress rather than fear to drive people
away.
> UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
> --
>
> Key: HADOOP-15162
> URL: https://issues.apache.org/jira/browse/HADOOP-15162
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Eric Yang
>
> {{UserGroupInformation.createRemoteUser(String user)}} is hard coded
> Authentication method to SIMPLE by HADOOP-10683. This by passed proxyuser
> ACL check, isSecurityEnabled check, and allow caller to impersonate as
> anyone. This method could be abused in the main code base, which can cause
> part of Hadoop to become insecure without proxyuser check for both SIMPLE or
> Kerberos enabled environment.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15162) UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
[
https://issues.apache.org/jira/browse/HADOOP-15162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316725#comment-16316725
]
Eric Yang commented on HADOOP-15162:
Hi [~jlowe], Webhdfs and YARN allows impersonation through usage of
?user.name=foobar in HTTP URL. This allows SIMPLE security mode to run as any
other user without check. If the cluster is configured with Linux Container
Executor, then it can be carry out as a privileges escalation exploit in
combination with vulnerability found in YARN-7590.
Trust and verify are very important processes to enforce authentication
security. Server side must do either username/password challenge or token
validation to enforce security like you said. In Hadoop implementation of
SIMPLE security, there is no authentication challenge in form of
username/password prompt or token challenge to client. Therefore, Hadoop doAs
call trusts everyone who claims to be someone else when using SIMPLE security.
The fix must happen to intercepting RPC or HTTP requests to add authentication
challenge to enforce security. createRemoteUser should be intelligent to know
the preference of the authentication to apply to avoid security holes. Without
addressing these issues, we are encouraging developer to write code such as:
{code}
UserGroupInformation proxyUser;
UserGroupInformation ugi;
String remoteUser = request.getRemoteUser();
try {
if (UserGroupInformation.isSecurityEnabled()) {
proxyUser = UserGroupInformation.getLoginUser();
ugi = UserGroupInformation
.createProxyUser(remoteUser, proxyUser);
} else {
ugi = UserGroupInformation.createRemoteUser(remoteUser);
}
return ugi;
} catch (IOException e) {
throw new AccessControlException(e.getCause());
}
{code}
If security is not enabled, allow proxy without checking for the current user
is allowed to proxy. Unfortunately, SIMPLE security is tight to no security
due to improper interpretation in isSecurityEnabled method since Hadoop 0.20
security releases.
If authentication are in place, server side code can be simplified to:
{code}
UserGroupInformation proxyUser;
UserGroupInformation ugi;
String remoteUser = request.getRemoteUser();
try {
ugi = UserGroupInformation.createRemoteUser(remoteUser);
return ugi;
} catch (IOException e) {
throw new AccessControlException(e.getCause());
}
{code}
createRemoteUser(String user), can get the current login user, then
createProxyUser. At minimum proxy user ACL list will be verified for simple
security to set a security perimeter, and combined with authentication
challenge to provide simple security.
Client code can assign any arbitrary user, and trigger authentication challenge
to occur when communicate with the server side. This is happening when
Kerberos security is enabled. It would be nice if the same practice can apply
to SIMPLE security without open up security holes regardless Kerberos security
is enabled or not.
The very first design of Hadoop security was attempting to solve replay attack.
Kerberos security or a combination of proxy user/host ACL list can both serve
the same purpose. For some reason, during the implementation, Kerberos and
proxy ACL list became only enforced when Kerberos security is enabled.
Kerberos and proxy ACL are in fact redundant checks. This left SIMPLE security
to be completely open, no security and no proxy check.
If developers blindly utilized the current implementation of
UserGroupInformation.createRemoteUser(remoteUser);, part of Hadoop can be
opened up to run without any security without anyone knowing. This is not good
security practice, hence, we probably want to mitigate this risk by improving
the logic in createRemoteUser and review if there is a need to revise SIMPLE
security definition.
If isSecurityEnabled() is based on hadoop.security.authentication==null, then
the implementation of HTTP basic + proxy ACL and Kerberos could be two methods
that enforce security. This provide a way to apply security measure on cloud
without deploying Kerberos.
> UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
> --
>
> Key: HADOOP-15162
> URL: https://issues.apache.org/jira/browse/HADOOP-15162
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Eric Yang
>
> {{UserGroupInformation.createRemoteUser(String user)}} is hard coded
> Authentication method to SIMPLE by HADOOP-10683. This by passed proxyuser
> ACL check, isSecurityEnabled check, and allow caller to impersonate as
> anyone. This method could be abused in the main code base, which can cause
> part of Hadoop to become insecure without
[jira] [Commented] (HADOOP-15162) UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
[
https://issues.apache.org/jira/browse/HADOOP-15162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16316581#comment-16316581
]
Daryn Sharp commented on HADOOP-15162:
--
Unless I'm misunderstanding the description, this appears to be conjecture.
bq. This by passed proxyuser ACL check, isSecurityEnabled check, and allow
caller to impersonate as anyone.
No, isSecurityEnabled is dictated by the conf, not the auth method of a ugi
instance.
bq. \[...\] which can cause part of Hadoop to become insecure without proxyuser
check for both SIMPLE or Kerberos enabled environment.
Assuming it's a RPC or HttpServer, no, the proxyuser ACL is always applied when
the ugi is anything but token, ie. simple or kerberos. If it's token, a proxy
request is rejected (can't impersonate when already impersonating).
If you have a specific risk case, please take it up on the security list.
Don't irresponsibly post publicly.
> UserGroupInformation.createRemoteUser hardcode authentication method to SIMPLE
> --
>
> Key: HADOOP-15162
> URL: https://issues.apache.org/jira/browse/HADOOP-15162
> Project: Hadoop Common
> Issue Type: Bug
> Components: security
>Reporter: Eric Yang
>
> {{UserGroupInformation.createRemoteUser(String user)}} is hard coded
> Authentication method to SIMPLE by HADOOP-10683. This by passed proxyuser
> ACL check, isSecurityEnabled check, and allow caller to impersonate as
> anyone. This method could be abused in the main code base, which can cause
> part of Hadoop to become insecure without proxyuser check for both SIMPLE or
> Kerberos enabled environment.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
