[jira] [Commented] (HADOOP-15235) Authentication Tokens should use HMAC instead of MAC
[
https://issues.apache.org/jira/browse/HADOOP-15235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16385209#comment-16385209
]
Hudson commented on HADOOP-15235:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13767 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13767/])
HADOOP-15282. HADOOP-15235 broke TestHttpFSServerWebServer (aajisaka: rev
21176a8ba7aa8acc4b1fe04c013b8716c2b25aa7)
* (edit)
hadoop-hdfs-project/hadoop-hdfs-httpfs/src/test/java/org/apache/hadoop/fs/http/server/TestHttpFSServerWebServer.java
> Authentication Tokens should use HMAC instead of MAC
>
>
> Key: HADOOP-15235
> URL: https://issues.apache.org/jira/browse/HADOOP-15235
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
>Affects Versions: 2.10.0, 3.2.0
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Major
> Fix For: 2.10.0, 3.2.0
>
> Attachments: HADOOP-15235.001.patch, HADOOP-15235.002.patch
>
>
> We currently use {{MessageDigest}} to compute a "SHA" MAC for signing
> Authentication Tokens. Firstly, what "SHA" maps to is dependent on the JVM
> and Cryptography Provider. While they _should_ do something reasonable, it's
> probably a safer idea to pick a specific algorithm. It looks like the Oracle
> JVM picks SHA-1; though something like SHA-256 would be better.
> In any case, it would also be better to use an HMAC algorithm instead.
> Changing from SHA-1 to SHA-256 or MAC to HMAC won't generate equivalent
> signatures, so this would normally be an incompatible change because the
> server wouldn't accept previous tokens it issued with the older algorithm.
> However, Authentication Tokens are used as a cheaper shortcut for Kerberos,
> so it's expected for users to also have Kerberos credentials; in this case,
> the Authentication Token will be rejected, but it will silently retry using
> Kerberos, and get an updated token. So this should all be transparent to the
> user.
> And finally, the code where we verify a signature uses a non-constant-time
> comparison, which could be subject to timing attacks. I believe it would be
> quite difficult in this case to do so, but we're probably better off using a
> constant-time comparison.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15235) Authentication Tokens should use HMAC instead of MAC
[
https://issues.apache.org/jira/browse/HADOOP-15235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16384124#comment-16384124
]
Robert Kanter commented on HADOOP-15235:
You're right! I've filed HADOOP-15282 to fix it.
> Authentication Tokens should use HMAC instead of MAC
>
>
> Key: HADOOP-15235
> URL: https://issues.apache.org/jira/browse/HADOOP-15235
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
>Affects Versions: 2.10.0, 3.2.0
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Major
> Fix For: 2.10.0, 3.2.0
>
> Attachments: HADOOP-15235.001.patch, HADOOP-15235.002.patch
>
>
> We currently use {{MessageDigest}} to compute a "SHA" MAC for signing
> Authentication Tokens. Firstly, what "SHA" maps to is dependent on the JVM
> and Cryptography Provider. While they _should_ do something reasonable, it's
> probably a safer idea to pick a specific algorithm. It looks like the Oracle
> JVM picks SHA-1; though something like SHA-256 would be better.
> In any case, it would also be better to use an HMAC algorithm instead.
> Changing from SHA-1 to SHA-256 or MAC to HMAC won't generate equivalent
> signatures, so this would normally be an incompatible change because the
> server wouldn't accept previous tokens it issued with the older algorithm.
> However, Authentication Tokens are used as a cheaper shortcut for Kerberos,
> so it's expected for users to also have Kerberos credentials; in this case,
> the Authentication Token will be rejected, but it will silently retry using
> Kerberos, and get an updated token. So this should all be transparent to the
> user.
> And finally, the code where we verify a signature uses a non-constant-time
> comparison, which could be subject to timing attacks. I believe it would be
> quite difficult in this case to do so, but we're probably better off using a
> constant-time comparison.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15235) Authentication Tokens should use HMAC instead of MAC
[
https://issues.apache.org/jira/browse/HADOOP-15235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16375379#comment-16375379
]
Xiao Chen commented on HADOOP-15235:
Hi [~rkanter],
Anecdotally I'm reviewing an hdfs jira, and it seems this one broke
{{TestHttpFSServerWebServer#testStartStop}}:
{noformat}
2018-02-23 23:13:29,791 WARN ServletHandler - /webhdfs/v1/
java.lang.IllegalArgumentException: Empty key
at javax.crypto.spec.SecretKeySpec.(SecretKeySpec.java:96)
at
org.apache.hadoop.security.authentication.util.Signer.computeSignature(Signer.java:93)
at
org.apache.hadoop.security.authentication.util.Signer.sign(Signer.java:59)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:587)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1751)
at
org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1617)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at
org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at
org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at
org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at
org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at
org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at
org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at
org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:534)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
at
org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at
org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at
org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at
org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at
org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at
org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:745)
java.lang.AssertionError:
Expected :500
Actual :200
{noformat}
Mind take a look?
> Authentication Tokens should use HMAC instead of MAC
>
>
> Key: HADOOP-15235
> URL: https://issues.apache.org/jira/browse/HADOOP-15235
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
>Affects Versions: 2.10.0, 3.2.0
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Major
> Fix For: 2.10.0, 3.2.0
>
> Attachments: HADOOP-15235.001.patch, HADOOP-15235.002.patch
>
>
> We currently use {{MessageDigest}} to compute a "SHA" MAC for signing
> Authentication Tokens. Firstly, what "SHA" maps to is dependent on the JVM
> and Cryptography Provider. While they _should_ do something reasonable, it's
> probably a safer idea to pick a specific algorithm. It looks like the Oracle
> JVM picks SHA-1; though something like SHA-256 would be better.
> In any case, it would also be better to use an HMAC algorithm instead.
> Changing from SHA-1 to SHA-256 or MAC to HMAC won't generate equivalent
> signatures, so this would normally be an incompatible change because the
> server wouldn't accept previous tokens it issued with the older algorithm.
> However, Authentication Tokens are used as a cheaper shortcut for Kerberos,
> so it's expected for users to also have Kerberos credentials; in this case,
> the Authentication Token will be rejected, but it
[jira] [Commented] (HADOOP-15235) Authentication Tokens should use HMAC instead of MAC
[
https://issues.apache.org/jira/browse/HADOOP-15235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16370833#comment-16370833
]
Hudson commented on HADOOP-15235:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13690 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13690/])
HADOOP-15235. Authentication Tokens should use HMAC instead of MAC (rkanter:
rev 324e5a7cf2bdb6f93e7c6fd9023817528f243dcf)
* (edit)
hadoop-common-project/hadoop-auth/src/main/java/org/apache/hadoop/security/authentication/util/Signer.java
> Authentication Tokens should use HMAC instead of MAC
>
>
> Key: HADOOP-15235
> URL: https://issues.apache.org/jira/browse/HADOOP-15235
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
>Affects Versions: 2.10.0, 3.2.0
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Major
> Fix For: 2.10.0, 3.2.0
>
> Attachments: HADOOP-15235.001.patch, HADOOP-15235.002.patch
>
>
> We currently use {{MessageDigest}} to compute a "SHA" MAC for signing
> Authentication Tokens. Firstly, what "SHA" maps to is dependent on the JVM
> and Cryptography Provider. While they _should_ do something reasonable, it's
> probably a safer idea to pick a specific algorithm. It looks like the Oracle
> JVM picks SHA-1; though something like SHA-256 would be better.
> In any case, it would also be better to use an HMAC algorithm instead.
> Changing from SHA-1 to SHA-256 or MAC to HMAC won't generate equivalent
> signatures, so this would normally be an incompatible change because the
> server wouldn't accept previous tokens it issued with the older algorithm.
> However, Authentication Tokens are used as a cheaper shortcut for Kerberos,
> so it's expected for users to also have Kerberos credentials; in this case,
> the Authentication Token will be rejected, but it will silently retry using
> Kerberos, and get an updated token. So this should all be transparent to the
> user.
> And finally, the code where we verify a signature uses a non-constant-time
> comparison, which could be subject to timing attacks. I believe it would be
> quite difficult in this case to do so, but we're probably better off using a
> constant-time comparison.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15235) Authentication Tokens should use HMAC instead of MAC
[
https://issues.apache.org/jira/browse/HADOOP-15235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16369702#comment-16369702
]
Xiao Chen commented on HADOOP-15235:
+1, thanks Robert!
> Authentication Tokens should use HMAC instead of MAC
>
>
> Key: HADOOP-15235
> URL: https://issues.apache.org/jira/browse/HADOOP-15235
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
>Affects Versions: 2.10.0, 3.2.0
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Major
> Attachments: HADOOP-15235.001.patch, HADOOP-15235.002.patch
>
>
> We currently use {{MessageDigest}} to compute a "SHA" MAC for signing
> Authentication Tokens. Firstly, what "SHA" maps to is dependent on the JVM
> and Cryptography Provider. While they _should_ do something reasonable, it's
> probably a safer idea to pick a specific algorithm. It looks like the Oracle
> JVM picks SHA-1; though something like SHA-256 would be better.
> In any case, it would also be better to use an HMAC algorithm instead.
> Changing from SHA-1 to SHA-256 or MAC to HMAC won't generate equivalent
> signatures, so this would normally be an incompatible change because the
> server wouldn't accept previous tokens it issued with the older algorithm.
> However, Authentication Tokens are used as a cheaper shortcut for Kerberos,
> so it's expected for users to also have Kerberos credentials; in this case,
> the Authentication Token will be rejected, but it will silently retry using
> Kerberos, and get an updated token. So this should all be transparent to the
> user.
> And finally, the code where we verify a signature uses a non-constant-time
> comparison, which could be subject to timing attacks. I believe it would be
> quite difficult in this case to do so, but we're probably better off using a
> constant-time comparison.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15235) Authentication Tokens should use HMAC instead of MAC
[
https://issues.apache.org/jira/browse/HADOOP-15235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364995#comment-16364995
]
Robert Kanter commented on HADOOP-15235:
There are already existing unit tests that cover this.
> Authentication Tokens should use HMAC instead of MAC
>
>
> Key: HADOOP-15235
> URL: https://issues.apache.org/jira/browse/HADOOP-15235
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
>Affects Versions: 2.10.0, 3.2.0
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Major
> Attachments: HADOOP-15235.001.patch, HADOOP-15235.002.patch
>
>
> We currently use {{MessageDigest}} to compute a "SHA" MAC for signing
> Authentication Tokens. Firstly, what "SHA" maps to is dependent on the JVM
> and Cryptography Provider. While they _should_ do something reasonable, it's
> probably a safer idea to pick a specific algorithm. It looks like the Oracle
> JVM picks SHA-1; though something like SHA-256 would be better.
> In any case, it would also be better to use an HMAC algorithm instead.
> Changing from SHA-1 to SHA-256 or MAC to HMAC won't generate equivalent
> signatures, so this would normally be an incompatible change because the
> server wouldn't accept previous tokens it issued with the older algorithm.
> However, Authentication Tokens are used as a cheaper shortcut for Kerberos,
> so it's expected for users to also have Kerberos credentials; in this case,
> the Authentication Token will be rejected, but it will silently retry using
> Kerberos, and get an updated token. So this should all be transparent to the
> user.
> And finally, the code where we verify a signature uses a non-constant-time
> comparison, which could be subject to timing attacks. I believe it would be
> quite difficult in this case to do so, but we're probably better off using a
> constant-time comparison.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15235) Authentication Tokens should use HMAC instead of MAC
[
https://issues.apache.org/jira/browse/HADOOP-15235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364994#comment-16364994
]
genericqa commented on HADOOP-15235:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 37s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 11m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 35s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
46s{color} | {color:green} hadoop-auth in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
35s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 68m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 |
| JIRA Issue | HADOOP-15235 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12910649/HADOOP-15235.002.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 6976e7bcd1a9 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 8f66aff |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14127/testReport/ |
| Max. process+thread count | 397 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-auth U:
hadoop-common-project/hadoop-auth |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/14127/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> Authentication Tokens should use HMAC
[jira] [Commented] (HADOOP-15235) Authentication Tokens should use HMAC instead of MAC
[
https://issues.apache.org/jira/browse/HADOOP-15235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364906#comment-16364906
]
Robert Kanter commented on HADOOP-15235:
The 002 patch replaces the call to {{Base64(0).encodeAsString(sig);}} with
{{Base64(0).encodeToString(sig);}}, which works in branch-2 and branch-3.
> Authentication Tokens should use HMAC instead of MAC
>
>
> Key: HADOOP-15235
> URL: https://issues.apache.org/jira/browse/HADOOP-15235
> Project: Hadoop Common
> Issue Type: Improvement
> Components: security
>Affects Versions: 2.10.0, 3.2.0
>Reporter: Robert Kanter
>Assignee: Robert Kanter
>Priority: Major
> Attachments: HADOOP-15235.001.patch, HADOOP-15235.002.patch
>
>
> We currently use {{MessageDigest}} to compute a "SHA" MAC for signing
> Authentication Tokens. Firstly, what "SHA" maps to is dependent on the JVM
> and Cryptography Provider. While they _should_ do something reasonable, it's
> probably a safer idea to pick a specific algorithm. It looks like the Oracle
> JVM picks SHA-1; though something like SHA-256 would be better.
> In any case, it would also be better to use an HMAC algorithm instead.
> Changing from SHA-1 to SHA-256 or MAC to HMAC won't generate equivalent
> signatures, so this would normally be an incompatible change because the
> server wouldn't accept previous tokens it issued with the older algorithm.
> However, Authentication Tokens are used as a cheaper shortcut for Kerberos,
> so it's expected for users to also have Kerberos credentials; in this case,
> the Authentication Token will be rejected, but it will silently retry using
> Kerberos, and get an updated token. So this should all be transparent to the
> user.
> And finally, the code where we verify a signature uses a non-constant-time
> comparison, which could be subject to timing attacks. I believe it would be
> quite difficult in this case to do so, but we're probably better off using a
> constant-time comparison.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
