[jira] [Commented] (HADOOP-15445) TestCryptoAdminCLI test failure when upgrading to JDK8 patch 171.
[
https://issues.apache.org/jira/browse/HADOOP-15445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16475037#comment-16475037
]
Wei-Chiu Chuang commented on HADOOP-15445:
--
Hi [~lmc...@apache.org], looks like [~gabor.bota]'s got a patch ready for
review for the same issue.
> TestCryptoAdminCLI test failure when upgrading to JDK8 patch 171.
> -
>
> Key: HADOOP-15445
> URL: https://issues.apache.org/jira/browse/HADOOP-15445
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ewan Higgs
>Priority: Major
>
> JDK8 patch 171 introduces a new feature:
> {quote}
> h3. New Features
> security-libs/javax.crypto*[!https://www.oracle.com/webfolder/s/dm/st/images/lp-external-link-arrow.png!|http://www.oracle.com/technetwork/java/javase/8u171-relnotes-430.html#JDK-8189997]
> Enhanced KeyStore Mechanisms*
> A new security property named {{jceks.key.serialFilter}} has been introduced.
> If this filter is configured, the JCEKS KeyStore uses it during the
> deserialization of the encrypted Key object stored inside a SecretKeyEntry.
> If it is not configured or if the filter result is UNDECIDED (for example,
> none of the patterns match), then the filter configured by
> {{jdk.serialFilter}} is consulted.
> If the system property {{jceks.key.serialFilter}} is also supplied, it
> supersedes the security property value defined here.
> The filter pattern uses the same format as {{jdk.serialFilter}}. The default
> pattern allows {{java.lang.Enum}}, {{java.security.KeyRep}},
> {{java.security.KeyRep$Type}}, and {{javax.crypto.spec.SecretKeySpec}} but
> rejects all the others.
> Customers storing a SecretKey that does not serialize to the above types must
> modify the filter to make the key extractable.
> {quote}
> We believe this causes some test failures:
>
> {quote}{{{color:#33}java.io.IOException: Can't recover key for myKey from
> keystore
> file:/{color}{color:#33}home/{color}{color:#33}jenkins/{color}{color:#33}workspace/{color}{color:#33}hadoopFullBuild/{color}{color:#33}hadoop-hdfs-project/{color}{color:#33}hadoop-hdfs/{color}{color:#33}target/{color}{color:#33}test/{color}{color:#33}data/{color}{color:#33}53406117-0132-401e-a67d-6672f1b6a14a/{color}{color:#33}test.jks
> at
> org.apache.hadoop.crypto.key.JavaKeyStoreProvider.getMetadata(JavaKeyStoreProvider.java:424)
> at
> org.apache.hadoop.crypto.key.KeyProviderExtension.getMetadata(KeyProviderExtension.java:100)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirEncryptionZoneOp.ensureKeyIsInitialized(FSDirEncryptionZoneOp.java:124)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.createEncryptionZone(FSNamesystem.java:7227)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.createEncryptionZone(NameNodeRpcServer.java:2082)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.createEncryptionZone(ClientNamenodeProtocolServerSideTranslatorPB.java:1524)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991) at
> org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869) at
> org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675) Caused by:
> java.security.UnrecoverableKeyException: Rejected by the
> jceks.key.serialFilter or jdk.serialFilter property at
> com.sun.crypto.provider.KeyProtector.unseal(KeyProtector.java:352) at
> com.sun.crypto.provider.JceKeyStore.engineGetKey(JceKeyStore.java:136) at
> java.security.KeyStore.getKey(KeyStore.java:1023){color}}}
> {quote}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-15445) TestCryptoAdminCLI test failure when upgrading to JDK8 patch 171.
[
https://issues.apache.org/jira/browse/HADOOP-15445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462166#comment-16462166
]
Steve Loughran commented on HADOOP-15445:
-
FYI, [~lmc...@apache.org]
This is due to
[CVE-2018-2794|http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-2794];
looks like the JVM was deserializing any class. So if you had, say, Groovy <
2.4 on the CP, you could serialize a script and then have it exec'd on the
deser. Get some service to read your jceks file and you can run code as that
entity.
This is only a risk for hadoop if there are cases where services are
deserializing JCECKs files from users with lower rights, services which
wouldn't otherwise be running arbitrary code from the user. Hive might be the
weakness, if its services are only executing locked down (UDF-less) SQL.
> TestCryptoAdminCLI test failure when upgrading to JDK8 patch 171.
> -
>
> Key: HADOOP-15445
> URL: https://issues.apache.org/jira/browse/HADOOP-15445
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ewan Higgs
>Priority: Major
>
> JDK8 patch 171 introduces a new feature:
> {quote}
> h3. New Features
> security-libs/javax.crypto*[!https://www.oracle.com/webfolder/s/dm/st/images/lp-external-link-arrow.png!|http://www.oracle.com/technetwork/java/javase/8u171-relnotes-430.html#JDK-8189997]
> Enhanced KeyStore Mechanisms*
> A new security property named {{jceks.key.serialFilter}} has been introduced.
> If this filter is configured, the JCEKS KeyStore uses it during the
> deserialization of the encrypted Key object stored inside a SecretKeyEntry.
> If it is not configured or if the filter result is UNDECIDED (for example,
> none of the patterns match), then the filter configured by
> {{jdk.serialFilter}} is consulted.
> If the system property {{jceks.key.serialFilter}} is also supplied, it
> supersedes the security property value defined here.
> The filter pattern uses the same format as {{jdk.serialFilter}}. The default
> pattern allows {{java.lang.Enum}}, {{java.security.KeyRep}},
> {{java.security.KeyRep$Type}}, and {{javax.crypto.spec.SecretKeySpec}} but
> rejects all the others.
> Customers storing a SecretKey that does not serialize to the above types must
> modify the filter to make the key extractable.
> {quote}
> We believe this causes some test failures:
>
> {quote}{{{color:#33}java.io.IOException: Can't recover key for myKey from
> keystore
> file:/{color}{color:#33}home/{color}{color:#33}jenkins/{color}{color:#33}workspace/{color}{color:#33}hadoopFullBuild/{color}{color:#33}hadoop-hdfs-project/{color}{color:#33}hadoop-hdfs/{color}{color:#33}target/{color}{color:#33}test/{color}{color:#33}data/{color}{color:#33}53406117-0132-401e-a67d-6672f1b6a14a/{color}{color:#33}test.jks
> at
> org.apache.hadoop.crypto.key.JavaKeyStoreProvider.getMetadata(JavaKeyStoreProvider.java:424)
> at
> org.apache.hadoop.crypto.key.KeyProviderExtension.getMetadata(KeyProviderExtension.java:100)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirEncryptionZoneOp.ensureKeyIsInitialized(FSDirEncryptionZoneOp.java:124)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.createEncryptionZone(FSNamesystem.java:7227)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.createEncryptionZone(NameNodeRpcServer.java:2082)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.createEncryptionZone(ClientNamenodeProtocolServerSideTranslatorPB.java:1524)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991) at
> org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869) at
> org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675) Caused by:
> java.security.UnrecoverableKeyException: Rejected by the
> jceks.key.serialFilter or jdk.serialFilter property at
> com.sun.crypto.provider.KeyProtector.unseal(KeyProtector.java:352) at
> com.sun.crypto.provider.JceKeyStore.engineGetKey(JceKeyStore.java:136) at
> java.security.KeyStore.getKey(KeyStore.java:1023){color}}}
> {quote}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail:
[jira] [Commented] (HADOOP-15445) TestCryptoAdminCLI test failure when upgrading to JDK8 patch 171.
[
https://issues.apache.org/jira/browse/HADOOP-15445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16462146#comment-16462146
]
Ewan Higgs commented on HADOOP-15445:
-
OpenJDK will get the same feature:
https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2018-2794
> TestCryptoAdminCLI test failure when upgrading to JDK8 patch 171.
> -
>
> Key: HADOOP-15445
> URL: https://issues.apache.org/jira/browse/HADOOP-15445
> Project: Hadoop Common
> Issue Type: Bug
>Reporter: Ewan Higgs
>Priority: Major
>
> JDK8 patch 171 introduces a new feature:
> {quote}
> h3. New Features
> security-libs/javax.crypto*[!https://www.oracle.com/webfolder/s/dm/st/images/lp-external-link-arrow.png!|http://www.oracle.com/technetwork/java/javase/8u171-relnotes-430.html#JDK-8189997]
> Enhanced KeyStore Mechanisms*
> A new security property named {{jceks.key.serialFilter}} has been introduced.
> If this filter is configured, the JCEKS KeyStore uses it during the
> deserialization of the encrypted Key object stored inside a SecretKeyEntry.
> If it is not configured or if the filter result is UNDECIDED (for example,
> none of the patterns match), then the filter configured by
> {{jdk.serialFilter}} is consulted.
> If the system property {{jceks.key.serialFilter}} is also supplied, it
> supersedes the security property value defined here.
> The filter pattern uses the same format as {{jdk.serialFilter}}. The default
> pattern allows {{java.lang.Enum}}, {{java.security.KeyRep}},
> {{java.security.KeyRep$Type}}, and {{javax.crypto.spec.SecretKeySpec}} but
> rejects all the others.
> Customers storing a SecretKey that does not serialize to the above types must
> modify the filter to make the key extractable.
> {quote}
> We believe this causes some test failures:
>
> {quote}{{{color:#33}java.io.IOException: Can't recover key for myKey from
> keystore
> file:/{color}{color:#33}home/{color}{color:#33}jenkins/{color}{color:#33}workspace/{color}{color:#33}hadoopFullBuild/{color}{color:#33}hadoop-hdfs-project/{color}{color:#33}hadoop-hdfs/{color}{color:#33}target/{color}{color:#33}test/{color}{color:#33}data/{color}{color:#33}53406117-0132-401e-a67d-6672f1b6a14a/{color}{color:#33}test.jks
> at
> org.apache.hadoop.crypto.key.JavaKeyStoreProvider.getMetadata(JavaKeyStoreProvider.java:424)
> at
> org.apache.hadoop.crypto.key.KeyProviderExtension.getMetadata(KeyProviderExtension.java:100)
> at
> org.apache.hadoop.hdfs.server.namenode.FSDirEncryptionZoneOp.ensureKeyIsInitialized(FSDirEncryptionZoneOp.java:124)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.createEncryptionZone(FSNamesystem.java:7227)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.createEncryptionZone(NameNodeRpcServer.java:2082)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.createEncryptionZone(ClientNamenodeProtocolServerSideTranslatorPB.java:1524)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991) at
> org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869) at
> org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:422) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1965)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675) Caused by:
> java.security.UnrecoverableKeyException: Rejected by the
> jceks.key.serialFilter or jdk.serialFilter property at
> com.sun.crypto.provider.KeyProtector.unseal(KeyProtector.java:352) at
> com.sun.crypto.provider.JceKeyStore.engineGetKey(JceKeyStore.java:136) at
> java.security.KeyStore.getKey(KeyStore.java:1023){color}}}
> {quote}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
