[jira] [Comment Edited] (HDFS-15098) Add SM4 encryption method for HDFS
[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179470#comment-17179470 ] liusheng edited comment on HDFS-15098 at 8/19/20, 8:54 AM: --- Hi [~lindongdong], Thank you for your review, I am not an expert and not sure about the potential compatible issue in rolling upgrade process, but should we keep consistency in interacted components to avoid old jars called by new native libaries when do rolling upgrade ? I have tried adding the method definition you mentioned, but CI raised error as bellow: !image-2020-08-19-16-54-41-341.png! Thank for your suggestion about [OpensslSecureRandom.c|https://github.com/apache/hadoop/pull/2211/files#diff-3ee504e8c2a27c840c39c4496a27cc02], will check later. was (Author: seanlau): Hi [~lindongdong], Thank you for your review, I am not an expert and not sure about the potential compatible issue in rolling upgrade process, but should we keep consistency in interacted components to avoid old jars called by new native libaries when do rolling upgrade ? I have tried adding the method definition you mentioned, but CI raised error as bellow: !image-2020-08-18-16-40-30-090.png! Thank for your suggestion about [OpensslSecureRandom.c|https://github.com/apache/hadoop/pull/2211/files#diff-3ee504e8c2a27c840c39c4496a27cc02], will check later. > Add SM4 encryption method for HDFS > -- > > Key: HDFS-15098 > URL: https://issues.apache.org/jira/browse/HDFS-15098 > Project: Hadoop HDFS > Issue Type: New Feature >Affects Versions: 3.4.0 >Reporter: liusheng >Assignee: liusheng >Priority: Major > Labels: sm4 > Attachments: HDFS-15098.001.patch, HDFS-15098.002.patch, > HDFS-15098.003.patch, HDFS-15098.004.patch, HDFS-15098.005.patch, > HDFS-15098.006.patch, HDFS-15098.007.patch, HDFS-15098.008.patch, > HDFS-15098.009.patch, image-2020-08-19-16-54-41-341.png > > > SM4 (formerly SMS4)is a block cipher used in the Chinese National Standard > for Wireless LAN WAPI (Wired Authentication and Privacy Infrastructure). > SM4 was a cipher proposed to for the IEEE 802.11i standard, but has so far > been rejected by ISO. One of the reasons for the rejection has been > opposition to the WAPI fast-track proposal by the IEEE. please see: > [https://en.wikipedia.org/wiki/SM4_(cipher)] > > *Use sm4 on hdfs as follows:* > 1.Configure Hadoop KMS > 2.test HDFS sm4 > hadoop key create key1 -cipher 'SM4/CTR/NoPadding' > hdfs dfs -mkdir /benchmarks > hdfs crypto -createZone -keyName key1 -path /benchmarks > *requires:* > 1.openssl version >=1.1.1 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-15098) Add SM4 encryption method for HDFS
[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179285#comment-17179285 ] liusheng edited comment on HDFS-15098 at 8/18/20, 1:06 AM: --- [~lindongdong] you can check the previous CI results, if we define an unused method, CI will be unhappy. and I still don't know why there is a compatible issue, the method you said will not be used any place, can you please explain more ? The modification of [OpensslSecureRandom.c|https://github.com/apache/hadoop/pull/2211/files#diff-3ee504e8c2a27c840c39c4496a27cc02] I think is to make OpenSSL interface to support SM4, such as check of the OpenSSL version(SM4 feature require OpenSSL>=1.1.1). was (Author: seanlau): [~lindongdong] you can check the previous CI results, if we define an unused method, CI will be unhappy. and I still don't know why there is a compatible issue, the method you said will not be used any place, can you please explain more ? The modification of [OpensslSecureRandom.c|https://github.com/apache/hadoop/pull/2211/files#diff-3ee504e8c2a27c840c39c4496a27cc02] I think is to make OpenSSL interface to support SM4, such as check of the OpenSSL version(SM4 feature require ). it is > Add SM4 encryption method for HDFS > -- > > Key: HDFS-15098 > URL: https://issues.apache.org/jira/browse/HDFS-15098 > Project: Hadoop HDFS > Issue Type: New Feature >Affects Versions: 3.4.0 >Reporter: liusheng >Assignee: liusheng >Priority: Major > Labels: sm4 > Attachments: HDFS-15098.001.patch, HDFS-15098.002.patch, > HDFS-15098.003.patch, HDFS-15098.004.patch, HDFS-15098.005.patch, > HDFS-15098.006.patch, HDFS-15098.007.patch, HDFS-15098.008.patch, > HDFS-15098.009.patch > > > SM4 (formerly SMS4)is a block cipher used in the Chinese National Standard > for Wireless LAN WAPI (Wired Authentication and Privacy Infrastructure). > SM4 was a cipher proposed to for the IEEE 802.11i standard, but has so far > been rejected by ISO. One of the reasons for the rejection has been > opposition to the WAPI fast-track proposal by the IEEE. please see: > [https://en.wikipedia.org/wiki/SM4_(cipher)] > > *Use sm4 on hdfs as follows:* > 1.Configure Hadoop KMS > 2.test HDFS sm4 > hadoop key create key1 -cipher 'SM4/CTR/NoPadding' > hdfs dfs -mkdir /benchmarks > hdfs crypto -createZone -keyName key1 -path /benchmarks > *requires:* > 1.openssl version >=1.1.1 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-15098) Add SM4 encryption method for HDFS
[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179285#comment-17179285 ] liusheng edited comment on HDFS-15098 at 8/18/20, 1:06 AM: --- [~lindongdong] you can check the previous CI results, if we define an unused method, CI will be unhappy. and I still don't know why there is a compatible issue, the method you said will not be used any place, can you please explain more ? The modification of [OpensslSecureRandom.c|https://github.com/apache/hadoop/pull/2211/files#diff-3ee504e8c2a27c840c39c4496a27cc02] I think is to make OpenSSL interface to support SM4, such as check of the OpenSSL version(SM4 feature require ). it is was (Author: seanlau): [~lindongdong] you can check the previous CI results, if we define an unused method, CI will be unhappy. and I still don't know why there is a compatible issue, the method you said will not be used any place, can you please explain more ? @[~zZtai] can you please answer the reason of modification of OpensslSecureRandom.c for lindongdong? > Add SM4 encryption method for HDFS > -- > > Key: HDFS-15098 > URL: https://issues.apache.org/jira/browse/HDFS-15098 > Project: Hadoop HDFS > Issue Type: New Feature >Affects Versions: 3.4.0 >Reporter: liusheng >Assignee: liusheng >Priority: Major > Labels: sm4 > Attachments: HDFS-15098.001.patch, HDFS-15098.002.patch, > HDFS-15098.003.patch, HDFS-15098.004.patch, HDFS-15098.005.patch, > HDFS-15098.006.patch, HDFS-15098.007.patch, HDFS-15098.008.patch, > HDFS-15098.009.patch > > > SM4 (formerly SMS4)is a block cipher used in the Chinese National Standard > for Wireless LAN WAPI (Wired Authentication and Privacy Infrastructure). > SM4 was a cipher proposed to for the IEEE 802.11i standard, but has so far > been rejected by ISO. One of the reasons for the rejection has been > opposition to the WAPI fast-track proposal by the IEEE. please see: > [https://en.wikipedia.org/wiki/SM4_(cipher)] > > *Use sm4 on hdfs as follows:* > 1.Configure Hadoop KMS > 2.test HDFS sm4 > hadoop key create key1 -cipher 'SM4/CTR/NoPadding' > hdfs dfs -mkdir /benchmarks > hdfs crypto -createZone -keyName key1 -path /benchmarks > *requires:* > 1.openssl version >=1.1.1 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-15098) Add SM4 encryption method for HDFS
[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17179285#comment-17179285 ] liusheng edited comment on HDFS-15098 at 8/18/20, 1:01 AM: --- [~lindongdong] you can check the previous CI results, if we define an unused method, CI will be unhappy. and I still don't know why there is a compatible issue, the method you said will not be used any place, can you please explain more ? @[~zZtai] can you please answer the reason of modification of OpensslSecureRandom.c for lindongdong? was (Author: seanlau): [~lindongdong] you can check the previous CI results, if we define an unused method, CI will unhappy. and I still don't know why there is a compatible issue, the method you said will not be used any place, can you please explain more ? @[~zZtai] can you please answer the reason of modification of OpensslSecureRandom.c for lindongdong? > Add SM4 encryption method for HDFS > -- > > Key: HDFS-15098 > URL: https://issues.apache.org/jira/browse/HDFS-15098 > Project: Hadoop HDFS > Issue Type: New Feature >Affects Versions: 3.4.0 >Reporter: liusheng >Assignee: liusheng >Priority: Major > Labels: sm4 > Attachments: HDFS-15098.001.patch, HDFS-15098.002.patch, > HDFS-15098.003.patch, HDFS-15098.004.patch, HDFS-15098.005.patch, > HDFS-15098.006.patch, HDFS-15098.007.patch, HDFS-15098.008.patch, > HDFS-15098.009.patch > > > SM4 (formerly SMS4)is a block cipher used in the Chinese National Standard > for Wireless LAN WAPI (Wired Authentication and Privacy Infrastructure). > SM4 was a cipher proposed to for the IEEE 802.11i standard, but has so far > been rejected by ISO. One of the reasons for the rejection has been > opposition to the WAPI fast-track proposal by the IEEE. please see: > [https://en.wikipedia.org/wiki/SM4_(cipher)] > > *Use sm4 on hdfs as follows:* > 1.Configure Hadoop KMS > 2.test HDFS sm4 > hadoop key create key1 -cipher 'SM4/CTR/NoPadding' > hdfs dfs -mkdir /benchmarks > hdfs crypto -createZone -keyName key1 -path /benchmarks > *requires:* > 1.openssl version >=1.1.1 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-15098) Add SM4 encryption method for HDFS
[
https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17154413#comment-17154413
]
Vinayakumar B edited comment on HDFS-15098 at 7/9/20, 11:19 AM:
Thanks [~zZtai] for the contribution
Overall changes looks good. Following are my comments. Please check.
1. Adding this provider should be configurable. And update the document as
required.
As already mentioned by [~lindongdong] no need to add to JDK dirs. May be
Issue descreption can be updated.
so, following addition of Provider needs to be done only if its configured.
Because direct adding of {{BounctCatleProvider}} seems to change the existing
default behavior in some cases. Ex: {{TestKeyShell#createInvalidKeySize()}}
suppose to fail with keysize 56. But it passes when provider is BC. So it
should be used only on user's demand. So making it configurable would be wise
choise.
{code:java}
+ Security.addProvider(new BouncyCastleProvider());
{code}
In KeyProvider.java it can be added as below.
{code:java}
String jceProvider = conf.get(HADOOP_SECURITY_CRYPTO_JCE_PROVIDER_KEY);
if (BouncyCastleProvider.PROVIDER_NAME.equals(jceProvider)) {
Security.addProvider(new BouncyCastleProvider());
}
{code}
In JceSm4CtrCryptoCodec.java should add on setConf() instead of constructor.
{code:java}
provider = conf.get(HADOOP_SECURITY_CRYPTO_JCE_PROVIDER_KEY,
BouncyCastleProvider.PROVIDER_NAME);
final String secureRandomAlg = conf.get(
HADOOP_SECURITY_JAVA_SECURE_RANDOM_ALGORITHM_KEY,
HADOOP_SECURITY_JAVA_SECURE_RANDOM_ALGORITHM_DEFAULT);
if (BouncyCastleProvider.PROVIDER_NAME.equals(provider)) {
Security.addProvider(new BouncyCastleProvider());
}
{code}
2. With Above change, {{TestKeyShell#testInvalidKeySize()}} will not fail
anymore, as BC provider will not be added by default. So changes in
{{TestKeyShell}} can be reverted.
3. In {{TestCryptoCodec.java}}
Remove these lines from every test.
{code:java}
try {
KeyGenerator keyGenerator = KeyGenerator.getInstance("SM4");
} catch (Exception e) {
Assume.assumeTrue(false);
}
{code}
4. In {{TestCryptoCodec#testJceSm4CtrCryptoCodec}} change this config as below.
{code:java}
conf.set(HADOOP_SECURITY_CRYPTO_CODEC_CLASSES_SM4_CTR_NOPADDING_KEY,
JceSm4CtrCryptoCodec.class.getName());{code}
Uncomment following lines
{code:java}
//cryptoCodecTest(conf, seed, count,
//jceSm4CodecClass, opensslSm4CodecClass, iv);
{code}
{code:java}
//cryptoCodecTest(conf, seed, count,
//jceSm4CodecClass, opensslSm4CodecClass, iv);
{code}
5. Avoid import statements with * in all classes. import only required classes
directly.
6. {{HdfsKMSUtil.getCryptoCodec()}} is not logging {{JceSm4CTRCodec}}. May be
can log all classnames, when its not null without checking the instanceof ?
7. I can see lot of code is same between AES and SM4 codecs, except the
classnames and algorithm names. May be refactoring would help to reduce the
duplicate code.
8. I think in {{hdfs.proto}} SM4 enum value can be changed to 3 directly.
{code}enum CipherSuiteProto {
UNKNOWN = 1;
AES_CTR_NOPADDING = 2;
SM4_CTR_NOPADDING = 3;
}{code}
9. In {{OpenSecureRandom.c}} following functions' declarations and definitions
can be kept within {{OPENSSL_VERSION_NUMBER < 0x1010L}} block. i.e.
following fuctions should be used only when {{OPENSSL_VERSION_NUMBER <
0x1010L}} is true:
{code}
static void locks_setup(void)
static void locks_cleanup(void)
static void pthreads_locking_callback(int mode, int type, char *file, int
line)
static unsigned long pthreads_thread_id(void)
{code}
was (Author: vinayrpet):
Thanks [~zZtai] for the contribution
Overall changes looks good. Following are my comments. Please check.
1. Adding this provider should be configurable. And update the document as
required.
As already mentioned by [~lindongdong] no need to add to JDK dirs. May be
Issue descreption can be updated.
so, following addition of Provider needs to be done only if its configured.
Because direct adding of {{BounctCatleProvider}} seems to change the existing
default behavior in some cases. Ex: {{TestKeyShell#createInvalidKeySize()}}
suppose to fail with keysize 56. But it passes when provider is BC. So it
should be used only on user's demand. So making it configurable would be wise
choise.
{code:java}
+ Security.addProvider(new BouncyCastleProvider());
{code}
In KeyProvider.java it can be added as below.
{code:java}
String jceProvider = conf.get(HADOOP_SECURITY_CRYPTO_JCE_PROVIDER_KEY);
if (BouncyCastleProvider.PROVIDER_NAME.equals(jceProvider)) {
Security.addProvider(new BouncyCastleProvider());
}
{code}
In JceSm4CtrCryptoCodec.java should add on setConf() instead of constructor.
[jira] [Comment Edited] (HDFS-15098) Add SM4 encryption method for HDFS
[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17143435#comment-17143435 ] liusheng edited comment on HDFS-15098 at 6/24/20, 1:30 AM: --- Hi [~weichiu], I am so sorry that we have a delay for this feature, now we have updated the patches and tested OK locally, we have added test cases, config options, docs in the patch. currently, the SM4 is supported in openssl>=1.1.1, if this requirement is unstatisfied, it will fall back to use the SM4 implementation of BouncyCastleProvider which is already a dependency of Hadoop. So, now we only need to cofigure KMS services to enable SM4 support. Could you please help to review again ? was (Author: seanlau): Hi [~weichiu], I am so sorry that we have a delay for this feature, now we have updated the patches and tested OK locally, we have added test cases, config options, docs in the patch. currently, the SM4 is supported in openssl>=1.1.1, if this requirement is unstatisfied, it will fall back to use the SM4 implementation BouncyCastleProvider which is already a dependency of Hadoop. So, now we only need to cofigure KMS services to enable SM4 support. Could you please help to review again ? > Add SM4 encryption method for HDFS > -- > > Key: HDFS-15098 > URL: https://issues.apache.org/jira/browse/HDFS-15098 > Project: Hadoop HDFS > Issue Type: New Feature >Affects Versions: 3.4.0 >Reporter: liusheng >Assignee: zZtai >Priority: Major > Labels: sm4 > Attachments: HDFS-15098.001.patch, HDFS-15098.002.patch, > HDFS-15098.003.patch, HDFS-15098.004.patch, HDFS-15098.005.patch, > HDFS-15098.006.patch, HDFS-15098.007.patch > > > SM4 (formerly SMS4)is a block cipher used in the Chinese National Standard > for Wireless LAN WAPI (Wired Authentication and Privacy Infrastructure). > SM4 was a cipher proposed to for the IEEE 802.11i standard, but has so far > been rejected by ISO. One of the reasons for the rejection has been > opposition to the WAPI fast-track proposal by the IEEE. please see: > [https://en.wikipedia.org/wiki/SM4_(cipher)] > > *Use sm4 on hdfs as follows:* > 1.download Bouncy Castle Crypto APIs from bouncycastle.org > [https://bouncycastle.org/download/bcprov-ext-jdk15on-165.jar] > 2.Configure JDK > Place bcprov-ext-jdk15on-165.jar in $JAVA_HOME/jre/lib/ext directory, > add "security.provider.10=org.bouncycastle.jce.provider.BouncyCastleProvider" > to $JAVA_HOME/jre/lib/security/java.security file > 3.Configure Hadoop KMS > 4.test HDFS sm4 > hadoop key create key1 -cipher 'SM4/CTR/NoPadding' > hdfs dfs -mkdir /benchmarks > hdfs crypto -createZone -keyName key1 -path /benchmarks > *requires:* > 1.openssl version >=1.1.1 > 2.configure Bouncy Castle Crypto on JDK -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-15098) Add SM4 encryption method for HDFS
[
https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17134088#comment-17134088
]
liusheng edited comment on HDFS-15098 at 6/12/20, 9:44 AM:
---
Hi [~Andrea_Julianos_one] [~lindongdong],
Thanks for your verification, we don't need the following 2 steps now
{code:java}
1.download Bouncy Castle Crypto APIs from bouncycastle.org
https://bouncycastle.org/download/bcprov-ext-jdk15on-165.jar
2.Configure JDK
Place bcprov-ext-jdk15on-165.jar in $JAVA_HOME/jre/lib/ext directory,
add "security.provider.10=org.bouncycastle.jce.provider.BouncyCastleProvider"
to $JAVA_HOME/jre/lib/security/java.security file
{code}
because the BouncyCastleProvider is alread a denpendency of Hadoop currently,
and we can initial and add the provider than manually editing the
"java.security" file. see:
[https://stackoverflow.com/questions/8970/bouncycastle-nosuchproviderexception-even-though-its-a-maven-dependency]
was (Author: seanlau):
[~Andrea_Julianos_one] [~lindongdong]
We don't need the following 2 steps now
{code:java}
1.download Bouncy Castle Crypto APIs from bouncycastle.org
https://bouncycastle.org/download/bcprov-ext-jdk15on-165.jar
2.Configure JDK
Place bcprov-ext-jdk15on-165.jar in $JAVA_HOME/jre/lib/ext directory,
add "security.provider.10=org.bouncycastle.jce.provider.BouncyCastleProvider"
to $JAVA_HOME/jre/lib/security/java.security file
{code}
because the BouncyCastleProvider is alread a denpendency of Hadoop currently,
and we can initial and add the provider than manually editing the
"java.security" file. see:
> Add SM4 encryption method for HDFS
> --
>
> Key: HDFS-15098
> URL: https://issues.apache.org/jira/browse/HDFS-15098
> Project: Hadoop HDFS
> Issue Type: New Feature
>Affects Versions: 3.4.0
>Reporter: liusheng
>Assignee: zZtai
>Priority: Major
> Labels: sm4
> Attachments: HDFS-15098.001.patch, HDFS-15098.002.patch,
> HDFS-15098.003.patch, HDFS-15098.004.patch, HDFS-15098.005.patch,
> HDFS-15098.006.patch, HDFS-15098.007.patch
>
>
> SM4 (formerly SMS4)is a block cipher used in the Chinese National Standard
> for Wireless LAN WAPI (Wired Authentication and Privacy Infrastructure).
> SM4 was a cipher proposed to for the IEEE 802.11i standard, but has so far
> been rejected by ISO. One of the reasons for the rejection has been
> opposition to the WAPI fast-track proposal by the IEEE. please see:
> [https://en.wikipedia.org/wiki/SM4_(cipher)]
>
> *Use sm4 on hdfs as follows:*
> 1.download Bouncy Castle Crypto APIs from bouncycastle.org
> [https://bouncycastle.org/download/bcprov-ext-jdk15on-165.jar]
> 2.Configure JDK
> Place bcprov-ext-jdk15on-165.jar in $JAVA_HOME/jre/lib/ext directory,
> add "security.provider.10=org.bouncycastle.jce.provider.BouncyCastleProvider"
> to $JAVA_HOME/jre/lib/security/java.security file
> 3.Configure Hadoop KMS
> 4.test HDFS sm4
> hadoop key create key1 -cipher 'SM4/CTR/NoPadding'
> hdfs dfs -mkdir /benchmarks
> hdfs crypto -createZone -keyName key1 -path /benchmarks
> *requires:*
> 1.openssl version >=1.1.1
> 2.configure Bouncy Castle Crypto on JDK
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-15098) Add SM4 encryption method for HDFS
[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17128774#comment-17128774 ] liusheng edited comment on HDFS-15098 at 6/9/20, 3:40 AM: -- Hi [~lindongdong], As [~zZtai] explained, the SM4 feature is supportted in OpenSSL >=1.1.1 version, if this requirement is satisfied in environment, we don't need these 2 steps, if not, the SM4 feature will fallback to use an alternative implementation of SM4 feature with the Bouncy Castle Crypto provider jar package. see: [https://www.bouncycastle.org/specifications.html] The corresponding AES encryption support also have similar implementation. but the AES is internally supportted in JDK, while SM4 JCE implementation need the external jar package. was (Author: seanlau): Hi [~lindongdong], As [~zZtai] explained, the SM4 feature is supportted in OpenSSL >=1.1.1 version, if this requirement is satisfied in environment, we don't need these 2 steps, if not, the SM4 feature will fallback to use an alternative implementation of SM4 feature with the Bouncy Castle Crypto provider jar package. see: [https://www.bouncycastle.org/specifications.html] The corresponding AES encryption support also have similar implementation. > Add SM4 encryption method for HDFS > -- > > Key: HDFS-15098 > URL: https://issues.apache.org/jira/browse/HDFS-15098 > Project: Hadoop HDFS > Issue Type: New Feature >Affects Versions: 3.4.0 >Reporter: liusheng >Assignee: zZtai >Priority: Major > Labels: sm4 > Attachments: HDFS-15098.001.patch, HDFS-15098.002.patch, > HDFS-15098.003.patch, HDFS-15098.004.patch, HDFS-15098.005.patch > > > SM4 (formerly SMS4)is a block cipher used in the Chinese National Standard > for Wireless LAN WAPI (Wired Authentication and Privacy Infrastructure). > SM4 was a cipher proposed to for the IEEE 802.11i standard, but has so far > been rejected by ISO. One of the reasons for the rejection has been > opposition to the WAPI fast-track proposal by the IEEE. please see: > [https://en.wikipedia.org/wiki/SM4_(cipher)] > > *Use sm4 on hdfs as follows:* > 1.download Bouncy Castle Crypto APIs from bouncycastle.org > [https://bouncycastle.org/download/bcprov-ext-jdk15on-165.jar] > 2.Configure JDK > Place bcprov-ext-jdk15on-165.jar in $JAVA_HOME/jre/lib/ext directory, > add "security.provider.10=org.bouncycastle.jce.provider.BouncyCastleProvider" > to $JAVA_HOME/jre/lib/security/java.security file > 3.Configure Hadoop KMS > 4.test HDFS sm4 > hadoop key create key1 -cipher 'SM4/CTR/NoPadding' > hdfs dfs -mkdir /benchmarks > hdfs crypto -createZone -keyName key1 -path /benchmarks > *requires:* > 1.openssl version >=1.1.1 > 2.configure Bouncy Castle Crypto on JDK -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-15098) Add SM4 encryption method for HDFS
[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17128774#comment-17128774 ] liusheng edited comment on HDFS-15098 at 6/9/20, 3:39 AM: -- Hi [~lindongdong], As [~zZtai] explained, the SM4 feature is supportted in OpenSSL >=1.1.1 version, if this requirement is satisfied in environment, we don't need these 2 steps, if not, the SM4 feature will fallback to use an alternative implementation of SM4 feature with the Bouncy Castle Crypto provider jar package. see: [https://www.bouncycastle.org/specifications.html] The corresponding AES encryption support also have similar implementation. was (Author: seanlau): Hi [~lindongdong], As [~zZtai] explained, the SM4 feature is supportted in OpenSSL >=1.1.1 version, if this requirement is satisfied in environment, we don't need these 2 steps, if not, the SM4 feature will fail back to use an alternative implementation of SM4 feature with the Bouncy Castle Crypto provider jar package. see: [https://www.bouncycastle.org/specifications.html] > Add SM4 encryption method for HDFS > -- > > Key: HDFS-15098 > URL: https://issues.apache.org/jira/browse/HDFS-15098 > Project: Hadoop HDFS > Issue Type: New Feature >Affects Versions: 3.4.0 >Reporter: liusheng >Assignee: zZtai >Priority: Major > Labels: sm4 > Attachments: HDFS-15098.001.patch, HDFS-15098.002.patch, > HDFS-15098.003.patch, HDFS-15098.004.patch, HDFS-15098.005.patch > > > SM4 (formerly SMS4)is a block cipher used in the Chinese National Standard > for Wireless LAN WAPI (Wired Authentication and Privacy Infrastructure). > SM4 was a cipher proposed to for the IEEE 802.11i standard, but has so far > been rejected by ISO. One of the reasons for the rejection has been > opposition to the WAPI fast-track proposal by the IEEE. please see: > [https://en.wikipedia.org/wiki/SM4_(cipher)] > > *Use sm4 on hdfs as follows:* > 1.download Bouncy Castle Crypto APIs from bouncycastle.org > [https://bouncycastle.org/download/bcprov-ext-jdk15on-165.jar] > 2.Configure JDK > Place bcprov-ext-jdk15on-165.jar in $JAVA_HOME/jre/lib/ext directory, > add "security.provider.10=org.bouncycastle.jce.provider.BouncyCastleProvider" > to $JAVA_HOME/jre/lib/security/java.security file > 3.Configure Hadoop KMS > 4.test HDFS sm4 > hadoop key create key1 -cipher 'SM4/CTR/NoPadding' > hdfs dfs -mkdir /benchmarks > hdfs crypto -createZone -keyName key1 -path /benchmarks > *requires:* > 1.openssl version >=1.1.1 > 2.configure Bouncy Castle Crypto on JDK -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-15098) Add SM4 encryption method for HDFS
[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17128774#comment-17128774 ] liusheng edited comment on HDFS-15098 at 6/9/20, 2:53 AM: -- Hi [~lindongdong], As [~zZtai] explained, the SM4 feature is supportted in OpenSSL >=1.1.1 version, if this requirement is satisfied in environment, we don't need these 2 steps, if not, the SM4 feature will fail back to use an alternative implementation of SM4 feature with the Bouncy Castle Crypto provider jar package. see: [https://www.bouncycastle.org/specifications.html] was (Author: seanlau): Hi [~lindongdong], As [~zZtai] explained, the SM4 feature is supportted in OpenSSL >=1.1.1 version, if this requirement is satisfied in environment, we don't need these 2 steps, if not, the SM4 feature will fail back to use an alternative implementation of SM4 feature with the Bouncy Castle Crypto provider jar package. > Add SM4 encryption method for HDFS > -- > > Key: HDFS-15098 > URL: https://issues.apache.org/jira/browse/HDFS-15098 > Project: Hadoop HDFS > Issue Type: New Feature >Affects Versions: 3.4.0 >Reporter: liusheng >Assignee: zZtai >Priority: Major > Labels: sm4 > Attachments: HDFS-15098.001.patch, HDFS-15098.002.patch, > HDFS-15098.003.patch, HDFS-15098.004.patch, HDFS-15098.005.patch > > > SM4 (formerly SMS4)is a block cipher used in the Chinese National Standard > for Wireless LAN WAPI (Wired Authentication and Privacy Infrastructure). > SM4 was a cipher proposed to for the IEEE 802.11i standard, but has so far > been rejected by ISO. One of the reasons for the rejection has been > opposition to the WAPI fast-track proposal by the IEEE. please see: > [https://en.wikipedia.org/wiki/SM4_(cipher)] > > *Use sm4 on hdfs as follows:* > 1.download Bouncy Castle Crypto APIs from bouncycastle.org > [https://bouncycastle.org/download/bcprov-ext-jdk15on-165.jar] > 2.Configure JDK > Place bcprov-ext-jdk15on-165.jar in $JAVA_HOME/jre/lib/ext directory, > add "security.provider.10=org.bouncycastle.jce.provider.BouncyCastleProvider" > to $JAVA_HOME/jre/lib/security/java.security file > 3.Configure Hadoop KMS > 4.test HDFS sm4 > hadoop key create key1 -cipher 'SM4/CTR/NoPadding' > hdfs dfs -mkdir /benchmarks > hdfs crypto -createZone -keyName key1 -path /benchmarks > *requires:* > 1.openssl version >=1.1.1 > 2.configure Bouncy Castle Crypto on JDK -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-15098) Add SM4 encryption method for HDFS
[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17128774#comment-17128774 ] liusheng edited comment on HDFS-15098 at 6/9/20, 2:22 AM: -- Hi [~lindongdong], As [~zZtai] explained, the SM4 feature is supportted in OpenSSL >=1.1.1 version, if this requirement is satisfied in environment, we don't need these 2 steps, if not, the SM4 feature will fail back to use an alternative implementation of SM4 feature with the Bouncy Castle Crypto provider jar package. was (Author: seanlau): Hi [~lindongdong], > Add SM4 encryption method for HDFS > -- > > Key: HDFS-15098 > URL: https://issues.apache.org/jira/browse/HDFS-15098 > Project: Hadoop HDFS > Issue Type: New Feature >Affects Versions: 3.4.0 >Reporter: liusheng >Assignee: zZtai >Priority: Major > Labels: sm4 > Attachments: HDFS-15098.001.patch, HDFS-15098.002.patch, > HDFS-15098.003.patch, HDFS-15098.004.patch, HDFS-15098.005.patch > > > SM4 (formerly SMS4)is a block cipher used in the Chinese National Standard > for Wireless LAN WAPI (Wired Authentication and Privacy Infrastructure). > SM4 was a cipher proposed to for the IEEE 802.11i standard, but has so far > been rejected by ISO. One of the reasons for the rejection has been > opposition to the WAPI fast-track proposal by the IEEE. please see: > [https://en.wikipedia.org/wiki/SM4_(cipher)] > > *Use sm4 on hdfs as follows:* > 1.download Bouncy Castle Crypto APIs from bouncycastle.org > [https://bouncycastle.org/download/bcprov-ext-jdk15on-165.jar] > 2.Configure JDK > Place bcprov-ext-jdk15on-165.jar in $JAVA_HOME/jre/lib/ext directory, > add "security.provider.10=org.bouncycastle.jce.provider.BouncyCastleProvider" > to $JAVA_HOME/jre/lib/security/java.security file > 3.Configure Hadoop KMS > 4.test HDFS sm4 > hadoop key create key1 -cipher 'SM4/CTR/NoPadding' > hdfs dfs -mkdir /benchmarks > hdfs crypto -createZone -keyName key1 -path /benchmarks > *requires:* > 1.openssl version >=1.1.1 > 2.configure Bouncy Castle Crypto on JDK -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-15098) Add SM4 encryption method for HDFS
[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17120129#comment-17120129 ] zZtai edited comment on HDFS-15098 at 5/30/20, 7:13 AM: [~lindongdong] as [~weichiu] mentioned , the existing crypto implementation should falls back to a Java implementation if openssl is not loaded, bouncycastle provides these capabilities . was (Author: zztai): [~lindongdong] > Add SM4 encryption method for HDFS > -- > > Key: HDFS-15098 > URL: https://issues.apache.org/jira/browse/HDFS-15098 > Project: Hadoop HDFS > Issue Type: New Feature >Affects Versions: 3.4.0 >Reporter: liusheng >Assignee: zZtai >Priority: Major > Labels: sm4 > Attachments: HDFS-15098.001.patch, HDFS-15098.002.patch, > HDFS-15098.003.patch, HDFS-15098.004.patch > > > SM4 (formerly SMS4)is a block cipher used in the Chinese National Standard > for Wireless LAN WAPI (Wired Authentication and Privacy Infrastructure). > SM4 was a cipher proposed to for the IEEE 802.11i standard, but has so far > been rejected by ISO. One of the reasons for the rejection has been > opposition to the WAPI fast-track proposal by the IEEE. please see: > [https://en.wikipedia.org/wiki/SM4_(cipher)] > > *Use sm4 on hdfs as follows:* > 1.download Bouncy Castle Crypto APIs from bouncycastle.org > [https://bouncycastle.org/download/bcprov-ext-jdk15on-165.jar] > 2.Configure JDK > Place bcprov-ext-jdk15on-165.jar in $JAVA_HOME/jre/lib/ext directory, > add "security.provider.10=org.bouncycastle.jce.provider.BouncyCastleProvider" > to $JAVA_HOME/jre/lib/security/java.security file > 3.Configure Hadoop KMS > 4.test HDFS sm4 > hadoop key create key1 -cipher 'SM4/CTR/NoPadding' > hdfs dfs -mkdir /benchmarks > hdfs crypto -createZone -keyName key1 -path /benchmarks > *requires:* > 1.openssl version >=1.1.1 > 2.configure Bouncy Castle Crypto on JDK -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (HDFS-15098) Add SM4 encryption method for HDFS
[
https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105426#comment-17105426
]
Andrea edited comment on HDFS-15098 at 5/12/20, 1:31 PM:
-
[~weichiu] [~zZtai]
Hi, I modified the method of KeyProvider.java that called generateKey(int size,
String algorithm). like this
{code:java}
// code placeholder
protected byte[] generateKey(int size, String algorithm)
throws NoSuchAlgorithmException {
algorithm = getAlgorithm(algorithm);
KeyGenerator keyGenerator = KeyGenerator.getInstance(algorithm);
keyGenerator.init(size);
byte[] key = keyGenerator.generateKey().getEncoded();
return key;
}
>
protected byte[] generateKey(int size, String algorithm)
throws NoSuchAlgorithmException {
if("SM4/CTR/NoPadding".equals(algorithm)){
algorithm = "AES/CTR/NoPadding"
};
algorithm = getAlgorithm(algorithm);
KeyGenerator keyGenerator = KeyGenerator.getInstance(algorithm);
keyGenerator.init(size);
byte[] key = keyGenerator.generateKey().getEncoded();
return key;
}
{code}
and run "hadoop key create key5 -cipher 'SM4/CTR/NoPadding' -size 128 -provider
kms://http@localhost:16000/kms "
I get a result like
{code:java}
// code placeholder
key5 has been successfully created with options
Options{cipher='SM4/CTR/NoPadding', bitLength=128, description='null',
attributes=null}.
KMSClientProvider[http://localhost:16000/kms/v1/] has been updated.
{code}
Now, I temporary fixed a bug, when I run "hadoop fs -put file /encryptZone"
that print console info : "Now Codec is OpensslSm4CryptoCodec",
In the past, I get the console info is : "Now Codec is
OpensslOpensslAesCtrCryptoCodec", that I used this patch.
the console info in DFSClient.java is
{code:java}
// code placeholder
private static CryptoCodec getCryptoCodec(Configuration conf,
FileEncryptionInfo feInfo) throws IOException {
final CipherSuite suite = feInfo.getCipherSuite();
if (suite.equals(CipherSuite.UNKNOWN)) {
throw new IOException("NameNode specified unknown CipherSuite with ID "
+ suite.getUnknownValue() + ", cannot instantiate CryptoCodec.");
}
final CryptoCodec codec = CryptoCodec.getInstance(conf, suite);
if (codec instanceof OpensslAesCtrCryptoCodec) {
System.out.println("Now Codec is OpensslAesCtrCryptoCodec");
}
if (codec instanceof OpensslSm4CtrCryptoCodec) {
System.out.println("Now Codec is OpensslSm4CtrCryptoCodec");
}
if (codec instanceof JceAesCtrCryptoCodec) {
System.out.println("Now Codec is JceAesCtrCryptoCodec");
}
{code}
It Seems like the method of PBHelper.java(Hadoop-hdfs), "convert(CipherSuite
suite)" or convert(CipherSuiteProto proto), They are still received
AES/CTR/NoPadding , If you do not specify SM4 as the cipher when execute "
hadoop key create "
So, what idea do you think?
Cheers!
was (Author: andrea_julianos_one):
[~weichiu] [~zZtai]
Hi, I modified the method of KeyProvider.java that called generateKey(int size,
String algorithm). like this
{code:java}
// code placeholder
protected byte[] generateKey(int size, String algorithm)
throws NoSuchAlgorithmException {
algorithm = getAlgorithm(algorithm);
KeyGenerator keyGenerator = KeyGenerator.getInstance(algorithm);
keyGenerator.init(size);
byte[] key = keyGenerator.generateKey().getEncoded();
return key;
}
>
protected byte[] generateKey(int size, String algorithm)
throws NoSuchAlgorithmException {
if("SM4/CTR/NoPadding".equals(algorithm)){
algorithm = "AES/CTR/NoPadding"
};
algorithm = getAlgorithm(algorithm);
KeyGenerator keyGenerator = KeyGenerator.getInstance(algorithm);
keyGenerator.init(size);
byte[] key = keyGenerator.generateKey().getEncoded();
return key;
}
{code}
and run "hadoop key create key5 -cipher 'SM4/CTR/NoPadding' -size 128 -provider
kms://http@localhost:16000/kms "
I get a result like
{code:java}
// code placeholder
key5 has been successfully created with options
Options{cipher='SM4/CTR/NoPadding', bitLength=128, description='null',
attributes=null}.
KMSClientProvider[http://localhost:16000/kms/v1/] has been updated.
{code}
Now, I temporary fixed a bug, when I run "hadoop fs -put file /encryptZone"
that print console info : "Now Codec is OpensslSm4CryptoCodec",
In the past, I get the console info is : "Now Codec is
OpensslOpensslAesCtrCryptoCodec", that I used this patch.
the console info in DFSClient.java is
{code:java}
// code placeholder
private static CryptoCodec getCryptoCodec(Configuration conf,
FileEncryptionInfo feInfo) throws IOException {
final CipherSuite suite = feInfo.getCipherSuite();
if (suite.equals(CipherSuite.UNKNOWN)) {
throw new IOException("NameNode specified unknown CipherSuite with ID "
+ suite.getUnknownValue() + ", cannot instantiate CryptoCodec.");
}
final CryptoCodec codec =
[jira] [Comment Edited] (HDFS-15098) Add SM4 encryption method for HDFS
[
https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105133#comment-17105133
]
Andrea edited comment on HDFS-15098 at 5/12/20, 6:44 AM:
-
[~weichiu] [~zZtai]
Hi, the message is KMS server side. I can know that "
java.security.NoSuchAlgorithmException: SM4 KeyGenerator not available" is
important. but there is nothing about SM4 KeyGenerator in this patch.
openssl1.1.1 is Adaptable, and bcprov-ext-jdk15on-165.jar was put in
JDK8_HOME/jre/lib/ext, and add info to java.security.
but Configure Hadoop KMS, I hava no info for how to set it.
Thank you for watch. cheers
{code:java}
// code placeholder
User keyAdmin1 (auth:SIMPLE) request POST http://localhost:16000/kms/v1/keys
caused exception.
java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1930)
at org.apache.hadoop.crypto.key.kms.server.KMS.createKey(KMS.java:148)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.hadoop.crypto.key.kms.server.KMSMDCFilter.doFilter(KMSMDCFilter.java:84)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:631)
at
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:301)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:579)
at
org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:130)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at
[jira] [Comment Edited] (HDFS-15098) Add SM4 encryption method for HDFS
[
https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17105133#comment-17105133
]
Andrea edited comment on HDFS-15098 at 5/12/20, 6:39 AM:
-
[~weichiu] [~zZtai]
Hi, the message is KMS server side. I can know that "
java.security.NoSuchAlgorithmException: SM4 KeyGenerator not available" is
important. but there is nothing about SM4 KeyGenerator in this patch.
openssl1.1.1 is Adaptable, and bcprov-ext-jdk15on-165.jar was put in
JDK8_HOME/jre/lib/ext, and add info to java.security.
Thank you for watch. cheers
{code:java}
// code placeholder
User keyAdmin1 (auth:SIMPLE) request POST http://localhost:16000/kms/v1/keys
caused exception.
java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1930)
at org.apache.hadoop.crypto.key.kms.server.KMS.createKey(KMS.java:148)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
at
com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
at
com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
at
com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
at
com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
at
com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
at
com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
at
com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
at
com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
at
com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:723)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.hadoop.crypto.key.kms.server.KMSMDCFilter.doFilter(KMSMDCFilter.java:84)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:631)
at
org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.doFilter(DelegationTokenAuthenticationFilter.java:301)
at
org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:579)
at
org.apache.hadoop.crypto.key.kms.server.KMSAuthenticationFilter.doFilter(KMSAuthenticationFilter.java:130)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at
[jira] [Comment Edited] (HDFS-15098) Add SM4 encryption method for HDFS
[ https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043209#comment-17043209 ] Andrea edited comment on HDFS-15098 at 2/24/20 11:52 AM: - This patch can be used which hadoop version and openssl version ? was (Author: andrea_julianos_one): This patch can be used whice hadoop version and openssl version ? > Add SM4 encryption method for HDFS > -- > > Key: HDFS-15098 > URL: https://issues.apache.org/jira/browse/HDFS-15098 > Project: Hadoop HDFS > Issue Type: New Feature >Reporter: liusheng >Priority: Major > Attachments: HDFS-15098.001.patch > > > SM4 (formerly SMS4)is a block cipher used in the Chinese National Standard > for Wireless LAN WAPI (Wired Authentication and Privacy Infrastructure). > SM4 was a cipher proposed to for the IEEE 802.11i standard, but has so far > been rejected by ISO. One of the reasons for the rejection has been > opposition to the WAPI fast-track proposal by the IEEE. please see: > [https://en.wikipedia.org/wiki/SM4_(cipher)] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
