[
https://issues.apache.org/jira/browse/HDFS-15098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17185104#comment-17185104
]
Vinayakumar B commented on HDFS-15098:
--------------------------------------
Hi [~seanlau], Please check comments
1. Change the name {{JceCtrcryptoCodec}} to {{JceCtrCryptoCodec}}
2. No need to have following redundant constants. Instead directly use the
original constants itself. For example, instead of using {{AES_BLOCK_SIZE}} use
{{CipherSuite.AES_CTR_NOPADDING.getAlgorithmBlockSize()}}
{code:java}
protected static final CipherSuite AES_SUITE = CipherSuite.AES_CTR_NOPADDING;
protected static final CipherSuite SM4_SUITE = CipherSuite.SM4_CTR_NOPADDING;
protected static final int AES_BLOCK_SIZE = AES_SUITE.getAlgorithmBlockSize();
protected static final int SM4_BLOCK_SIZE = SM4_SUITE.getAlgorithmBlockSize();
{code}
Example in JceAesCtrCryptoCodec
{code:java}
@Override
public CipherSuite getCipherSuite() {
return CipherSuite.AES_CTR_NOPADDING;
}
@Override
public void calculateIV(byte[] initIV, long counter, byte[] iv) {
super.calculateIV(initIV, counter, iv,
getCipherSuite().getAlgorithmBlockSize());
}
@Override
public Encryptor createEncryptor() throws GeneralSecurityException {
return new JceCtrCipher(Cipher.ENCRYPT_MODE, getProvider(),
getCipherSuite(), "AES");
}
@Override
public Decryptor createDecryptor() throws GeneralSecurityException {
return new JceCtrCipher(Cipher.DECRYPT_MODE, getProvider(),
getCipherSuite(), "AES");
}
{code}
3. In JceCtrCryptoCodec,
Instead of following method,
{code:java}
public void log(GeneralSecurityException e) {
}
{code}
make an abstract method to get Logger and use that logger to log message
in JceCtrCryptoCodec.java
{code:java}
protected abstract Logger getLogger();
{code}
and
{code:java}
} catch(GeneralSecurityException e) {
getLogger().warn(e.getMessage());
random = new SecureRandom();
}
{code}
In JceAesCtrCryptoCodec and JceSm4CtrCryptoCodec
{code:java}
@Override
public Logger getLogger() {
return LOG;
}
{code}
4. Similar to #2 and #3, make the changes in OpensslCtrCryptoCodec,
OpensslAesCtrCryptoCodec and OpensslSm4CtrCryptoCodec. Instead of {{useLog}}
use {{getLogger()}} method to get the corresponding logger and move following
to corresponding classes, make them private and return them in {{getLogger()}}.
{code:java}
protected static final Logger SM4_LOG =
LoggerFactory.getLogger(OpensslSm4CtrCryptoCodec.class.getName());
protected static final Logger AES_LOG =
LoggerFactory.getLogger(OpensslAesCtrCryptoCodec.class.getName());
{code}
5. In OpensslCtrCryptoCodec, initialization of engineId is done only in case of
SM4. So this part of code can be moved to
{{OpensslSm4CtrCryptoCodec#setConf()}} as below.
{code:java}
@Override
public void setConf(Configuration conf) {
super.setConf(conf);
setEngineId(conf.get(HADOOP_SECURITY_OPENSSL_ENGINE_ID_KEY));
}
{code}
6. {{OpensslCtrCryptoCodec#close()}} can be changed as below
{code:java}
public void close() throws IOException {
if (this.random instanceof Closeable) {
Closeable r = (Closeable) this.random;
IOUtils.cleanupWithLogger(getLogger(), r);
}
}
{code}
7. Revert unnecessary changes in HdfsKMSUtil.java
8. in KeyProvider constructor move addition of BouncyCastleProvider out of if
block as following
{code:java}
public KeyProvider(Configuration conf) {
this.conf = new Configuration(conf);
// Added for HADOOP-15473. Configured serialFilter property fixes
// java.security.UnrecoverableKeyException in JDK 8u171.
if(System.getProperty(JCEKS_KEY_SERIAL_FILTER) == null) {
String serialFilter =
conf.get(HADOOP_SECURITY_CRYPTO_JCEKS_KEY_SERIALFILTER,
JCEKS_KEY_SERIALFILTER_DEFAULT);
System.setProperty(JCEKS_KEY_SERIAL_FILTER, serialFilter);
}
String jceProvider = conf.get(HADOOP_SECURITY_CRYPTO_JCE_PROVIDER_KEY);
if (BouncyCastleProvider.PROVIDER_NAME.equals(jceProvider)) {
Security.addProvider(new BouncyCastleProvider());
}
}
{code}
9. Right now all tests of {{TestCryptoStreamsWithJceSm4CtrCryptoCodec}} are
getting skipped. Do following changes to get it fixed.
Remove below code in {{TestCryptoStreamsWithJceSm4CtrCryptoCodec#init()}}
{code:java}
try {
KeyGenerator keyGenerator = KeyGenerator.getInstance("SM4");
} catch (Exception e) {
Assume.assumeTrue(false);
}
{code}
Add Following config in {{TestCryptoStreamsWithJceSm4CtrCryptoCodec#init()}}
{code:java}
conf.set(HADOOP_SECURITY_CRYPTO_JCE_PROVIDER_KEY,
BouncyCastleProvider.PROVIDER_NAME);
{code}
10. To avoid javac warnings in
{{TestCryptoStreamsWithOpensslSm4CtrCryptoCodec}} and
{{TestCryptoStreamsWithOpensslAesCtrCryptoCodec}} related to usage of
deprecated {{WhiteBox}} do following changes
add a method {{isClosed()}} in {{OsSecureRandom}}
{code:java}
@VisibleForTesting
public boolean isClosed() {
return stream == null;
}
{code}
and use this for assertions in test {{testCodecClosesRandom()}} in
{{TestCryptoStreamsWithOpensslSm4CtrCryptoCodec}} and
{{TestCryptoStreamsWithOpensslAesCtrCryptoCodec}} as shown below
{code:java}
@Test
public void testCodecClosesRandom() throws Exception {
GenericTestUtils.assumeInNativeProfile();
Configuration conf = new Configuration();
conf.set(HADOOP_SECURITY_CRYPTO_CIPHER_SUITE_KEY, "SM4/CTR/NoPadding");
conf.set(HADOOP_SECURITY_CRYPTO_CODEC_CLASSES_SM4_CTR_NOPADDING_KEY,
OpensslSm4CtrCryptoCodec.class.getName());
conf.set(
CommonConfigurationKeysPublic.
HADOOP_SECURITY_SECURE_RANDOM_IMPL_KEY,
OsSecureRandom.class.getName());
CryptoCodec codecWithRandom = CryptoCodec.getInstance(conf);
assertNotNull("Unable to instantiate codec " +
OpensslSm4CtrCryptoCodec.class.getName() + ", is the required "
+ "version of OpenSSL installed?", codecWithRandom);
OsSecureRandom random = (OsSecureRandom)
((OpensslSm4CtrCryptoCodec) codecWithRandom).getRandom();
// trigger the OsSecureRandom to create an internal FileInputStream
random.nextBytes(new byte[10]);
assertFalse(random.isClosed());
// verify closing the codec closes the codec's random's stream.
codecWithRandom.close();
assertTrue(random.isClosed());
}
{code}
> Add SM4 encryption method for HDFS
> ----------------------------------
>
> Key: HDFS-15098
> URL: https://issues.apache.org/jira/browse/HDFS-15098
> Project: Hadoop HDFS
> Issue Type: New Feature
> Affects Versions: 3.4.0
> Reporter: liusheng
> Assignee: liusheng
> Priority: Major
> Labels: sm4
> Attachments: HDFS-15098.001.patch, HDFS-15098.002.patch,
> HDFS-15098.003.patch, HDFS-15098.004.patch, HDFS-15098.005.patch,
> HDFS-15098.006.patch, HDFS-15098.007.patch, HDFS-15098.008.patch,
> HDFS-15098.009.patch, image-2020-08-19-16-54-41-341.png
>
>
> SM4 (formerly SMS4)is a block cipher used in the Chinese National Standard
> for Wireless LAN WAPI (Wired Authentication and Privacy Infrastructure).
> SM4 was a cipher proposed to for the IEEE 802.11i standard, but has so far
> been rejected by ISO. One of the reasons for the rejection has been
> opposition to the WAPI fast-track proposal by the IEEE. please see:
> [https://en.wikipedia.org/wiki/SM4_(cipher)]
>
> *Use sm4 on hdfs as follows:*
> 1.Configure Hadoop KMS
> 2.test HDFS sm4
> hadoop key create key1 -cipher 'SM4/CTR/NoPadding'
> hdfs dfs -mkdir /benchmarks
> hdfs crypto -createZone -keyName key1 -path /benchmarks
> *requires:*
> 1.openssl version >=1.1.1
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
