[
https://issues.apache.org/jira/browse/HDFS-6606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14186802#comment-14186802
]
Hudson commented on HDFS-6606:
------------------------------
FAILURE: Integrated in Hadoop-trunk-Commit #6367 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/6367/])
HDFS-6606. Optimize HDFS Encrypted Transport performance. (yliu) (yliu: rev
58c0bb9ed9f4a2491395b63c68046562a73526c9)
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/DataTransferSaslUtil.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelper.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/SaslParticipant.java
* hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/SaslDataTransferServer.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/SaslDataTransferClient.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/DNConf.java
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/CipherOption.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/resources/hdfs-default.xml
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/DFSConfigKeys.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/balancer/Dispatcher.java
*
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/CryptoInputStream.java
*
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/sasl/SaslResponseWithNegotiatedCipherOption.java
*
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestEncryptedTransfer.java
* hadoop-hdfs-project/hadoop-hdfs/src/main/proto/hdfs.proto
* hadoop-hdfs-project/hadoop-hdfs/src/main/proto/datatransfer.proto
> Optimize HDFS Encrypted Transport performance
> ---------------------------------------------
>
> Key: HDFS-6606
> URL: https://issues.apache.org/jira/browse/HDFS-6606
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, hdfs-client, security
> Reporter: Yi Liu
> Assignee: Yi Liu
> Attachments: HDFS-6606.001.patch, HDFS-6606.002.patch,
> HDFS-6606.003.patch, HDFS-6606.004.patch, HDFS-6606.005.patch,
> HDFS-6606.006.patch, HDFS-6606.007.patch, HDFS-6606.008.patch,
> HDFS-6606.009.patch, OptimizeHdfsEncryptedTransportperformance.pdf
>
>
> In HDFS-3637, [~atm] added support for encrypting the DataTransferProtocol,
> it was a great work.
> It utilizes SASL {{Digest-MD5}} mechanism (use Qop: auth-conf), it supports
> three security strength:
> * high 3des or rc4 (128bits)
> * medium des or rc4(56bits)
> * low rc4(40bits)
> 3des and rc4 are slow, only *tens of MB/s*,
> http://www.javamex.com/tutorials/cryptography/ciphers.shtml
> http://www.cs.wustl.edu/~jain/cse567-06/ftp/encryption_perf/
> I will give more detailed performance data in future. Absolutely it’s
> bottleneck and will vastly affect the end to end performance.
> AES(Advanced Encryption Standard) is recommended as a replacement of DES,
> it’s more secure; with AES-NI support, the throughput can reach nearly
> *2GB/s*, it won’t be the bottleneck any more, AES and CryptoCodec work is
> supported in HADOOP-10150, HADOOP-10603 and HADOOP-10693 (We may need to add
> a new mode support for AES).
> This JIRA will use AES with AES-NI support as encryption algorithm for
> DataTransferProtocol.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
