[
https://issues.apache.org/jira/browse/HDFS-6606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14048624#comment-14048624
]
Yi Liu commented on HDFS-6606:
------------------------------
Thanks [~tucu00], [~yoderme] and [[email protected]] for your comments.
[~tucu00]:
I file JIRA HADOOP-10768 for Optimizing Hadoop RPC encryption performance. Not
file that JIRA before because 1) Hadoop utilizes SASL {{GSSAPI}} and
{{DIGEST-MD5}} mechanisms for secure authentication and data protection for
RPC, not able to add custom encryption to them. 2) PRC message is small,
whether it is worth.
For #1, you remained me we could only use GssKrb5 to exchange user secrets, not
do encryption for whole RPC message, instead use the same way in this JIRA to
encrypt RPC message. You are right.
For #2, we all think we can have benchmark to see real benefit, then we make a
trade-off.
[[email protected]]:
Thanks for the information, you are right, but it doesn't support AES-NI by
default. Maybe we can handle it in the same way as in this JIRA. It's more
flexiable and can resolve encryption issue of {{DIGEST-MD5}}.
> Optimize encryption support in DataTransfer Protocol with High performance
> --------------------------------------------------------------------------
>
> Key: HDFS-6606
> URL: https://issues.apache.org/jira/browse/HDFS-6606
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, hdfs-client, security
> Affects Versions: 3.0.0
> Reporter: Yi Liu
> Assignee: Yi Liu
> Fix For: 3.0.0
>
>
> In HDFS-3637, [~atm] added support for encrypting the DataTransferProtocol,
> it was a great work.
> It utilizes SASL {{Digest-MD5}} mechanism, it supports three security
> strength:
> * high 3des or rc4 (126bits)
> * medium des or rc4(56bits)
> * low rc4(40bits)
> 3des and rc4 are slow, only *tens of MB/s*,
> http://www.javamex.com/tutorials/cryptography/ciphers.shtml
> http://www.cs.wustl.edu/~jain/cse567-06/ftp/encryption_perf/
> I will give more detailed performance data in future. Absolutely it’s
> bottleneck and will vastly affect the end to end performance.
> AES(Advanced Encryption Standard) is recommended as a replacement of DES,
> it’s more secure; with AES-NI support, the throughput can reach nearly
> *2GB/s*, it won’t be the bottleneck any more, AES and CryptoCodec work is
> supported in HADOOP-10150, HADOOP-10603 and HADOOP-10693 (We may need to add
> a new mode support for AES).
> This JIRA will use AES with AES-NI support as encryption algorithm for
> DataTransferProtocol.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
