[
https://issues.apache.org/jira/browse/HDFS-13541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16491147#comment-16491147
]
Benoy Antony commented on HDFS-13541:
-------------------------------------
{quote}
in our environment, cross data center traffic actually compose a small
fraction of all traffic, having additional DataXceiverServer thread sitting and
listening on every single data node, but being idle most of time does not seem
to be ideal.
{quote}
If its a small fraction, then one option would be to use (1) swebhdfs or (2)
HTTPFS gateway which uses RPC. They work seamlessly for clients via the
FileSystem implementation and swebhdfs scheme.
(1) introduces HTTP protocol overhead and lack of QOS on namenode.
(2) introduces HTTP protocol overhead and an additional hop.
But these are the common solutions to this problem for small fractions of data
which needs to be treated differently. But your use case may be different
which warrants secure and non secure RPC and data transfer.
> NameNode Port based selective encryption
> ----------------------------------------
>
> Key: HDFS-13541
> URL: https://issues.apache.org/jira/browse/HDFS-13541
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, namenode, security
> Reporter: Chen Liang
> Assignee: Chen Liang
> Priority: Major
> Attachments: NameNode Port based selective encryption-v1.pdf
>
>
> Here at LinkedIn, one issue we face is that we need to enforce different
> security requirement based on the location of client and the cluster.
> Specifically, for clients from outside of the data center, it is required by
> regulation that all traffic must be encrypted. But for clients within the
> same data center, unencrypted connections are more desired to avoid the high
> encryption overhead.
> HADOOP-10221 introduced pluggable SASL resolver, based on which HADOOP-10335
> introduced WhitelistBasedResolver which solves the same problem. However we
> found it difficult to fit into our environment for several reasons. In this
> JIRA, on top of pluggable SASL resolver, *we propose a different approach of
> running RPC two ports on NameNode, and the two ports will be enforcing
> encrypted and unencrypted connections respectively, and the following
> DataNode access will simply follow the same behaviour of
> encryption/unencryption*. Then by blocking unencrypted port on datacenter
> firewall, we can completely block unencrypted external access.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
