[
https://issues.apache.org/jira/browse/HDFS-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13988389#comment-13988389
]
Chris Nauroth commented on HDFS-2856:
-------------------------------------
I think we can achieve compatibility on the 2.x line by having the client
decide the correct protocol. The client can make this decision based on
observing a few things in its runtime environment:
# Datanode address port - We know that existing secured data nodes are on a
privileged port, and future secured data nodes that don't start as root will be
on a non-privileged port.
# {{dfs.data.transfer.protection}} - I propose adding this as a new
configuration property for setting the desired SASL QOP on
{{DataTransferProtocol}}. Its values would have the same syntax as the
existing {{hadoop.rpc.protection}} property.
# {{dfs.encrypt.data.transfer}} - We must maintain the existing behavior for
deployments that have turned this on. In addition to using SASL with the
auth-conf QOP, this property also requires use of an NN-issued encryption key
and imposes strict enforcement that all connections must be encrypted.
Effectively, this property must supersede {{dfs.data.transfer.protection}} and
cause rejection of SASL attempts that use any QOP other than auth-conf.
Using that information, pseudo-code for protocol selection in the client would
be:
{code}
if security is on
if datanode port < 1024
if dfs.encrypt.data.transfer is on
use encrypted SASL handshake (HDFS-3637)
else
do not use SASL
else
if dfs.encrypt.data.transfer is on
use encrypted SASL handshake (HDFS-3637)
else if dfs.data.transfer.protection defined
use general SASL handshake (HDFS-2856)
else
error - secured connection on non-privileged port without SASL not
possible
else
do not use SASL
{code}
>From an upgrade perspective, existing deployments that don't mind sticking
>with a privileged port can just keep running as usual, because the protocol
>would keep working the same way it works today. For existing deployments that
>want to stop using a privileged port and switch to a non-privileged port, it's
>more complex. First, they'll need to deploy the code update everywhere.
>Then, they'll need to restart datanodes to pick up 2 configuration changes
>simultaneously: 1) switch the port number and 2) set
>{{dfs.data.transfer.protection}}. While this is happening, you could have a
>mix of datanodes in the cluster running in different modes: some with a
>privileged port and some with a non-privileged port. This is OK, because the
>client-side logic above knows how to negotiate the correct protocol on a
>per-DN basis.
One thing that would be impossible under this scheme is using a privileged port
in combination with the new SASL handshake. The whole motivation for this
change is to prevent the need for root access though, so I think this is an
acceptable limitation.
The most recent version of the design document talks about upgrading the
{{DATA_TRANSFER_VERSION}}. I now believe this isn't necessary. Old clients
can keep using the existing protocol version. New clients can trigger the new
behavior based on {{dfs.data.transfer.protection}}, so a new protocol version
isn't necessary. I need to refresh the design doc.
I believe all of the above fits into our compatibility policies.
> Fix block protocol so that Datanodes don't require root or jsvc
> ---------------------------------------------------------------
>
> Key: HDFS-2856
> URL: https://issues.apache.org/jira/browse/HDFS-2856
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode, security
> Reporter: Owen O'Malley
> Assignee: Chris Nauroth
> Attachments: Datanode-Security-Design.pdf,
> Datanode-Security-Design.pdf, Datanode-Security-Design.pdf,
> HDFS-2856.prototype.patch
>
>
> Since we send the block tokens unencrypted to the datanode, we currently
> start the datanode as root using jsvc and get a secure (< 1024) port.
> If we have the datanode generate a nonce and send it on the connection and
> the sends an hmac of the nonce back instead of the block token it won't
> reveal any secrets. Thus, we wouldn't require a secure port and would not
> require root or jsvc.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
