[
https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14357838#comment-14357838
]
Marcelo Vanzin commented on SPARK-6229:
---------------------------------------
/cc [~rxin] [~adav]
Hey Reynold, Aaron,
I started looking at this and it seems like there's no easy way to insert SASL
encryption into the current channel pipeline due to the way the code is
organized.
On the client side, {{TransportClientFactory.createClient}} seems like the best
candidate, but it would have to be changed so that {{SaslClientBootstrap}} can
add a handler to the channel pipeline to do the encryption.
On the server side, it seems more complicated. I couldn't find a place where
the current SASL code has access to the channel, so it can't really set up a
handler at the same layer as the client code. At first I thought I could just
encrypt things in {{SaslRpcHandler}}, but that looks fishy since that handler
is before all the framing and serialization handlers, and we kinda want to
encrypt after those.
So I'd like to suggest a bigger change that I think would make the code easier
to use and at the same time allow this change: internalize all SASL handling in
the network library.
Basically, let the network library decide whether to add SASL to the picture by
looking at the TransportConf; code using the library wouldn't need to care
about it anymore (aside from providing a TransportConf and a SecretKeyHolder).
So, instead of code like this (from StandaloneWorkerShuffleService):
{code}
private val transportContext: TransportContext = {
val handler = if (useSasl) new SaslRpcHandler(blockHandler,
securityManager) else blockHandler
new TransportContext(transportConf, handler)
}
{code}
You'd have just:
{code}
private val transportContext: TransportContext = new
TransportContext(transportConf, securityManager, blockHandler)
{code}
And similarly in other places. What do you guys think?
BTW here's some code that does this and is totally transparent to the code
creating the RPC server / client:
https://github.com/apache/hive/blob/trunk/spark-client/src/main/java/org/apache/hive/spark/client/rpc/SaslHandler.java
And the two implementations:
https://github.com/apache/hive/blob/trunk/spark-client/src/main/java/org/apache/hive/spark/client/rpc/RpcServer.java#L197
https://github.com/apache/hive/blob/trunk/spark-client/src/main/java/org/apache/hive/spark/client/rpc/Rpc.java#L390
That's just to give an idea of how it could be done internally in the network
library, without the consumers of the library having to care about it.
But please let me know if I missed something obvious here.
> Support encryption in network/common module
> -------------------------------------------
>
> Key: SPARK-6229
> URL: https://issues.apache.org/jira/browse/SPARK-6229
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core
> Reporter: Marcelo Vanzin
>
> After SASL support has been added to network/common, supporting encryption
> should be rather simple. Encryption is supported for DIGEST-MD5 and GSSAPI.
> Since the latter requires a valid kerberos login to work (and so doesn't
> really work with executors), encryption would require the use of DIGEST-MD5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
