[
https://issues.apache.org/jira/browse/SPARK-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376147#comment-14376147
]
Marcelo Vanzin commented on SPARK-6229:
---------------------------------------
[~adav] the problem of exposing the pipeline as an API is twofold:
* Every application needs to understand the internals of the pipeline. For
example, in your "EncryptionHandler" suggestion, SSL and SASL benefit from
being placed in different locations inside the pipeline. How do you expose that
in an external API? And why make client code even care about that? Also, I
don't necessarily agree that SSL is an application concern. It's a
transport-level protocol - which is one of the reasons it would be placed in a
separate place in the stack from a SASL handler, for example..
* Handling SASL and SSL inside the network library does not necessarily make it
any less unit-testable, stable or fast. It just makes it easier for clients to
use those things. Instead of writing a bunch of code that needs to be
synchronized between client and server, all they need is a proper configuration
object. Configuration (= data) is much easier to change and fix than code.
The SecurityManager issue is already solved in the transport library. When I
moved the SASL code to network/common I moved {{SecretKeyHolder}} with it. So
there you have, your application-agnostic interface for providing security
secrets for the network library.
So, again, what I'm suggesting here is not to hardcode SSL and SASL into the
library. I'm suggesting an easier interface for people to configure SSL and
SASL that doesn't require writing any extra code. If they don't want either of
those, they still have that option, but instead of deleting / disabling /
conditioning code, they'd change a couple of lines in a config file. They'd get
the same stable, fast network library without SSL nor SASL, without having to
change a single line of code.
Another problem with your example ({{transport.setEncryptionHandler}} vs. {{if
(sasl) ...}}) is that the latter would be needed anyway if you want SASL. Why
not then have also an "AuthenticationHandler" aside from the encryption handler?
As for your factory comment, that's already there, in a way. The "bootstrap"
functionality is basically a way to insert things into the channel being
instantiated. What I'm proposing here is twofold: first, extend that interface
so that the bootstrap implementation can modify the pipeline (and also allow
"server" bootstraps, for reasons I explained in my first long comment), and
second, control which bootstraps get activated via configuration, not via code.
Note that, internally within the library, you'd have basically what you're
saying: SSL and SASL would be "plugins" of sort that you can insert into the
transport layer only if you want. The different that I'm trying to convey here
is that the *external* interface of the library doesn't expose those.
> Support SASL encryption in network/common module
> ------------------------------------------------
>
> Key: SPARK-6229
> URL: https://issues.apache.org/jira/browse/SPARK-6229
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core
> Reporter: Marcelo Vanzin
>
> After SASL support has been added to network/common, supporting encryption
> should be rather simple. Encryption is supported for DIGEST-MD5 and GSSAPI.
> Since the latter requires a valid kerberos login to work (and so doesn't
> really work with executors), encryption would require the use of DIGEST-MD5.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
