vanzin commented on a change in pull request #23348: [SPARK-25857][core] Add developer documentation regarding delegation tokens. URL: https://github.com/apache/spark/pull/23348#discussion_r245393953
########## File path: core/src/main/scala/org/apache/spark/deploy/security/README.md ########## @@ -0,0 +1,239 @@ +# Delegation Token Handling In Spark + +This document aims to explain and demystify delegation tokens as they are used by Spark, since +this topic is generally a huge source of confusion. + + +## What are delegation tokens? + +Delegation tokens (DTs from now on) are authentication tokens used by some services to replace +Kerberos service tokens. Many services in the Hadoop ecosystem have support for DTs, since they +have two very desirable advantages over Kerberos tokens: + +* No need to distribute Kerberos credentials + +In a distributed application, distributing Kerberos credentials is tricky. Not all users have +keytabs, and when they do, it's generally frowned upon to distribute them over the network as +part of application data. + +DTs allow for a single place (e.g. the Spark driver) to require Kerberos credentials. That entity +can then distribute the DTs to other parts of the distributed application (e.g. Spark executors), +so they can authenticate to services. + +* A single token is used for authentication + +If Kerberos authentication were used, each client connection to a server would require a trip +to the Key Distribution Center (KDC) and generation of a service ticket. In a distributed system, Review comment: I'll just revert to KDC. Readers of this document are expected to at least be a little familiar with Kerberos, so those terms should be familiar. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
