On Wed, Jul 19, 2017 at 11:19 AM, Udit Mehrotra <udit.mehrotr...@gmail.com> wrote: > spark.network.crypto.saslFallback false > spark.authenticate true > > This seems to work fine with internal shuffle service of Spark. However, > when in I try it with Yarn’s external shuffle service the executors are > unable to register with the shuffle service as it still expects SASL > authentication. Here is the error I get: > > Can someone confirm that this is expected behavior? Or provide some > guidance, on how I can make it work with external shuffle service ?
Yes, that's the expected behavior, since you disabled SASL fallback in your configuration. If you set it back on, then you can talk to the old shuffle service. Or you could upgrade the version of the shuffle service running on your YARN cluster so that it also supports the new auth mechanism. -- Marcelo --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
