Github user NiharS commented on the issue:
https://github.com/apache/spark/pull/21885
Chatted with @squito about this. From what I understood from that
discussion, ExternalShuffleService shouldn't be controlled by configurations
passed into a spark application as it is its own independent service that is
separate from the application. The only other usage that immediately stands out
is IndexShuffleBlockResolver, but that only uses the TransportConf for other
conf values, never asking for the core count. Past that, it's used in a Mesos
Backend class, but that seems outside the scope, and in some testing files,
which are safe to leave as is since they don't do anything with the core count.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
