Hi, I was just wondering, if it is possible to register multiple versions of the aux-services with YARN as described in the documentation:
1. In the yarn-site.xml on each node, add spark_shuffle to yarn.nodemanager.aux-services, then set yarn.nodemanager.aux-services.spark_shuffle.class to org.apache.spark.network.yarn.YarnShuffleService. Additionally, set all relevantspark.shuffle.service.* configurations <http://spark.apache.org/docs/latest/configuration.html>. The reason for the question is: I am trying to run multiple versions of Spark in parallel. Does anybody have any experience on how such a dual version operation holds up in terms of downward-compatibility? Maybe sticking to the latest version of the aux-service will do the trick? Regards, Andreas [1] http://spark.apache.org/docs/latest/job-scheduling.html#dynamic-resource-allocation
