I have the same question about the history server. We are trying to run multiple versions of Spark and are wondering if the history server is backwards compatible.
*Alex Rovner* *Director, Data Engineering * *o:* 646.759.0052 * <http://www.magnetic.com/>* On Mon, Oct 5, 2015 at 9:22 AM, Andreas Fritzler <andreas.fritz...@gmail.com > wrote: > Hi, > > I was just wondering, if it is possible to register multiple versions of > the aux-services with YARN as described in the documentation: > > > > 1. In the yarn-site.xml on each node, add spark_shuffle to > yarn.nodemanager.aux-services, then set > yarn.nodemanager.aux-services.spark_shuffle.class to > org.apache.spark.network.yarn.YarnShuffleService. Additionally, set > all relevantspark.shuffle.service.* configurations > <http://spark.apache.org/docs/latest/configuration.html>. > > The reason for the question is: I am trying to run multiple versions of > Spark in parallel. Does anybody have any experience on how such a dual > version operation holds up in terms of downward-compatibility? > > Maybe sticking to the latest version of the aux-service will do the trick? > > Regards, > Andreas > > [1] > http://spark.apache.org/docs/latest/job-scheduling.html#dynamic-resource-allocation > >
