Hi Krishna, - Take a lokk at *http://hortonworks.com/hadoop-tutorial/apache-spark-1-4-1-technical-preview-with-hdp/ <http://hortonworks.com/hadoop-tutorial/apache-spark-1-4-1-technical-preview-with-hdp/>* - Or you can specify your 1.5.x jar as the Spark one using something like :
--conf "spark.yarn.jar=hdfs://master:8020/spark-assembly-1.5.0-hadoop2.6.0.jar" The main drawback is : *Known Issues* *Spark YARN ATS integration does not work in this tech preview. You will not see the history of Spark jobs in the Jobs server after a job is finished.* 2015-09-23 1:31 GMT+02:00 Zhan Zhang <zzh...@hortonworks.com>: > Hi Krishna, > > For the time being, you can download from upstream, and it should be > running OK for HDP2.3. For hdp specific problem, you can ask in > Hortonworks forum. > > Thanks. > > Zhan Zhang > > On Sep 22, 2015, at 3:42 PM, Krishna Sankar <ksanka...@gmail.com> wrote: > > Guys, > > - We have HDP 2.3 installed just now. It comes with Spark 1.3.x. The > current wisdom is that it will support the 1.4.x train (which is good, need > DataFrame et al). > - What is the plan to support Spark 1.5.x ? Can we install 1.5.0 on > HDP 2.3 ? Or will Spark 1.5.x support be in HDP 2.3.x and if so ~when ? > > Cheers & Thanks > <k/> > > >
