Thanks Guys. Yep, now I would install 1.5.1 over HDP 2.3, if that works. Cheers <k/>
On Mon, Sep 28, 2015 at 9:47 AM, Ted Yu <yuzhih...@gmail.com> wrote: > Krishna: > If you want to query ORC files, see the following JIRA: > > [SPARK-10623] [SQL] Fixes ORC predicate push-down > > which is in the 1.5.1. release. > > FYI > > On Mon, Sep 28, 2015 at 9:42 AM, Fabien Martin <fabien.marti...@gmail.com> > wrote: > >> Hi Krishna, >> >> - Take a lokk at >> *http://hortonworks.com/hadoop-tutorial/apache-spark-1-4-1-technical-preview-with-hdp/ >> >> <http://hortonworks.com/hadoop-tutorial/apache-spark-1-4-1-technical-preview-with-hdp/>* >> - Or you can specify your 1.5.x jar as the Spark one using something >> like : >> >> --conf >> "spark.yarn.jar=hdfs://master:8020/spark-assembly-1.5.0-hadoop2.6.0.jar" >> >> The main drawback is : >> >> *Known Issues* >> >> *Spark YARN ATS integration does not work in this tech preview. You will >> not see the history of Spark jobs in the Jobs server after a job is >> finished.* >> >> 2015-09-23 1:31 GMT+02:00 Zhan Zhang <zzh...@hortonworks.com>: >> >>> Hi Krishna, >>> >>> For the time being, you can download from upstream, and it should be >>> running OK for HDP2.3. For hdp specific problem, you can ask in >>> Hortonworks forum. >>> >>> Thanks. >>> >>> Zhan Zhang >>> >>> On Sep 22, 2015, at 3:42 PM, Krishna Sankar <ksanka...@gmail.com> wrote: >>> >>> Guys, >>> >>> - We have HDP 2.3 installed just now. It comes with Spark 1.3.x. The >>> current wisdom is that it will support the 1.4.x train (which is good, >>> need >>> DataFrame et al). >>> - What is the plan to support Spark 1.5.x ? Can we install 1.5.0 on >>> HDP 2.3 ? Or will Spark 1.5.x support be in HDP 2.3.x and if so ~when ? >>> >>> Cheers & Thanks >>> <k/> >>> >>> >>> >> >
