As Sean suggested you can actually build Spark 1.4 for CDH 5.4.x and also include Hive libraries for 0.13.1, but *this will be completely unsupported by Cloudera*. I would suggest to do that only if you just want to experiment with new features from Spark 1.4. I.e. Run SparkSQL with sort-merge join to run expensive BI types of queries over a Hadoop YARN cluster. You just need to keep the build in your user folder across the cluster, then include the Path for the binaries (spark-shell, spark-subimt, pyspark,etc...) in your user profile and make sure you include CDH Classpath to load all dependencies.
On Sun, Jul 12, 2015 at 8:19 PM, Sean Owen <so...@cloudera.com> wrote: > Yeah, it won't technically be supported, and you shouldn't go > modifying the actual installation, but if you just make your own build > of 1.4 for CDH 5.4 and use that build to launch YARN-based apps, I > imagine it will Just Work for most any use case. > > On Sun, Jul 12, 2015 at 7:34 PM, Ruslan Dautkhanov <dautkha...@gmail.com> > wrote: > > Good question. I'd like to know the same. Although I think you'll loose > > supportability. > > > > > > > > -- > > Ruslan Dautkhanov > > > > On Wed, Jul 8, 2015 at 2:03 AM, Ashish Dutt <ashish.du...@gmail.com> > wrote: > >> > >> > >> Hi, > >> I need to upgrade spark version 1.3 to version 1.4 on CDH 5.4. > >> I checked the documentation here but I do not see any thing relevant > >> > >> Any suggestions directing to a solution are welcome. > >> > >> Thanks, > >> Ashish > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
