Hi, As per requirement I need to use Spark 1.4.1.But HDP doesnt comes with Spark 1.4.1 version. As instructed in this hortonworks page <http://hortonworks.com/hadoop-tutorial/apache-spark-1-4-1-technical-preview-with-hdp/> I am able to set up Spark 1.4 in HDP ,but when I run the spark shell It shows Spark 1.3.1 REPL instead of spark 1.4.1 . Do I need to make any configuration changes apart from instructions given in above mentioned page. How do I set spark 1.4.1 as the default spark engine. Options : 1.Do I need to remove the current spark 1.3.1 dir. 2. I named my spark installation dir names spark 1.4.1,Do I need to rename as Spark. 3.Is there configuration needs to change in HDP to set spark 1.4.1 as default spark engine .
Would really appreciate your help. Thanks, Regards,
