Spark 2.4.x does not require scala 2.12, in fact is is marked as “experimental” here: https://spark.apache.org/releases/spark-release-2-4-0.html
<https://spark.apache.org/releases/spark-release-2-4-0.html> Moving to a new scala version is often a pain, because the libs you use may not be upgraded and version matter *unlike typical Java updates). Scala creates JVM objects and names them as it pleases. Sometimes naming changes from version to version of Scala and this causes big problem in using mixed libs from different versions of Scala. I’m no expert in Livy, but imagine you may need to build against a newer Spark. But avoid Scala 2.12 for now. From: santosh.dan...@ubs.com <santosh.dan...@ubs.com> <santosh.dan...@ubs.com> Reply: user@livy.incubator.apache.org <user@livy.incubator.apache.org> <user@livy.incubator.apache.org> Date: June 3, 2019 at 12:51:20 PM To: user@livy.incubator.apache.org <user@livy.incubator.apache.org> <user@livy.incubator.apache.org> Subject: Support for Livy with Scala 2.12 Hi, We have just upgraded our spark cluster version 2.3 to 2.4.2 and it broke Livy. It's throwing exception "Cannot Find Livy REPL Jars". Looks like I have to build Livy using Scala 2.12 version. Can anyone advise how to build Livy with Scala 2.12 with Maven? Will changing the scala version from 2.11 to 2.12 would build livy? Please advise. The code failed because of a fatal error: Invalid status code '400' from http://localhost:8998/sessions with error payload: {"msg":"requirement failed: Cannot find Livy REPL jars."}. Thanks Santosh Please visit our website at http://financialservicesinc.ubs.com/wealth/E-maildisclaimer.html for important disclosures and information about our e-mail policies. For your protection, please do not transmit orders or instructions by e-mail or include account numbers, Social Security numbers, credit card numbers, passwords, or other personal information.
