Update the docs related Livy build and Spark 2 support Change-Id: Id59c628e73da46a124f718b192fc92336ac8eace
Project: http://git-wip-us.apache.org/repos/asf/incubator-livy/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-livy/commit/97251bdc Tree: http://git-wip-us.apache.org/repos/asf/incubator-livy/tree/97251bdc Diff: http://git-wip-us.apache.org/repos/asf/incubator-livy/diff/97251bdc Branch: refs/heads/revert-237-LIVY-255 Commit: 97251bdcfa9543997d85a06149820a7c794fa8d2 Parents: b8a0839 Author: jerryshao <ss...@hortonworks.com> Authored: Mon Nov 28 15:41:58 2016 +0800 Committer: jerryshao <ss...@hortonworks.com> Committed: Mon Nov 28 15:41:58 2016 +0800 ---------------------------------------------------------------------- README.rst | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/incubator-livy/blob/97251bdc/README.rst ---------------------------------------------------------------------- diff --git a/README.rst b/README.rst index b23136a..1d2d2e5 100644 --- a/README.rst +++ b/README.rst @@ -51,8 +51,14 @@ Required python packages for building Livy: To run Livy, you will also need a Spark installation. You can get Spark releases at -https://spark.apache.org/downloads.html. Livy requires at least Spark 1.6 and currently -only supports Scala 2.10 builds of Spark. +https://spark.apache.org/downloads.html. + +Livy requires at least Spark 1.6 and supports both Scala 2.10 and 2.11 builds of Spark, Livy +will automatically pick repl dependencies through detecting the Scala version of Spark. + +Livy also supports Spark 2.0+ for both interactive and batch submission, you could seamlessly +switch to different versions of Spark through ``SPARK_HOME`` configuration, without needing to +rebuild Livy. Building Livy @@ -66,16 +72,11 @@ Livy is built using `Apache Maven`_. To check out and build Livy, run: cd livy mvn package -By default Livy is built against the CDH 5.5 distribution of Spark (based off Spark 1.5.0). You can -build Livy against a different version of Spark by setting the ``spark.version`` property: - -.. code:: shell - - mvn -Dspark.version=1.6.1 package - -The version of Spark used when running Livy does not need to match the version used to build Livy. -The Livy package itself does not contain a Spark distribution, and will work with any supported -version of Spark. +By default Livy is built against Apache Spark 1.6.2, but the version of Spark used when running +Livy does not need to match the version used to build Livy. Livy internally uses reflection to +mitigate the gaps of different Spark versions, also Livy package itself does not +contain a Spark distribution, so it will work with any supported version of Spark (Spark 1.6+) +without to rebuild against specific version of Spark. .. _Apache Maven: http://maven.apache.org