Repository: bahir-website Updated Branches: refs/heads/master 3d284c480 -> feca55874
Make Spark/Scala versions configurable Project: http://git-wip-us.apache.org/repos/asf/bahir-website/repo Commit: http://git-wip-us.apache.org/repos/asf/bahir-website/commit/9bc561e4 Tree: http://git-wip-us.apache.org/repos/asf/bahir-website/tree/9bc561e4 Diff: http://git-wip-us.apache.org/repos/asf/bahir-website/diff/9bc561e4 Branch: refs/heads/master Commit: 9bc561e47d370349c2cb780c1b15ff5aa0770be8 Parents: 3d284c4 Author: Luciano Resende <[email protected]> Authored: Wed Sep 6 11:05:24 2017 -0700 Committer: Luciano Resende <[email protected]> Committed: Wed Sep 6 13:03:36 2017 -0700 ---------------------------------------------------------------------- site/_config.yml | 7 ++++++- site/docs/spark/current/spark-sql-cloudant.md | 12 ++++++------ site/docs/spark/current/spark-sql-streaming-akka.md | 8 ++++---- site/docs/spark/current/spark-sql-streaming-mqtt.md | 8 ++++---- site/docs/spark/current/spark-streaming-akka.md | 8 ++++---- site/docs/spark/current/spark-streaming-mqtt.md | 8 ++++---- site/docs/spark/current/spark-streaming-pubsub.md | 8 ++++---- site/docs/spark/current/spark-streaming-twitter.md | 8 ++++---- site/docs/spark/current/spark-streaming-zeromq.md | 8 ++++---- 9 files changed, 40 insertions(+), 35 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/bahir-website/blob/9bc561e4/site/_config.yml ---------------------------------------------------------------------- diff --git a/site/_config.yml b/site/_config.yml index f4daf91..bf96c09 100644 --- a/site/_config.yml +++ b/site/_config.yml @@ -38,4 +38,9 @@ JB: provider: google_universal google_universal: domain: bahir.apache.org - tracking_id: UA-79140859-1 \ No newline at end of file + tracking_id: UA-79140859-1 + +# These allow the documentation to be updated with newer releases +SPARK_VERSION: 2.2.0.1.0.0-SNAPSHOT +SCALA_BINARY_VERSION: "2.11" +SCALA_VERSION: "2.11.8" http://git-wip-us.apache.org/repos/asf/bahir-website/blob/9bc561e4/site/docs/spark/current/spark-sql-cloudant.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/current/spark-sql-cloudant.md b/site/docs/spark/current/spark-sql-cloudant.md index 1065f29..b0f9c76 100644 --- a/site/docs/spark/current/spark-sql-cloudant.md +++ b/site/docs/spark/current/spark-sql-cloudant.md @@ -38,19 +38,19 @@ clusters, desktop PCs, and mobile devices. Using SBT: - libraryDependencies += "org.apache.bahir" %% "spark-sql-cloudant" % "2.2.0-SNAPSHOT" + libraryDependencies += "org.apache.bahir" %% "spark-sql-cloudant" % "{{site.SPARK_VERSION}}" Using Maven: <dependency> <groupId>org.apache.bahir</groupId> - <artifactId>spark-sql-cloudant_2.11</artifactId> - <version>2.2.0-SNAPSHOT</version> + <artifactId>spark-sql-cloudant_{{site.SCALA_BINARY_VERSION}}</artifactId> + <version>{{site.SPARK_VERSION}}</version> </dependency> This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. - $ bin/spark-shell --packages org.apache.bahir:spark-sql-cloudant_2.11:2.2.0-SNAPSHOT + $ bin/spark-shell --packages org.apache.bahir:spark-sql-cloudant_{{site.SCALA_BINARY_VERSION}}:{{site.SPARK_VERSION}} Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. The `--packages` argument can also be used with `bin/spark-submit`. @@ -164,7 +164,7 @@ See [CloudantApp.py](examples/python/CloudantApp.py) for examples. Submit job example: ``` -spark-submit --packages org.apache.bahir:spark-sql-cloudant_2.11:2.2.0-SNAPSHOT --conf spark.cloudant.host=ACCOUNT.cloudant.com --conf spark.cloudant.username=USERNAME --conf spark.cloudant.password=PASSWORD sql-cloudant/examples/python/CloudantApp.py +spark-submit --packages org.apache.bahir:spark-sql-cloudant_{{site.SCALA_BINARY_VERSION}}:{{site.SPARK_VERSION}} --conf spark.cloudant.host=ACCOUNT.cloudant.com --conf spark.cloudant.username=USERNAME --conf spark.cloudant.password=PASSWORD sql-cloudant/examples/python/CloudantApp.py ``` #### Using DataFrame In Python @@ -234,7 +234,7 @@ See [CloudantApp.scala](examples/scala/src/main/scala/mytest/spark/CloudantApp.s Submit job example: ``` -spark-submit --class org.apache.spark.examples.sql.cloudant.CloudantApp --packages org.apache.bahir:spark-sql-cloudant_2.11:2.2.0-SNAPSHOT --conf spark.cloudant.host=ACCOUNT.cloudant.com --conf spark.cloudant.username=USERNAME --conf spark.cloudant.password=PASSWORD /path/to/spark-sql-cloudant_2.11-2.2.0-SNAPSHOT-tests.jar +spark-submit --class org.apache.spark.examples.sql.cloudant.CloudantApp --packages org.apache.bahir:spark-sql-cloudant_{{site.SCALA_BINARY_VERSION}}:{{site.SPARK_VERSION}} --conf spark.cloudant.host=ACCOUNT.cloudant.com --conf spark.cloudant.username=USERNAME --conf spark.cloudant.password=PASSWORD /path/to/spark-sql-cloudant_{{site.SCALA_BINARY_VERSION}}-{{site.SPARK_VERSION}}-tests.jar ``` ### Using DataFrame In Scala http://git-wip-us.apache.org/repos/asf/bahir-website/blob/9bc561e4/site/docs/spark/current/spark-sql-streaming-akka.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/current/spark-sql-streaming-akka.md b/site/docs/spark/current/spark-sql-streaming-akka.md index 40ad088..459c3f6 100644 --- a/site/docs/spark/current/spark-sql-streaming-akka.md +++ b/site/docs/spark/current/spark-sql-streaming-akka.md @@ -30,20 +30,20 @@ A library for reading data from Akka Actors using Spark SQL Streaming ( or Struc Using SBT: - libraryDependencies += "org.apache.bahir" %% "spark-sql-streaming-akka" % "2.2.0-SNAPSHOT" + libraryDependencies += "org.apache.bahir" %% "spark-sql-streaming-akka" % "{{site.SPARK_VERSION}}" Using Maven: <dependency> <groupId>org.apache.bahir</groupId> - <artifactId>spark-sql-streaming-akka_2.11</artifactId> - <version>2.2.0-SNAPSHOT</version> + <artifactId>spark-sql-streaming-akka_{{site.SCALA_BINARY_VERSION}}</artifactId> + <version>{{site.SPARK_VERSION}}</version> </dependency> This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. For example, to include it when starting the spark shell: - $ bin/spark-shell --packages org.apache.bahir:spark-sql-streaming-akka_2.11:2.2.0-SNAPSHOT + $ bin/spark-shell --packages org.apache.bahir:spark-sql-streaming-akka_{{site.SCALA_BINARY_VERSION}}:{{site.SPARK_VERSION}} Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. The `--packages` argument can also be used with `bin/spark-submit`. http://git-wip-us.apache.org/repos/asf/bahir-website/blob/9bc561e4/site/docs/spark/current/spark-sql-streaming-mqtt.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/current/spark-sql-streaming-mqtt.md b/site/docs/spark/current/spark-sql-streaming-mqtt.md index 83d9c82..fa533d1 100644 --- a/site/docs/spark/current/spark-sql-streaming-mqtt.md +++ b/site/docs/spark/current/spark-sql-streaming-mqtt.md @@ -31,20 +31,20 @@ A library for reading data from MQTT Servers using Spark SQL Streaming ( or Stru Using SBT: - libraryDependencies += "org.apache.bahir" %% "spark-sql-streaming-mqtt" % "2.1.0-SNAPSHOT" + libraryDependencies += "org.apache.bahir" %% "spark-sql-streaming-mqtt" % "{{site.SPARK_VERSION}}" Using Maven: <dependency> <groupId>org.apache.bahir</groupId> - <artifactId>spark-sql-streaming-mqtt_2.11</artifactId> - <version>2.2.0-SNAPSHOT</version> + <artifactId>spark-sql-streaming-mqtt_{{site.SCALA_BINARY_VERSION}}</artifactId> + <version>{{site.SPARK_VERSION}}</version> </dependency> This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. For example, to include it when starting the spark shell: - $ bin/spark-shell --packages org.apache.bahir:spark-sql-streaming-mqtt_2.11:2.1.0-SNAPSHOT + $ bin/spark-shell --packages org.apache.bahir:spark-sql-streaming-mqtt_{{site.SCALA_BINARY_VERSION}}:{{site.SPARK_VERSION}} Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. The `--packages` argument can also be used with `bin/spark-submit`. http://git-wip-us.apache.org/repos/asf/bahir-website/blob/9bc561e4/site/docs/spark/current/spark-streaming-akka.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/current/spark-streaming-akka.md b/site/docs/spark/current/spark-streaming-akka.md index 17a91bf..41cc57e 100644 --- a/site/docs/spark/current/spark-streaming-akka.md +++ b/site/docs/spark/current/spark-streaming-akka.md @@ -31,20 +31,20 @@ A library for reading data from Akka Actors using Spark Streaming. Using SBT: - libraryDependencies += "org.apache.bahir" %% "spark-streaming-akka" % "2.1.0-SNAPSHOT" + libraryDependencies += "org.apache.bahir" %% "spark-streaming-akka" % "{{site.SPARK_VERSION}}" Using Maven: <dependency> <groupId>org.apache.bahir</groupId> - <artifactId>spark-streaming-akka_2.11</artifactId> - <version>2.2.0-SNAPSHOT</version> + <artifactId>spark-streaming-akka_{{site.SCALA_BINARY_VERSION}}</artifactId> + <version>{{site.SPARK_VERSION}}</version> </dependency> This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. For example, to include it when starting the spark shell: - $ bin/spark-shell --packages org.apache.bahir:spark-streaming-akka_2.11:2.1.0-SNAPSHOT + $ bin/spark-shell --packages org.apache.bahir:spark-streaming-akka_{{site.SCALA_BINARY_VERSION}}:{{site.SPARK_VERSION}} Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. The `--packages` argument can also be used with `bin/spark-submit`. http://git-wip-us.apache.org/repos/asf/bahir-website/blob/9bc561e4/site/docs/spark/current/spark-streaming-mqtt.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/current/spark-streaming-mqtt.md b/site/docs/spark/current/spark-streaming-mqtt.md index 14b072b..42b8704 100644 --- a/site/docs/spark/current/spark-streaming-mqtt.md +++ b/site/docs/spark/current/spark-streaming-mqtt.md @@ -32,20 +32,20 @@ limitations under the License. Using SBT: - libraryDependencies += "org.apache.bahir" %% "spark-streaming-mqtt" % "2.1.0-SNAPSHOT" + libraryDependencies += "org.apache.bahir" %% "spark-streaming-mqtt" % "{{site.SPARK_VERSION}}" Using Maven: <dependency> <groupId>org.apache.bahir</groupId> - <artifactId>spark-streaming-mqtt_2.11</artifactId> - <version>2.2.0-SNAPSHOT</version> + <artifactId>spark-streaming-mqtt_{{site.SCALA_BINARY_VERSION}}</artifactId> + <version>{{site.SPARK_VERSION}}</version> </dependency> This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. For example, to include it when starting the spark shell: - $ bin/spark-shell --packages org.apache.bahir:spark-streaming-mqtt_2.11:2.1.0-SNAPSHOT + $ bin/spark-shell --packages org.apache.bahir:spark-streaming-mqtt_{{site.SCALA_BINARY_VERSION}}:{{site.SPARK_VERSION}} Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. The `--packages` argument can also be used with `bin/spark-submit`. http://git-wip-us.apache.org/repos/asf/bahir-website/blob/9bc561e4/site/docs/spark/current/spark-streaming-pubsub.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/current/spark-streaming-pubsub.md b/site/docs/spark/current/spark-streaming-pubsub.md index 6e36a60..a448c94 100644 --- a/site/docs/spark/current/spark-streaming-pubsub.md +++ b/site/docs/spark/current/spark-streaming-pubsub.md @@ -30,20 +30,20 @@ A library for reading data from [Google Cloud Pub/Sub](https://cloud.google.com/ Using SBT: - libraryDependencies += "org.apache.bahir" %% "spark-streaming-pubsub" % "2.2.0-SNAPSHOT" + libraryDependencies += "org.apache.bahir" %% "spark-streaming-pubsub" % "{{site.SPARK_VERSION}}" Using Maven: <dependency> <groupId>org.apache.bahir</groupId> - <artifactId>spark-streaming-pubsub_2.11</artifactId> - <version>2.2.0-SNAPSHOT</version> + <artifactId>spark-streaming-pubsub_{{site.SCALA_BINARY_VERSION}}</artifactId> + <version>{{site.SPARK_VERSION}}</version> </dependency> This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. For example, to include it when starting the spark shell: - $ bin/spark-shell --packages org.apache.bahir:spark-streaming-pubsub_2.11:2.2.0-SNAPSHOT + $ bin/spark-shell --packages org.apache.bahir:spark-streaming-pubsub_{{site.SCALA_BINARY_VERSION}}:{{site.SPARK_VERSION}} Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. The `--packages` argument can also be used with `bin/spark-submit`. http://git-wip-us.apache.org/repos/asf/bahir-website/blob/9bc561e4/site/docs/spark/current/spark-streaming-twitter.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/current/spark-streaming-twitter.md b/site/docs/spark/current/spark-streaming-twitter.md index 74029af..616fd2a 100644 --- a/site/docs/spark/current/spark-streaming-twitter.md +++ b/site/docs/spark/current/spark-streaming-twitter.md @@ -31,20 +31,20 @@ A library for reading social data from [twitter](http://twitter.com/) using Spar Using SBT: - libraryDependencies += "org.apache.bahir" %% "spark-streaming-twitter" % "2.1.0-SNAPSHOT" + libraryDependencies += "org.apache.bahir" %% "spark-streaming-twitter" % "{{site.SPARK_VERSION}}" Using Maven: <dependency> <groupId>org.apache.bahir</groupId> - <artifactId>spark-streaming-twitter_2.11</artifactId> - <version>2.2.0-SNAPSHOT</version> + <artifactId>spark-streaming-twitter_{{site.SCALA_BINARY_VERSION}}</artifactId> + <version>{{site.SPARK_VERSION}}</version> </dependency> This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. For example, to include it when starting the spark shell: - $ bin/spark-shell --packages org.apache.bahir:spark-streaming-twitter_2.11:2.1.0-SNAPSHOT + $ bin/spark-shell --packages org.apache.bahir:spark-streaming-twitter_{{site.SCALA_BINARY_VERSION}}:{{site.SPARK_VERSION}} Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. The `--packages` argument can also be used with `bin/spark-submit`. http://git-wip-us.apache.org/repos/asf/bahir-website/blob/9bc561e4/site/docs/spark/current/spark-streaming-zeromq.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/current/spark-streaming-zeromq.md b/site/docs/spark/current/spark-streaming-zeromq.md index 828aa1c..4543d90 100644 --- a/site/docs/spark/current/spark-streaming-zeromq.md +++ b/site/docs/spark/current/spark-streaming-zeromq.md @@ -31,20 +31,20 @@ A library for reading data from [ZeroMQ](http://zeromq.org/) using Spark Streami Using SBT: - libraryDependencies += "org.apache.bahir" %% "spark-streaming-zeromq" % "2.1.0-SNAPSHOT" + libraryDependencies += "org.apache.bahir" %% "spark-streaming-zeromq" % "{{site.SPARK_VERSION}}" Using Maven: <dependency> <groupId>org.apache.bahir</groupId> - <artifactId>spark-streaming-zeromq_2.11</artifactId> - <version>2.2.0-SNAPSHOT</version> + <artifactId>spark-streaming-zeromq_{{site.SCALA_BINARY_VERSION}}</artifactId> + <version>{{site.SPARK_VERSION}}</version> </dependency> This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. For example, to include it when starting the spark shell: - $ bin/spark-shell --packages org.apache.bahir:spark-streaming-zeromq_2.11:2.1.0-SNAPSHOT + $ bin/spark-shell --packages org.apache.bahir:spark-streaming-zeromq_{{site.SCALA_BINARY_VERSION}}:{{site.SPARK_VERSION}} Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. The `--packages` argument can also be used with `bin/spark-submit`.
