Repository: bahir-website Updated Branches: refs/heads/master 92ca3c99e -> 0d53ccdd4
Update website for Bahir release 2.1.0 - add release notes for version 2.1.0 - add doc for 2.1.0, update current, add link in navbar - correct page title and description for release note pages Project: http://git-wip-us.apache.org/repos/asf/bahir-website/repo Commit: http://git-wip-us.apache.org/repos/asf/bahir-website/commit/0d53ccdd Tree: http://git-wip-us.apache.org/repos/asf/bahir-website/tree/0d53ccdd Diff: http://git-wip-us.apache.org/repos/asf/bahir-website/diff/0d53ccdd Branch: refs/heads/master Commit: 0d53ccdd4d30507741a1ae45b51f876ef9b8e915 Parents: 92ca3c9 Author: Christian Kadner <[email protected]> Authored: Wed Feb 22 03:35:57 2017 -0800 Committer: Christian Kadner <[email protected]> Committed: Wed Feb 22 03:35:57 2017 -0800 ---------------------------------------------------------------------- site/_data/navigation.yml | 2 + site/_data/project.yml | 4 +- site/_data/releases.yml | 3 + site/docs/spark/2.1.0/documentation.md | 46 ++++++ .../spark/2.1.0/spark-sql-streaming-mqtt.md | 147 +++++++++++++++++++ site/docs/spark/2.1.0/spark-streaming-akka.md | 89 +++++++++++ site/docs/spark/2.1.0/spark-streaming-mqtt.md | 94 ++++++++++++ .../docs/spark/2.1.0/spark-streaming-twitter.md | 74 ++++++++++ site/docs/spark/2.1.0/spark-streaming-zeromq.md | 65 ++++++++ .../spark/current/spark-sql-streaming-mqtt.md | 6 +- site/docs/spark/current/spark-streaming-akka.md | 6 +- site/docs/spark/current/spark-streaming-mqtt.md | 6 +- .../spark/current/spark-streaming-twitter.md | 6 +- .../spark/current/spark-streaming-zeromq.md | 6 +- site/releases/2.0.0/release-notes.md | 6 +- site/releases/2.0.1/release-notes.md | 6 +- site/releases/2.0.2/release-notes.md | 6 +- site/releases/2.1.0/release-notes.md | 36 +++++ 18 files changed, 582 insertions(+), 26 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/_data/navigation.yml ---------------------------------------------------------------------- diff --git a/site/_data/navigation.yml b/site/_data/navigation.yml index 98376ce..b8a01b2 100644 --- a/site/_data/navigation.yml +++ b/site/_data/navigation.yml @@ -52,6 +52,8 @@ topnav: subcategories: - title: Bahir Spark Extensions - Current url: /docs/spark/current/documentation + - title: Bahir Spark Extensions - 2.1.0 + url: /docs/spark/2.1.0/documentation - title: Bahir Spark Extensions - 2.0.2 url: /docs/spark/2.0.2/documentation - title: Bahir Spark Extensions - 2.0.1 http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/_data/project.yml ---------------------------------------------------------------------- diff --git a/site/_data/project.yml b/site/_data/project.yml index 9618694..33454a2 100644 --- a/site/_data/project.yml +++ b/site/_data/project.yml @@ -22,8 +22,8 @@ github_project_name: bahir description: Apache Bahir provides extensions to distributed analytic platforms such as Apache Spark. download: /download -latest_release: 2.0.2 -latest_release_date: 01/27/2017 +latest_release: 2.1.0 +latest_release_date: 02/22/2017 dev_list: [email protected] dev_list_subscribe: [email protected] http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/_data/releases.yml ---------------------------------------------------------------------- diff --git a/site/_data/releases.yml b/site/_data/releases.yml index d539950..1858729 100644 --- a/site/_data/releases.yml +++ b/site/_data/releases.yml @@ -14,6 +14,9 @@ # limitations under the License. # +- version: 2.1.0 + date: 02/22/2017 + - version: 2.0.2 date: 01/27/2017 http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/docs/spark/2.1.0/documentation.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/2.1.0/documentation.md b/site/docs/spark/2.1.0/documentation.md new file mode 100644 index 0000000..38148e9 --- /dev/null +++ b/site/docs/spark/2.1.0/documentation.md @@ -0,0 +1,46 @@ +--- +layout: page +title: Extensions for Apache Spark +description: Extensions for Apache Spark +group: nav-right +--- +<!-- +{% comment %} +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to you under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +{% endcomment %} +--> + +{% include JB/setup %} + +### Apache Bahir Extensions for Apache Spark + +<br/> + +#### Structured Streaming Data Sources + +[MQTT data source](../spark-sql-streaming-mqtt) + +<br/> + +#### Discretized Streams (DStreams) Connectors + +[Akka connector](../spark-streaming-akka) + +[MQTT connector](../spark-streaming-mqtt) + +[Twitter connector](../spark-streaming-twitter) + +[ZeroMQ connector](../spark-streaming-zeromq) \ No newline at end of file http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/docs/spark/2.1.0/spark-sql-streaming-mqtt.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/2.1.0/spark-sql-streaming-mqtt.md b/site/docs/spark/2.1.0/spark-sql-streaming-mqtt.md new file mode 100644 index 0000000..eeda845 --- /dev/null +++ b/site/docs/spark/2.1.0/spark-sql-streaming-mqtt.md @@ -0,0 +1,147 @@ +--- +layout: page +title: Spark Structured Streaming MQTT +description: Spark Structured Streaming MQTT +group: nav-right +--- +<!-- +{% comment %} +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to you under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +{% endcomment %} +--> + +{% include JB/setup %} + +A library for reading data from MQTT Servers using Spark SQL Streaming ( or Structured streaming.). + +## Linking + +Using SBT: + + libraryDependencies += "org.apache.bahir" %% "spark-sql-streaming-mqtt" % "2.1.0" + +Using Maven: + + <dependency> + <groupId>org.apache.bahir</groupId> + <artifactId>spark-sql-streaming-mqtt_2.11</artifactId> + <version>2.1.0</version> + </dependency> + +This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. +For example, to include it when starting the spark shell: + + $ bin/spark-shell --packages org.apache.bahir:spark-sql-streaming-mqtt_2.11:2.1.0 + +Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. +The `--packages` argument can also be used with `bin/spark-submit`. + +This library is compiled for Scala 2.11 only, and intends to support Spark 2.0 onwards. + +## Examples + +A SQL Stream can be created with data streams received through MQTT Server using, + + sqlContext.readStream + .format("org.apache.bahir.sql.streaming.mqtt.MQTTStreamSourceProvider") + .option("topic", "mytopic") + .load("tcp://localhost:1883") + +## Enable recovering from failures. + +Setting values for option `localStorage` and `clientId` helps in recovering in case of a restart, by restoring the state where it left off before the shutdown. + + sqlContext.readStream + .format("org.apache.bahir.sql.streaming.mqtt.MQTTStreamSourceProvider") + .option("topic", "mytopic") + .option("localStorage", "/path/to/localdir") + .option("clientId", "some-client-id") + .load("tcp://localhost:1883") + +## Configuration options. + +This source uses [Eclipse Paho Java Client](https://eclipse.org/paho/clients/java/). Client API documentation is located [here](http://www.eclipse.org/paho/files/javadoc/index.html). + + * `brokerUrl` A url MqttClient connects to. Set this or `path` as the url of the Mqtt Server. e.g. tcp://localhost:1883. + * `persistence` By default it is used for storing incoming messages on disk. If `memory` is provided as value for this option, then recovery on restart is not supported. + * `topic` Topic MqttClient subscribes to. + * `clientId` clientId, this client is assoicated with. Provide the same value to recover a stopped client. + * `QoS` The maximum quality of service to subscribe each topic at. Messages published at a lower quality of service will be received at the published QoS. Messages published at a higher quality of service will be received using the QoS specified on the subscribe. + * `username` Sets the user name to use for the connection to Mqtt Server. Do not set it, if server does not need this. Setting it empty will lead to errors. + * `password` Sets the password to use for the connection. + * `cleanSession` Setting it true starts a clean session, removes all checkpointed messages by a previous run of this source. This is set to false by default. + * `connectionTimeout` Sets the connection timeout, a value of 0 is interpretted as wait until client connects. See `MqttConnectOptions.setConnectionTimeout` for more information. + * `keepAlive` Same as `MqttConnectOptions.setKeepAliveInterval`. + * `mqttVersion` Same as `MqttConnectOptions.setMqttVersion`. + +### Scala API + +An example, for scala API to count words from incoming message stream. + + // Create DataFrame representing the stream of input lines from connection to mqtt server + val lines = spark.readStream + .format("org.apache.bahir.sql.streaming.mqtt.MQTTStreamSourceProvider") + .option("topic", topic) + .load(brokerUrl).as[(String, Timestamp)] + + // Split the lines into words + val words = lines.map(_._1).flatMap(_.split(" ")) + + // Generate running word count + val wordCounts = words.groupBy("value").count() + + // Start running the query that prints the running counts to the console + val query = wordCounts.writeStream + .outputMode("complete") + .format("console") + .start() + + query.awaitTermination() + +Please see `MQTTStreamWordCount.scala` for full example. + +### Java API + +An example, for Java API to count words from incoming message stream. + + // Create DataFrame representing the stream of input lines from connection to mqtt server. + Dataset<String> lines = spark + .readStream() + .format("org.apache.bahir.sql.streaming.mqtt.MQTTStreamSourceProvider") + .option("topic", topic) + .load(brokerUrl).select("value").as(Encoders.STRING()); + + // Split the lines into words + Dataset<String> words = lines.flatMap(new FlatMapFunction<String, String>() { + @Override + public Iterator<String> call(String x) { + return Arrays.asList(x.split(" ")).iterator(); + } + }, Encoders.STRING()); + + // Generate running word count + Dataset<Row> wordCounts = words.groupBy("value").count(); + + // Start running the query that prints the running counts to the console + StreamingQuery query = wordCounts.writeStream() + .outputMode("complete") + .format("console") + .start(); + + query.awaitTermination(); + +Please see `JavaMQTTStreamWordCount.java` for full example. + http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/docs/spark/2.1.0/spark-streaming-akka.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/2.1.0/spark-streaming-akka.md b/site/docs/spark/2.1.0/spark-streaming-akka.md new file mode 100644 index 0000000..57ef91d --- /dev/null +++ b/site/docs/spark/2.1.0/spark-streaming-akka.md @@ -0,0 +1,89 @@ +--- +layout: page +title: Spark Streaming Akka +description: Spark Streaming Akka +group: nav-right +--- +<!-- +{% comment %} +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to you under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +{% endcomment %} +--> + +{% include JB/setup %} + +A library for reading data from Akka Actors using Spark Streaming. + +## Linking + +Using SBT: + + libraryDependencies += "org.apache.bahir" %% "spark-streaming-akka" % "2.1.0" + +Using Maven: + + <dependency> + <groupId>org.apache.bahir</groupId> + <artifactId>spark-streaming-akka_2.11</artifactId> + <version>2.1.0</version> + </dependency> + +This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. +For example, to include it when starting the spark shell: + + $ bin/spark-shell --packages org.apache.bahir:spark-streaming-akka_2.11:2.1.0 + +Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. +The `--packages` argument can also be used with `bin/spark-submit`. + +This library is cross-published for Scala 2.10 and Scala 2.11, so users should replace the proper Scala version (2.10 or 2.11) in the commands listed above. + +## Examples + +DStreams can be created with data streams received through Akka actors by using `AkkaUtils.createStream(ssc, actorProps, actor-name)`. + +### Scala API + +You need to extend `ActorReceiver` so as to store received data into Spark using `store(...)` methods. The supervisor strategy of +this actor can be configured to handle failures, etc. + + class CustomActor extends ActorReceiver { + def receive = { + case data: String => store(data) + } + } + + // A new input stream can be created with this custom actor as + val ssc: StreamingContext = ... + val lines = AkkaUtils.createStream[String](ssc, Props[CustomActor](), "CustomReceiver") + +### Java API + +You need to extend `JavaActorReceiver` so as to store received data into Spark using `store(...)` methods. The supervisor strategy of +this actor can be configured to handle failures, etc. + + class CustomActor extends JavaActorReceiver { + @Override + public void onReceive(Object msg) throws Exception { + store((String) msg); + } + } + + // A new input stream can be created with this custom actor as + JavaStreamingContext jssc = ...; + JavaDStream<String> lines = AkkaUtils.<String>createStream(jssc, Props.create(CustomActor.class), "CustomReceiver"); + +See end-to-end examples at [Akka Examples](https://github.com/apache/bahir/tree/master/streaming-akka/examples) http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/docs/spark/2.1.0/spark-streaming-mqtt.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/2.1.0/spark-streaming-mqtt.md b/site/docs/spark/2.1.0/spark-streaming-mqtt.md new file mode 100644 index 0000000..38d2922 --- /dev/null +++ b/site/docs/spark/2.1.0/spark-streaming-mqtt.md @@ -0,0 +1,94 @@ +--- +layout: page +title: Spark Structured Streaming MQTT +description: Spark Structured Streaming MQTT +group: nav-right +--- +<!-- +{% comment %} +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to you under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +{% endcomment %} +--> + +{% include JB/setup %} + + +[MQTT](http://mqtt.org/) is MQTT is a machine-to-machine (M2M)/"Internet of Things" connectivity protocol. It was designed as an extremely lightweight publish/subscribe messaging transport. It is useful for connections with remote locations where a small code footprint is required and/or network bandwidth is at a premium. + +## Linking + +Using SBT: + + libraryDependencies += "org.apache.bahir" %% "spark-streaming-mqtt" % "2.1.0" + +Using Maven: + + <dependency> + <groupId>org.apache.bahir</groupId> + <artifactId>spark-streaming-mqtt_2.11</artifactId> + <version>2.1.0</version> + </dependency> + +This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. +For example, to include it when starting the spark shell: + + $ bin/spark-shell --packages org.apache.bahir:spark-streaming-mqtt_2.11:2.1.0 + +Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. +The `--packages` argument can also be used with `bin/spark-submit`. + +This library is cross-published for Scala 2.10 and Scala 2.11, so users should replace the proper Scala version (2.10 or 2.11) in the commands listed above. + +## Configuration options. + +This source uses the [Eclipse Paho Java Client](https://eclipse.org/paho/clients/java/). Client API documentation is located [here](http://www.eclipse.org/paho/files/javadoc/index.html). + + * `brokerUrl` A url MqttClient connects to. Set this as the url of the Mqtt Server. e.g. tcp://localhost:1883. + * `storageLevel` By default it is used for storing incoming messages on disk. + * `topic` Topic MqttClient subscribes to. + * `clientId` clientId, this client is assoicated with. Provide the same value to recover a stopped client. + * `QoS` The maximum quality of service to subscribe each topic at. Messages published at a lower quality of service will be received at the published QoS. Messages published at a higher quality of service will be received using the QoS specified on the subscribe. + * `username` Sets the user name to use for the connection to Mqtt Server. Do not set it, if server does not need this. Setting it empty will lead to errors. + * `password` Sets the password to use for the connection. + * `cleanSession` Setting it true starts a clean session, removes all checkpointed messages by a previous run of this source. This is set to false by default. + * `connectionTimeout` Sets the connection timeout, a value of 0 is interpreted as wait until client connects. See `MqttConnectOptions.setConnectionTimeout` for more information. + * `keepAlive` Same as `MqttConnectOptions.setKeepAliveInterval`. + * `mqttVersion` Same as `MqttConnectOptions.setMqttVersion`. + + +## Examples + +### Scala API + +You need to extend `ActorReceiver` so as to store received data into Spark using `store(...)` methods. The supervisor strategy of +this actor can be configured to handle failures, etc. + + val lines = MQTTUtils.createStream(ssc, brokerUrl, topic) + +Additional mqtt connection options can be provided: + +```Scala +val lines = MQTTUtils.createStream(ssc, brokerUrl, topic, storageLevel, clientId, username, password, cleanSession, qos, connectionTimeout, keepAliveInterval, mqttVersion) +``` + +### Java API + +You need to extend `JavaActorReceiver` so as to store received data into Spark using `store(...)` methods. The supervisor strategy of +this actor can be configured to handle failures, etc. + + JavaDStream<String> lines = MQTTUtils.createStream(jssc, brokerUrl, topic); + +See end-to-end examples at [MQTT Examples](https://github.com/apache/bahir/tree/master/streaming-mqtt/examples) \ No newline at end of file http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/docs/spark/2.1.0/spark-streaming-twitter.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/2.1.0/spark-streaming-twitter.md b/site/docs/spark/2.1.0/spark-streaming-twitter.md new file mode 100644 index 0000000..9e05f07 --- /dev/null +++ b/site/docs/spark/2.1.0/spark-streaming-twitter.md @@ -0,0 +1,74 @@ +--- +layout: page +title: Spark Streaming Twitter +description: Spark Streaming Twitter +group: nav-right +--- +<!-- +{% comment %} +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to you under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +{% endcomment %} +--> + +{% include JB/setup %} + +A library for reading social data from [twitter](http://twitter.com/) using Spark Streaming. + +## Linking + +Using SBT: + + libraryDependencies += "org.apache.bahir" %% "spark-streaming-twitter" % "2.1.0" + +Using Maven: + + <dependency> + <groupId>org.apache.bahir</groupId> + <artifactId>spark-streaming-twitter_2.11</artifactId> + <version>2.1.0</version> + </dependency> + +This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. +For example, to include it when starting the spark shell: + + $ bin/spark-shell --packages org.apache.bahir:spark-streaming-twitter_2.11:2.1.0 + +Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. +The `--packages` argument can also be used with `bin/spark-submit`. + +This library is cross-published for Scala 2.10 and Scala 2.11, so users should replace the proper Scala version (2.10 or 2.11) in the commands listed above. + + +## Examples + +`TwitterUtils` uses Twitter4j to get the public stream of tweets using [Twitter's Streaming API](https://dev.twitter.com/docs/streaming-apis). Authentication information +can be provided by any of the [methods](http://twitter4j.org/en/configuration.html) supported by Twitter4J library. You can import the `TwitterUtils` class and create a DStream with `TwitterUtils.createStream` as shown below. + +### Scala API + + import org.apache.spark.streaming.twitter._ + + TwitterUtils.createStream(ssc, None) + +### Java API + + import org.apache.spark.streaming.twitter.*; + + TwitterUtils.createStream(jssc); + + +You can also either get the public stream, or get the filtered stream based on keywords. +See end-to-end examples at [Twitter Examples](https://github.com/apache/bahir/tree/master/streaming-twitter/examples) \ No newline at end of file http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/docs/spark/2.1.0/spark-streaming-zeromq.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/2.1.0/spark-streaming-zeromq.md b/site/docs/spark/2.1.0/spark-streaming-zeromq.md new file mode 100644 index 0000000..cf10156 --- /dev/null +++ b/site/docs/spark/2.1.0/spark-streaming-zeromq.md @@ -0,0 +1,65 @@ +--- +layout: page +title: Spark Streaming ZeroMQ +description: Spark Streaming ZeroMQ +group: nav-right +--- +<!-- +{% comment %} +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to you under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +{% endcomment %} +--> + +{% include JB/setup %} + +A library for reading data from [ZeroMQ](http://zeromq.org/) using Spark Streaming. + +## Linking + +Using SBT: + + libraryDependencies += "org.apache.bahir" %% "spark-streaming-zeromq" % "2.1.0" + +Using Maven: + + <dependency> + <groupId>org.apache.bahir</groupId> + <artifactId>spark-streaming-zeromq_2.11</artifactId> + <version>2.1.0</version> + </dependency> + +This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. +For example, to include it when starting the spark shell: + + $ bin/spark-shell --packages org.apache.bahir:spark-streaming-zeromq_2.11:2.1.0 + +Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. +The `--packages` argument can also be used with `bin/spark-submit`. + +This library is cross-published for Scala 2.10 and Scala 2.11, so users should replace the proper Scala version (2.10 or 2.11) in the commands listed above. + +## Examples + + +### Scala API + + val lines = ZeroMQUtils.createStream(ssc, ...) + +### Java API + + JavaDStream<String> lines = ZeroMQUtils.createStream(jssc, ...); + +See end-to-end examples at [ZeroMQ Examples](https://github.com/apache/bahir/tree/master/streaming-zeromq/examples) \ No newline at end of file http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/docs/spark/current/spark-sql-streaming-mqtt.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/current/spark-sql-streaming-mqtt.md b/site/docs/spark/current/spark-sql-streaming-mqtt.md index a595565..eeda845 100644 --- a/site/docs/spark/current/spark-sql-streaming-mqtt.md +++ b/site/docs/spark/current/spark-sql-streaming-mqtt.md @@ -31,20 +31,20 @@ A library for reading data from MQTT Servers using Spark SQL Streaming ( or Stru Using SBT: - libraryDependencies += "org.apache.bahir" %% "spark-sql-streaming-mqtt" % "2.0.2" + libraryDependencies += "org.apache.bahir" %% "spark-sql-streaming-mqtt" % "2.1.0" Using Maven: <dependency> <groupId>org.apache.bahir</groupId> <artifactId>spark-sql-streaming-mqtt_2.11</artifactId> - <version>2.0.2</version> + <version>2.1.0</version> </dependency> This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. For example, to include it when starting the spark shell: - $ bin/spark-shell --packages org.apache.bahir:spark-sql-streaming-mqtt_2.11:2.0.2 + $ bin/spark-shell --packages org.apache.bahir:spark-sql-streaming-mqtt_2.11:2.1.0 Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. The `--packages` argument can also be used with `bin/spark-submit`. http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/docs/spark/current/spark-streaming-akka.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/current/spark-streaming-akka.md b/site/docs/spark/current/spark-streaming-akka.md index 6854bc4..57ef91d 100644 --- a/site/docs/spark/current/spark-streaming-akka.md +++ b/site/docs/spark/current/spark-streaming-akka.md @@ -31,20 +31,20 @@ A library for reading data from Akka Actors using Spark Streaming. Using SBT: - libraryDependencies += "org.apache.bahir" %% "spark-streaming-akka" % "2.0.2" + libraryDependencies += "org.apache.bahir" %% "spark-streaming-akka" % "2.1.0" Using Maven: <dependency> <groupId>org.apache.bahir</groupId> <artifactId>spark-streaming-akka_2.11</artifactId> - <version>2.0.2</version> + <version>2.1.0</version> </dependency> This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. For example, to include it when starting the spark shell: - $ bin/spark-shell --packages org.apache.bahir:spark-streaming-akka_2.11:2.0.2 + $ bin/spark-shell --packages org.apache.bahir:spark-streaming-akka_2.11:2.1.0 Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. The `--packages` argument can also be used with `bin/spark-submit`. http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/docs/spark/current/spark-streaming-mqtt.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/current/spark-streaming-mqtt.md b/site/docs/spark/current/spark-streaming-mqtt.md index 1be8bbb..38d2922 100644 --- a/site/docs/spark/current/spark-streaming-mqtt.md +++ b/site/docs/spark/current/spark-streaming-mqtt.md @@ -32,20 +32,20 @@ limitations under the License. Using SBT: - libraryDependencies += "org.apache.bahir" %% "spark-streaming-mqtt" % "2.0.2" + libraryDependencies += "org.apache.bahir" %% "spark-streaming-mqtt" % "2.1.0" Using Maven: <dependency> <groupId>org.apache.bahir</groupId> <artifactId>spark-streaming-mqtt_2.11</artifactId> - <version>2.0.2</version> + <version>2.1.0</version> </dependency> This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. For example, to include it when starting the spark shell: - $ bin/spark-shell --packages org.apache.bahir:spark-streaming-mqtt_2.11:2.0.2 + $ bin/spark-shell --packages org.apache.bahir:spark-streaming-mqtt_2.11:2.1.0 Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. The `--packages` argument can also be used with `bin/spark-submit`. http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/docs/spark/current/spark-streaming-twitter.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/current/spark-streaming-twitter.md b/site/docs/spark/current/spark-streaming-twitter.md index 5e61058..9e05f07 100644 --- a/site/docs/spark/current/spark-streaming-twitter.md +++ b/site/docs/spark/current/spark-streaming-twitter.md @@ -31,20 +31,20 @@ A library for reading social data from [twitter](http://twitter.com/) using Spar Using SBT: - libraryDependencies += "org.apache.bahir" %% "spark-streaming-twitter" % "2.0.2" + libraryDependencies += "org.apache.bahir" %% "spark-streaming-twitter" % "2.1.0" Using Maven: <dependency> <groupId>org.apache.bahir</groupId> <artifactId>spark-streaming-twitter_2.11</artifactId> - <version>2.0.2</version> + <version>2.1.0</version> </dependency> This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. For example, to include it when starting the spark shell: - $ bin/spark-shell --packages org.apache.bahir:spark-streaming-twitter_2.11:2.0.2 + $ bin/spark-shell --packages org.apache.bahir:spark-streaming-twitter_2.11:2.1.0 Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. The `--packages` argument can also be used with `bin/spark-submit`. http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/docs/spark/current/spark-streaming-zeromq.md ---------------------------------------------------------------------- diff --git a/site/docs/spark/current/spark-streaming-zeromq.md b/site/docs/spark/current/spark-streaming-zeromq.md index 937517d..cf10156 100644 --- a/site/docs/spark/current/spark-streaming-zeromq.md +++ b/site/docs/spark/current/spark-streaming-zeromq.md @@ -31,20 +31,20 @@ A library for reading data from [ZeroMQ](http://zeromq.org/) using Spark Streami Using SBT: - libraryDependencies += "org.apache.bahir" %% "spark-streaming-zeromq" % "2.0.2" + libraryDependencies += "org.apache.bahir" %% "spark-streaming-zeromq" % "2.1.0" Using Maven: <dependency> <groupId>org.apache.bahir</groupId> <artifactId>spark-streaming-zeromq_2.11</artifactId> - <version>2.0.2</version> + <version>2.1.0</version> </dependency> This library can also be added to Spark jobs launched through `spark-shell` or `spark-submit` by using the `--packages` command line option. For example, to include it when starting the spark shell: - $ bin/spark-shell --packages org.apache.bahir:spark-streaming-zeromq_2.11:2.0.2 + $ bin/spark-shell --packages org.apache.bahir:spark-streaming-zeromq_2.11:2.1.0 Unlike using `--jars`, using `--packages` ensures that this library and its dependencies will be added to the classpath. The `--packages` argument can also be used with `bin/spark-submit`. http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/releases/2.0.0/release-notes.md ---------------------------------------------------------------------- diff --git a/site/releases/2.0.0/release-notes.md b/site/releases/2.0.0/release-notes.md index f21b520..51f1caa 100644 --- a/site/releases/2.0.0/release-notes.md +++ b/site/releases/2.0.0/release-notes.md @@ -1,7 +1,7 @@ --- layout: page -title: Community -description: Project Community Page +title: Bahir Release 2.0.0 +description: Release Notes for Apache Bahir 2.0.0 group: nav-right --- <!-- @@ -26,7 +26,7 @@ limitations under the License. {% include JB/setup %} -# Apache Bahir 2.0.0 for Apache Spark +# Apache Bahir 2.0 for Apache Spark 2.0 ## Bug http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/releases/2.0.1/release-notes.md ---------------------------------------------------------------------- diff --git a/site/releases/2.0.1/release-notes.md b/site/releases/2.0.1/release-notes.md index 5c922ae..90d6d94 100644 --- a/site/releases/2.0.1/release-notes.md +++ b/site/releases/2.0.1/release-notes.md @@ -1,7 +1,7 @@ --- layout: page -title: Community -description: Project Community Page +title: Bahir Release 2.0.1 +description: Release Notes for Apache Bahir 2.0.1 group: nav-right --- <!-- @@ -25,7 +25,7 @@ limitations under the License. {% include JB/setup %} -# Apache Bahir 2.0.1 for Apache Spark +# Apache Bahir 2.0.1 for Apache Spark 2.0.1 ## Bug http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/releases/2.0.2/release-notes.md ---------------------------------------------------------------------- diff --git a/site/releases/2.0.2/release-notes.md b/site/releases/2.0.2/release-notes.md index fc3a4bc..2636f48 100644 --- a/site/releases/2.0.2/release-notes.md +++ b/site/releases/2.0.2/release-notes.md @@ -1,7 +1,7 @@ --- layout: page -title: Community -description: Project Community Page +title: Bahir Release 2.0.2 +description: Release Notes for Apache Bahir 2.0.2 group: nav-right --- <!-- @@ -26,7 +26,7 @@ limitations under the License. {% include JB/setup %} -# Apache Bahir 2.0.2 for Apache Spark +# Apache Bahir 2.0.2 for Apache Spark 2.0.2 ## Task http://git-wip-us.apache.org/repos/asf/bahir-website/blob/0d53ccdd/site/releases/2.1.0/release-notes.md ---------------------------------------------------------------------- diff --git a/site/releases/2.1.0/release-notes.md b/site/releases/2.1.0/release-notes.md new file mode 100644 index 0000000..e33f499 --- /dev/null +++ b/site/releases/2.1.0/release-notes.md @@ -0,0 +1,36 @@ +--- +layout: page +title: Bahir Release 2.1.0 +description: Release Notes for Apache Bahir 2.1.0 +group: nav-right +--- +<!-- +{% comment %} +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to you under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +{% endcomment %} +--> + +{% include JB/setup %} + + +# Apache Bahir 2.1 for Apache Spark 2.1 + +## Task + + [BAHIR-87] - Prepare release based on Apache Spark 2.1.0 + [MINOR] - Update Scaladoc in MQTTWordCount example + [MINOR] - Fix Maven artifact IDs in README.md files +
