Repository: bahir-website Updated Branches: refs/heads/master 93a994524 -> c68a138c9
[BAHIR-147] update with last extensions Closes #14 Project: http://git-wip-us.apache.org/repos/asf/bahir-website/repo Commit: http://git-wip-us.apache.org/repos/asf/bahir-website/commit/c68a138c Tree: http://git-wip-us.apache.org/repos/asf/bahir-website/tree/c68a138c Diff: http://git-wip-us.apache.org/repos/asf/bahir-website/diff/c68a138c Branch: refs/heads/master Commit: c68a138c94171fa36341dd723502dbd8aaf85777 Parents: 93a9945 Author: Joao Boto <b...@boto.pro> Authored: Tue Dec 11 22:23:55 2018 +0100 Committer: Luciano Resende <lrese...@apache.org> Committed: Tue Dec 11 22:20:45 2018 -0300 ---------------------------------------------------------------------- NOTICE | 2 +- site/_includes/themes/apache-clean/footer.html | 5 +- site/docs/flink/current/documentation.md | 8 +- .../flink/current/flink-streaming-activemq.md | 2 +- site/docs/flink/current/flink-streaming-akka.md | 2 +- .../docs/flink/current/flink-streaming-flume.md | 6 +- .../flink/current/flink-streaming-influxdb.md | 59 +++++++++ site/docs/flink/current/flink-streaming-kudu.md | 125 +++++++++++++++++++ .../docs/flink/current/flink-streaming-netty.md | 2 +- .../docs/flink/current/flink-streaming-redis.md | 2 +- site/docs/flink/overview.md | 2 +- .../templates/flink-streaming-influxdb.template | 27 ++++ .../templates/flink-streaming-kudu.template | 27 ++++ site/index.md | 4 +- 14 files changed, 258 insertions(+), 15 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/bahir-website/blob/c68a138c/NOTICE ---------------------------------------------------------------------- diff --git a/NOTICE b/NOTICE index 4127e40..7cf47a7 100644 --- a/NOTICE +++ b/NOTICE @@ -1,5 +1,5 @@ Apache Website Template -Copyright [2016-2017] The Apache Software Foundation +Copyright [2016-2018] The Apache Software Foundation This product includes software developed at The Apache Software Foundation (http://www.apache.org/). http://git-wip-us.apache.org/repos/asf/bahir-website/blob/c68a138c/site/_includes/themes/apache-clean/footer.html ---------------------------------------------------------------------- diff --git a/site/_includes/themes/apache-clean/footer.html b/site/_includes/themes/apache-clean/footer.html index 6070a43..f86e28d 100644 --- a/site/_includes/themes/apache-clean/footer.html +++ b/site/_includes/themes/apache-clean/footer.html @@ -18,9 +18,8 @@ </div> {% endif %} <div> - Copyright © 2016-2017 <a href="http://www.apache.org";>The Apache Software Foundation</a>. - Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version - 2.0</a>. + Copyright © 2016-<script>document.write(new Date().getFullYear());</script> <a href="http://www.apache.org";>The Apache Software Foundation</a>. + Licensed under the <a href="http://www.apache.org/licenses/LICENSE-2.0";>Apache License, Version 2.0</a>. <br> {% if site.data.project.podling %} Apache, the Apache Feather logo, and the Apache Incubator project logo are trademarks of The Apache http://git-wip-us.apache.org/repos/asf/bahir-website/blob/c68a138c/site/docs/flink/current/documentation.md ---------------------------------------------------------------------- diff --git a/site/docs/flink/current/documentation.md b/site/docs/flink/current/documentation.md index 55b557f..1af2a3c 100644 --- a/site/docs/flink/current/documentation.md +++ b/site/docs/flink/current/documentation.md @@ -1,7 +1,7 @@ --- layout: page -title: Extensions for Apache Flink (1.0.0-SNAPSHOT) -description: Extensions for Apache Flink (1.0.0-SNAPSHOT) +title: Extensions for Apache Flink (1.1-SNAPSHOT) +description: Extensions for Apache Flink (1.1-SNAPSHOT) group: nav-right --- <!-- @@ -37,6 +37,10 @@ limitations under the License. [Flume connector](../flink-streaming-flume) +[InfluxDB connector](../flink-streaming-influxdb) + +[Kudu connector](../flink-streaming-kudu) + [Netty connector](../flink-streaming-netty) [Redis connector](../flink-streaming-redis) http://git-wip-us.apache.org/repos/asf/bahir-website/blob/c68a138c/site/docs/flink/current/flink-streaming-activemq.md ---------------------------------------------------------------------- diff --git a/site/docs/flink/current/flink-streaming-activemq.md b/site/docs/flink/current/flink-streaming-activemq.md index ce8a162..d94bf50 100644 --- a/site/docs/flink/current/flink-streaming-activemq.md +++ b/site/docs/flink/current/flink-streaming-activemq.md @@ -33,7 +33,7 @@ To use this connector, add the following dependency to your project: <dependency> <groupId>org.apache.bahir</groupId> <artifactId>flink-connector-activemq_2.11</artifactId> - <version>1.0-SNAPSHOT</version> + <version>1.1-SNAPSHOT</version> </dependency> *Version Compatibility*: This module is compatible with ActiveMQ 5.14.0. http://git-wip-us.apache.org/repos/asf/bahir-website/blob/c68a138c/site/docs/flink/current/flink-streaming-akka.md ---------------------------------------------------------------------- diff --git a/site/docs/flink/current/flink-streaming-akka.md b/site/docs/flink/current/flink-streaming-akka.md index 7469666..0af3e13 100644 --- a/site/docs/flink/current/flink-streaming-akka.md +++ b/site/docs/flink/current/flink-streaming-akka.md @@ -33,7 +33,7 @@ To use this connector, add the following dependency to your project: <dependency> <groupId>org.apache.bahir</groupId> <artifactId>flink-connector-akka_2.11</artifactId> - <version>1.0-SNAPSHOT</version> + <version>1.1-SNAPSHOT</version> </dependency> *Version Compatibility*: This module is compatible with Akka 2.0+. http://git-wip-us.apache.org/repos/asf/bahir-website/blob/c68a138c/site/docs/flink/current/flink-streaming-flume.md ---------------------------------------------------------------------- diff --git a/site/docs/flink/current/flink-streaming-flume.md b/site/docs/flink/current/flink-streaming-flume.md index 01e1bae..c5e9a89 100644 --- a/site/docs/flink/current/flink-streaming-flume.md +++ b/site/docs/flink/current/flink-streaming-flume.md @@ -33,10 +33,10 @@ following dependency to your project: <dependency> <groupId>org.apache.bahir</groupId> <artifactId>flink-connector-flume_2.11</artifactId> - <version>1.0-SNAPSHOT</version> + <version>1.1-SNAPSHOT</version> </dependency> -*Version Compatibility*: This module is compatible with Flume 1.5.0. +*Version Compatibility*: This module is compatible with Flume 1.8.0. Note that the streaming connectors are not part of the binary distribution of Flink. You need to link them into your job jar for cluster execution. See how to link with them for cluster execution [here](https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/linking.html). @@ -44,4 +44,4 @@ See how to link with them for cluster execution [here](https://ci.apache.org/pro To create a `FlumeSink` instantiate the following constructor: FlumeSink(String host, int port, SerializationSchema<IN> schema) - \ No newline at end of file + http://git-wip-us.apache.org/repos/asf/bahir-website/blob/c68a138c/site/docs/flink/current/flink-streaming-influxdb.md ---------------------------------------------------------------------- diff --git a/site/docs/flink/current/flink-streaming-influxdb.md b/site/docs/flink/current/flink-streaming-influxdb.md new file mode 100644 index 0000000..fe0d946 --- /dev/null +++ b/site/docs/flink/current/flink-streaming-influxdb.md @@ -0,0 +1,59 @@ +--- +layout: page +title: Apache Flink Streaming Connector for InfluxDB +description: Apache Flink Streaming Connector for InfluxDB +group: nav-right +--- +<!-- +{% comment %} +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to you under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +{% endcomment %} +--> + +{% include JB/setup %} + +# Flink InfluxDB Connector + +This connector provides a sink that can send data to [InfluxDB](https://www.influxdata.com/). To use this connector, add the +following dependency to your project: + + <dependency> + <groupId>org.apache.bahir</groupId> + <artifactId>flink-connector-influxdb_2.11</artifactId> + <version>1.1-SNAPSHOT</version> + </dependency> + +*Version Compatibility*: This module is compatible with InfluxDB 1.3.x +*Requirements*: Java 1.8+ + +Note that the streaming connectors are not part of the binary distribution of Flink. You need to link them into your job jar for cluster execution. +See how to link with them for cluster execution [here](https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/linking.html). + +## Installing InfluxDB +Follow the instructions from the [InfluxDB download page](https://portal.influxdata.com/downloads#influxdb). + +## Examples + +### JAVA API + + DataStream<InfluxDBPoint> dataStream = ... + InfluxDBConfig influxDBConfig = InfluxDBConfig.builder(String host, String username, String password, String dbName) + dataStream.addSink(new InfluxDBSink(influxDBConfig)); + + +See end-to-end examples at [InfluxDB Examples](https://github.com/apache/bahir-flink/tree/master/flink-connector-influxdb/examples) + + http://git-wip-us.apache.org/repos/asf/bahir-website/blob/c68a138c/site/docs/flink/current/flink-streaming-kudu.md ---------------------------------------------------------------------- diff --git a/site/docs/flink/current/flink-streaming-kudu.md b/site/docs/flink/current/flink-streaming-kudu.md new file mode 100644 index 0000000..2eef38c --- /dev/null +++ b/site/docs/flink/current/flink-streaming-kudu.md @@ -0,0 +1,125 @@ +--- +layout: page +title: Apache Flink Streaming Connector for Apache Kudu +description: Apache Flink Streaming Connector for Apache Kudu +group: nav-right +--- +<!-- +{% comment %} +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to you under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +{% endcomment %} +--> + +{% include JB/setup %} + +# Flink Kudu Connector + +This connector provides a source (```KuduInputFormat```) and a sink/output (```KuduSink``` and ```KuduOutputFormat```, respectively) that can read and write to [Kudu](https://kudu.apache.org/). To use this connector, add the +following dependency to your project: + + <dependency> + <groupId>org.apache.bahir</groupId> + <artifactId>flink-connector-kudu_2.11</artifactId> + <version>1.1-SNAPSHOT</version> + </dependency> + +*Version Compatibility*: This module is compatible with Apache Kudu *1.7.1* (last stable version). + +Note that the streaming connectors are not part of the binary distribution of Flink. You need to link them into your job jar for cluster execution. +See how to link with them for cluster execution [here](https://ci.apache.org/projects/flink/flink-docs-stable/start/dependencies.html). + +## Installing Kudu + +Follow the instructions from the [Kudu Installation Guide](https://kudu.apache.org/docs/installation.html). +Optionally, you can use the docker images provided in dockers folder. + +## KuduInputFormat + +``` +ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment(); + +env.setParallelism(PARALLELISM); + +// create a table info object +KuduTableInfo tableInfo = KuduTableInfo.Builder + .create("books") + .addColumn(KuduColumnInfo.Builder.create("id", Type.INT32).key(true).hashKey(true).build()) + .addColumn(KuduColumnInfo.Builder.create("title", Type.STRING).build()) + .addColumn(KuduColumnInfo.Builder.create("author", Type.STRING).build()) + .addColumn(KuduColumnInfo.Builder.create("price", Type.DOUBLE).build()) + .addColumn(KuduColumnInfo.Builder.create("quantity", Type.INT32).build()) + .build(); + +// Pass the tableInfo to the KuduInputFormat and provide kuduMaster ips +env.createInput(new KuduInputFormat<>("172.25.0.6", tableInfo)) + .count(); + +env.execute(); +``` + +## KuduOutputFormat + +``` +ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment(); + +env.setParallelism(PARALLELISM); + +// create a table info object +KuduTableInfo tableInfo = KuduTableInfo.Builder + .create("books") + .createIfNotExist(true) + .replicas(1) + .addColumn(KuduColumnInfo.Builder.create("id", Type.INT32).key(true).hashKey(true).build()) + .addColumn(KuduColumnInfo.Builder.create("title", Type.STRING).build()) + .addColumn(KuduColumnInfo.Builder.create("author", Type.STRING).build()) + .addColumn(KuduColumnInfo.Builder.create("price", Type.DOUBLE).build()) + .addColumn(KuduColumnInfo.Builder.create("quantity", Type.INT32).build()) + .build(); + +... + +env.fromCollection(books) + .output(new KuduOutputFormat<>("172.25.0.6", tableInfo)); + +env.execute(); +``` + +## KuduSink + +``` +StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment(); + +env.setParallelism(PARALLELISM); + +// create a table info object +KuduTableInfo tableInfo = KuduTableInfo.Builder + .create("books") + .createIfNotExist(true) + .replicas(1) + .addColumn(KuduColumnInfo.Builder.create("id", Type.INT32).key(true).hashKey(true).build()) + .addColumn(KuduColumnInfo.Builder.create("title", Type.STRING).build()) + .addColumn(KuduColumnInfo.Builder.create("author", Type.STRING).build()) + .addColumn(KuduColumnInfo.Builder.create("price", Type.DOUBLE).build()) + .addColumn(KuduColumnInfo.Builder.create("quantity", Type.INT32).build()) + .build(); + +... + +env.fromCollection(books) + .addSink(new KuduSink<>("172.25.0.6", tableInfo)); + +env.execute(); +``` http://git-wip-us.apache.org/repos/asf/bahir-website/blob/c68a138c/site/docs/flink/current/flink-streaming-netty.md ---------------------------------------------------------------------- diff --git a/site/docs/flink/current/flink-streaming-netty.md b/site/docs/flink/current/flink-streaming-netty.md index 9ec2442..320537c 100644 --- a/site/docs/flink/current/flink-streaming-netty.md +++ b/site/docs/flink/current/flink-streaming-netty.md @@ -61,7 +61,7 @@ To use this connector, add the following dependency to your project: <dependency> <groupId>org.apache.bahir</groupId> <artifactId>flink-connector-netty_2.11</artifactId> - <version>1.0-SNAPSHOT</version> + <version>1.1-SNAPSHOT</version> </dependency> ``` http://git-wip-us.apache.org/repos/asf/bahir-website/blob/c68a138c/site/docs/flink/current/flink-streaming-redis.md ---------------------------------------------------------------------- diff --git a/site/docs/flink/current/flink-streaming-redis.md b/site/docs/flink/current/flink-streaming-redis.md index 01db8c9..1a2f8c2 100644 --- a/site/docs/flink/current/flink-streaming-redis.md +++ b/site/docs/flink/current/flink-streaming-redis.md @@ -34,7 +34,7 @@ following dependency to your project: <dependency> <groupId>org.apache.bahir</groupId> <artifactId>flink-connector-redis_2.11</artifactId> - <version>1.0-SNAPSHOT</version> + <version>1.1-SNAPSHOT</version> </dependency> *Version Compatibility*: This module is compatible with Redis 2.8.5. http://git-wip-us.apache.org/repos/asf/bahir-website/blob/c68a138c/site/docs/flink/overview.md ---------------------------------------------------------------------- diff --git a/site/docs/flink/overview.md b/site/docs/flink/overview.md index 86a4b60..a6793b6 100644 --- a/site/docs/flink/overview.md +++ b/site/docs/flink/overview.md @@ -27,5 +27,5 @@ limitations under the License. ### Apache Bahir Extensions for Apache Flink - - [Current - 1.0-SNAPSHOT](/docs/flink/current/documentation) + - [Current - 1.1-SNAPSHOT](/docs/flink/current/documentation) - [1.0](/docs/flink/1.0/documentation) http://git-wip-us.apache.org/repos/asf/bahir-website/blob/c68a138c/site/docs/flink/templates/flink-streaming-influxdb.template ---------------------------------------------------------------------- diff --git a/site/docs/flink/templates/flink-streaming-influxdb.template b/site/docs/flink/templates/flink-streaming-influxdb.template new file mode 100644 index 0000000..c507575 --- /dev/null +++ b/site/docs/flink/templates/flink-streaming-influxdb.template @@ -0,0 +1,27 @@ +--- +layout: page +title: Apache Flink Streaming Connector for InfluxDB +description: Apache Flink Streaming Connector for InfluxDB +group: nav-right +--- +<!-- +{% comment %} +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to you under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +{% endcomment %} +--> + +{% include JB/setup %} + http://git-wip-us.apache.org/repos/asf/bahir-website/blob/c68a138c/site/docs/flink/templates/flink-streaming-kudu.template ---------------------------------------------------------------------- diff --git a/site/docs/flink/templates/flink-streaming-kudu.template b/site/docs/flink/templates/flink-streaming-kudu.template new file mode 100644 index 0000000..53d007d --- /dev/null +++ b/site/docs/flink/templates/flink-streaming-kudu.template @@ -0,0 +1,27 @@ +--- +layout: page +title: Apache Flink Streaming Connector for Apache Kudu +description: Apache Flink Streaming Connector for Apache Kudu +group: nav-right +--- +<!-- +{% comment %} +Licensed to the Apache Software Foundation (ASF) under one or more +contributor license agreements. See the NOTICE file distributed with +this work for additional information regarding copyright ownership. +The ASF licenses this file to you under the Apache License, Version 2.0 +(the "License"); you may not use this file except in compliance with +the License. You may obtain a copy of the License at + +http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +{% endcomment %} +--> + +{% include JB/setup %} + http://git-wip-us.apache.org/repos/asf/bahir-website/blob/c68a138c/site/index.md ---------------------------------------------------------------------- diff --git a/site/index.md b/site/index.md index b105935..e6181d4 100644 --- a/site/index.md +++ b/site/index.md @@ -48,9 +48,11 @@ Currently, {{ site.data.project.short_name }} provides extensions for [Apache Sp ## Apache Flink extensions - Flink streaming connector for ActiveMQ + - Flink streaming connector for Akka - Flink streaming connector for Flume + - Flink streaming connector for InfluxDB {:height="36px" width="36px"} + - Flink streaming connector for Kudu {:height="36px" width="36px"} - Flink streaming connector for Redis - - Flink streaming connector for Akka - Flink streaming connector for Netty
