This is an automated email from the ASF dual-hosted git repository. dannycranmer pushed a commit to branch release-1.15 in repository https://gitbox.apache.org/repos/asf/flink.git
commit acb8ff35fc40d15376072aece091cfc3d9ddda57 Author: Danny Cranmer <[email protected]> AuthorDate: Mon Dec 5 10:20:05 2022 +0000 [FLINK-30291][Connector/DynamoDB] Update docs to render DynamoDB connector docs --- docs/config.toml | 12 ++++ .../docs/connectors/datastream/guarantees.md | 5 ++ .../docs/connectors/datastream/overview.md | 1 + docs/content.zh/docs/connectors/table/overview.md | 6 ++ .../docs/connectors/datastream/guarantees.md | 5 ++ .../content/docs/connectors/datastream/overview.md | 1 + docs/content/docs/connectors/table/overview.md | 6 ++ docs/data/sql_connectors.yml | 8 +++ docs/layouts/shortcodes/connector_artifact.html | 42 ++++++++++++++ .../shortcodes/sql_connector_download_table.html | 65 ++++++++++++++++++++++ docs/setup_docs.sh | 30 +++++++--- 11 files changed, 174 insertions(+), 7 deletions(-) diff --git a/docs/config.toml b/docs/config.toml index 4efd0bda304..85ff518f779 100644 --- a/docs/config.toml +++ b/docs/config.toml @@ -108,3 +108,15 @@ pygmentsUseClasses = true languageName = '中文版' contentDir = 'content.zh' weight = 2 + +[module] +[[module.imports]] + path = 'connectors' +[[module.imports.mounts]] + source = 'content' + target = 'content' + lang = 'en' +[[module.imports.mounts]] + source = 'content.zh' + target = 'content' + lang = 'zh' diff --git a/docs/content.zh/docs/connectors/datastream/guarantees.md b/docs/content.zh/docs/connectors/datastream/guarantees.md index 12da9b07788..3c2f9159968 100644 --- a/docs/content.zh/docs/connectors/datastream/guarantees.md +++ b/docs/content.zh/docs/connectors/datastream/guarantees.md @@ -106,6 +106,11 @@ under the License. <td>只有当更新是幂等时，保证精确一次</td> </tr> <tr> + <td>Amazon DynamoDB</td> + <td>至少一次</td> + <td></td> + </tr> + <tr> <td>Amazon Kinesis Data Streams</td> <td>至少一次</td> <td></td> diff --git a/docs/content.zh/docs/connectors/datastream/overview.md b/docs/content.zh/docs/connectors/datastream/overview.md index b61646f9c11..d2c0866f025 100644 --- a/docs/content.zh/docs/connectors/datastream/overview.md +++ b/docs/content.zh/docs/connectors/datastream/overview.md @@ -39,6 +39,7 @@ under the License. * [Apache Kafka]({{< ref "docs/connectors/datastream/kafka" >}}) (source/sink) * [Apache Cassandra]({{< ref "docs/connectors/datastream/cassandra" >}}) (sink) + * [Amazon DynamoDB]({{< ref "docs/connectors/datastream/dynamodb" >}}) (sink) * [Amazon Kinesis Data Streams]({{< ref "docs/connectors/datastream/kinesis" >}}) (source/sink) * [Amazon Kinesis Data Firehose]({{< ref "docs/connectors/datastream/firehose" >}}) (sink) * [Elasticsearch]({{< ref "docs/connectors/datastream/elasticsearch" >}}) (sink) diff --git a/docs/content.zh/docs/connectors/table/overview.md b/docs/content.zh/docs/connectors/table/overview.md index 7058e7ea102..d3dc4e58cd1 100644 --- a/docs/content.zh/docs/connectors/table/overview.md +++ b/docs/content.zh/docs/connectors/table/overview.md @@ -66,6 +66,12 @@ Flink natively support various connectors. The following tables list all availab <td>Unbounded Scan</td> <td>Streaming Sink, Batch Sink</td> </tr> + <tr> + <td><a href="{{< ref "docs/connectors/table/dynamodb" >}}">Amazon DynamoDB</a></td> + <td></td> + <td>Not supported</td> + <td>Streaming Sink, Batch Sink</td> + </tr> <tr> <td><a href="{{< ref "docs/connectors/table/kinesis" >}}">Amazon Kinesis Data Streams</a></td> <td></td> diff --git a/docs/content/docs/connectors/datastream/guarantees.md b/docs/content/docs/connectors/datastream/guarantees.md index b7915ba3066..be20256ff6a 100644 --- a/docs/content/docs/connectors/datastream/guarantees.md +++ b/docs/content/docs/connectors/datastream/guarantees.md @@ -110,6 +110,11 @@ state updates) of Flink coupled with bundled sinks: <td>at least once / exactly once</td> <td>exactly once only for idempotent updates</td> </tr> + <tr> + <td>Amazon DynamoDB</td> + <td>at least once</td> + <td></td> + </tr> <tr> <td>Amazon Kinesis Data Streams</td> <td>at least once</td> diff --git a/docs/content/docs/connectors/datastream/overview.md b/docs/content/docs/connectors/datastream/overview.md index 31a09cd213e..5b275b01b52 100644 --- a/docs/content/docs/connectors/datastream/overview.md +++ b/docs/content/docs/connectors/datastream/overview.md @@ -40,6 +40,7 @@ Connectors provide code for interfacing with various third-party systems. Curren * [Apache Kafka]({{< ref "docs/connectors/datastream/kafka" >}}) (source/sink) * [Apache Cassandra]({{< ref "docs/connectors/datastream/cassandra" >}}) (sink) + * [Amazon DynamoDB]({{< ref "docs/connectors/datastream/dynamodb" >}}) (sink) * [Amazon Kinesis Data Streams]({{< ref "docs/connectors/datastream/kinesis" >}}) (source/sink) * [Amazon Kinesis Data Firehose]({{< ref "docs/connectors/datastream/firehose" >}}) (sink) * [Elasticsearch]({{< ref "docs/connectors/datastream/elasticsearch" >}}) (sink) diff --git a/docs/content/docs/connectors/table/overview.md b/docs/content/docs/connectors/table/overview.md index 169ed340a25..dbe681958d9 100644 --- a/docs/content/docs/connectors/table/overview.md +++ b/docs/content/docs/connectors/table/overview.md @@ -66,6 +66,12 @@ Flink natively support various connectors. The following tables list all availab <td>Unbounded Scan</td> <td>Streaming Sink, Batch Sink</td> </tr> + <tr> + <td><a href="{{< ref "docs/connectors/table/dynamodb" >}}">Amazon DynamoDB</a></td> + <td></td> + <td>Not supported</td> + <td>Streaming Sink, Batch Sink</td> + </tr> <tr> <td><a href="{{< ref "docs/connectors/table/kinesis" >}}">Amazon Kinesis Data Streams</a></td> <td></td> diff --git a/docs/data/sql_connectors.yml b/docs/data/sql_connectors.yml index a3fc878003e..e98f9122d0a 100644 --- a/docs/data/sql_connectors.yml +++ b/docs/data/sql_connectors.yml @@ -171,3 +171,11 @@ aws-kinesis-firehose: category: connector maven: flink-connector-kinesis-firehose sql_url: https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-connector-aws-kinesis-firehose/$version/flink-sql-connector-aws-kinesis-firehose-$version.jar + +dynamodb: + name: Amazon DynamoDB + category: connector + versions: + - version: 3.0.0 + maven: flink-connector-dynamodb + sql_url: https://repo.maven.apache.org/maven2/org/apache/flink/flink-sql-connector-dynamodb/3.0.0-1.15/flink-sql-connector-dynamodb-3.0.0-1.15.jar diff --git a/docs/layouts/shortcodes/connector_artifact.html b/docs/layouts/shortcodes/connector_artifact.html new file mode 100644 index 00000000000..1e792bc52ba --- /dev/null +++ b/docs/layouts/shortcodes/connector_artifact.html @@ -0,0 +1,42 @@ +{{/* + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + */}} + {{/* + Generates an XML snippet for the maven artifact. + IMPORTANT: the whitespace is relevant. Do not change without looking at the + rendered documentation. + */}} + {{ $artifactId := .Get 0 }} + {{ $version := .Get 1 }} + + {{ $path := .Page.Path }} + + {{ $hash := md5 now }} + + {{ if $.Site.Params.IsStable }} + <div id="{{ $hash }}" onclick="selectTextAndCopy('{{ $hash }}')"class="highlight"><pre class="chroma"><code class="language-xml" data-lang="xml"><span class="nt"><dependency></span> + <span class="nt"><groupId></span>org.apache.flink<span class="nt"></groupId></span> + <span class="nt"><artifactId></span>{{- $artifactId -}}<span class="nt"></artifactId></span> + <span class="nt"><version></span>{{- $version -}}-{{- site.Params.VersionTitle -}}<span class="nt"></version></span> +<span class="nt"></dependency></span></code></pre></div> + <div class="book-hint info" style="text-align:center;display:none" copyable="flink-module" copyattribute="{{ $hash }}"> + Copied to clipboard! + </div> + {{ else }} + Only available for stable versions. + {{ end }} diff --git a/docs/layouts/shortcodes/sql_connector_download_table.html b/docs/layouts/shortcodes/sql_connector_download_table.html new file mode 100644 index 00000000000..22c058edbf4 --- /dev/null +++ b/docs/layouts/shortcodes/sql_connector_download_table.html @@ -0,0 +1,65 @@ +{{/* + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. + */}} + {{/* + Generates an XML snippet for the SQL connector download table. + IMPORTANT: the whitespace is relevant. Do not change without looking at the + rendered documentation. + */}} + {{ $artifactId := .Get 0 }} + {{ $version := .Get 1 }} + {{ $connectors := .Site.Data.sql_connectors }} + {{ $connector := index $connectors $artifactId }} + + {{ $path := .Page.Path }} + + {{ $hash := md5 now }} + + {{ if $.Site.Params.IsStable }} + <table style="display:table;margin-left:auto;margin-right:auto" id="download-table"> + <thead> + <th style="text-align:left">Maven dependency</th> + <th style="text-align:left">SQL Client</th> + </thead> + <tbody> + {{ range $connector.versions }} + <tr> + <td style="text-align: left"> + <div id="{{ $hash }}" onclick="selectTextAndCopy('{{ $hash }}')"class="highlight"> +<pre class="chroma"><code class="language-xml" data-lang="xml"><span class="nt"><dependency></span> + <span class="nt"><groupId></span>org.apache.flink<span class="nt"></groupId></span> + <span class="nt"><artifactId></span>{{- .maven -}}<span class="nt"></artifactId></span> + <span class="nt"><version></span>{{- .version -}}-{{- site.Params.VersionTitle -}}<span class="nt"></version></span> +<span class="nt"></dependency></span></code></pre></div> + <div class="book-hint info" style="text-align:center;display:none" copyable="flink-module" copyattribute="{{ $hash }}"> + Copied to clipboard! + </div> + </td> + <td style="text-align: left"> + <a href="{{- partial "docs/interpolate" .sql_url -}}">Download</a> + </td> + </tr> + {{ end }} + </tbody> + </table> + {{ else }} + Only available for stable versions. + {{ end }} + + + \ No newline at end of file diff --git a/docs/setup_docs.sh b/docs/setup_docs.sh index 01230c1c7a2..083d4b6b771 100755 --- a/docs/setup_docs.sh +++ b/docs/setup_docs.sh @@ -33,11 +33,27 @@ EOF echo "Created temporary file" $goModFileLocation/go.mod # Make Hugo retrieve modules which are used for externally hosted documentation -currentBranch=$(git branch --show-current) +currentBranch=$(git rev-parse --abbrev-ref HEAD) -if [[ ! "$currentBranch" =~ ^release- ]] || [[ -z "$currentBranch" ]]; then - # If the current branch is master or not provided, get the documentation from the main branch - $(command -v hugo) mod get -u github.com/apache/flink-connector-elasticsearch/docs@main - # Since there's no documentation yet available for a release branch, - # we only get the documentation from the main branch -fi +function integrate_connector_docs { + connector=$1 + ref=$2 + git clone --single-branch --branch ${ref} https://github.com/apache/flink-connector-${connector} + theme_dir="../themes/connectors" + mkdir -p "${theme_dir}" + rsync -a flink-connector-${connector}/docs/content* "${theme_dir}/" +} + +# Integrate the connector documentation + +rm -rf themes/connectors/* +rm -rf tmp +mkdir tmp +cd tmp + +# Since there's no documentation yet available for a release branch, +# we only get the documentation from the main branch +integrate_connector_docs aws v3.0.0 + +cd .. +rm -rf tmp
