This is an automated email from the ASF dual-hosted git repository.
lzljs3620320 pushed a commit to branch release-1.12
in repository https://gitbox.apache.org/repos/asf/flink.git
The following commit(s) were added to refs/heads/release-1.12 by this push:
new b1eb0d1 [FLINK-20292][doc] Improve the document about transforming
connector/format SPI resource files
b1eb0d1 is described below
commit b1eb0d11b0fac8c0292324dce54fa37ce46ecb0e
Author: Leonard Xu <[email protected]>
AuthorDate: Thu Dec 3 16:09:11 2020 +0800
[FLINK-20292][doc] Improve the document about transforming connector/format
SPI resource files
This closes #14229
---
docs/dev/table/connectors/index.md | 59 +++++++++++++++++++++++++++++++++++
docs/dev/table/connectors/index.zh.md | 58 ++++++++++++++++++++++++++++++++++
2 files changed, 117 insertions(+)
diff --git a/docs/dev/table/connectors/index.md
b/docs/dev/table/connectors/index.md
index 775d1824..6622cab 100644
--- a/docs/dev/table/connectors/index.md
+++ b/docs/dev/table/connectors/index.md
@@ -139,6 +139,65 @@ are taken into account when searching for exactly one
matching factory for each
If no factory can be found or multiple factories match for the given
properties, an exception will be
thrown with additional information about considered factories and supported
properties.
+
+Transform table connector/format resources
+--------
+
+Flink uses Java's [Service Provider Interfaces
(SPI)](https://docs.oracle.com/javase/tutorial/sound/SPI-intro.html) to load
the table connector/format factories by their identifiers. Since the SPI
resource file named `org.apache.flink.table.factories.Factory` for every table
connector/format is under the same directory `META-INF/services`, these
resource files will override each other when build the uber-jar of the project
which uses more than one table connector/format, which will cause [...]
+
+In this situation, the recommended way is transforming these resource files
under the directory `META-INF/services` by
[ServicesResourceTransformer](https://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html)
of maven shade plugin. Given the pom.xml file content of example that contains
connector `flink-sql-connector-hive-3.1.2` and format `flink-parquet` in a
project.
+
+{% highlight xml %}
+
+ <modelVersion>4.0.0</modelVersion>
+ <groupId>org.example</groupId>
+ <artifactId>myProject</artifactId>
+ <version>1.0-SNAPSHOT</version>
+
+ <dependencies>
+ <!-- other project dependencies ...-->
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+
<artifactId>flink-sql-connector-hive-3.1.2_${scala.binary.version}</artifactId>
+ <version>${flink-version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-parquet_${scala.binary.version}</artifactId>
+ <version>${flink-version}</version>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>shade</id>
+ <phase>package</phase>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ <configuration>
+ <transformers combine.children="append">
+ <!-- The service transformer is needed to
merge META-INF/services files -->
+ <transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
+ <!-- ... -->
+ </transformers>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+{% endhighlight %}
+
+After configured the `ServicesResourceTransformer`, the table connector/format
resource files under the directory `META-INF/services` would be merged rather
than overwritten each other when build the uber-jar of above project.
+
{% top %}
Schema Mapping
diff --git a/docs/dev/table/connectors/index.zh.md
b/docs/dev/table/connectors/index.zh.md
index 559f2b7..850d544 100644
--- a/docs/dev/table/connectors/index.zh.md
+++ b/docs/dev/table/connectors/index.zh.md
@@ -139,6 +139,64 @@ are taken into account when searching for exactly one
matching factory for each
If no factory can be found or multiple factories match for the given
properties, an exception will be
thrown with additional information about considered factories and supported
properties.
+Transform table connector/format resources
+--------
+
+Flink uses Java's [Service Provider Interfaces
(SPI)](https://docs.oracle.com/javase/tutorial/sound/SPI-intro.html) to load
the table connector/format factories by their identifiers. Since the SPI
resource file named `org.apache.flink.table.factories.Factory` for every table
connector/format is under the same directory `META-INF/services`, these
resource files will override each other when build the uber-jar of the project
which uses more than one table connector/format, which will cause [...]
+
+In this situation, the recommended way is transforming these resource files
under the directory `META-INF/services` by
[ServicesResourceTransformer](https://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html)
of maven shade plugin. Given the pom.xml file content of example that contains
connector `flink-sql-connector-hive-3.1.2` and format `flink-parquet` in a
project.
+
+{% highlight xml %}
+
+ <modelVersion>4.0.0</modelVersion>
+ <groupId>org.example</groupId>
+ <artifactId>myProject</artifactId>
+ <version>1.0-SNAPSHOT</version>
+
+ <dependencies>
+ <!-- other project dependencies ...-->
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+
<artifactId>flink-sql-connector-hive-3.1.2_${scala.binary.version}</artifactId>
+ <version>${flink-version}</version>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.flink</groupId>
+ <artifactId>flink-parquet_${scala.binary.version}</artifactId>
+ <version>${flink-version}</version>
+ </dependency>
+
+ </dependencies>
+
+ <build>
+ <plugins>
+ <plugin>
+ <groupId>org.apache.maven.plugins</groupId>
+ <artifactId>maven-shade-plugin</artifactId>
+ <executions>
+ <execution>
+ <id>shade</id>
+ <phase>package</phase>
+ <goals>
+ <goal>shade</goal>
+ </goals>
+ <configuration>
+ <transformers combine.children="append">
+ <!-- The service transformer is needed to
merge META-INF/services files -->
+ <transformer
implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
+ <!-- ... -->
+ </transformers>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+ </plugins>
+ </build>
+{% endhighlight %}
+
+After configured the `ServicesResourceTransformer`, the table connector/format
resource files under the directory `META-INF/services` would be merged rather
than overwritten each other when build the uber-jar of above project.
+
{% top %}
Schema Mapping
