This is an automated email from the ASF dual-hosted git repository.
ulyssesyou pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-kyuubi.git
The following commit(s) were added to refs/heads/master by this push:
new aad575c [KYUUBI #1329] Support download Flink binary package in
kyuubi-download module
aad575c is described below
commit aad575ce3e9121f2adc6d5280a01e66383e2fe67
Author: yanghua <[email protected]>
AuthorDate: Fri Nov 5 18:03:57 2021 +0800
[KYUUBI #1329] Support download Flink binary package in kyuubi-download
module
…
<!--
Thanks for sending a pull request!
Here are some tips for you:
1. If this is your first time, please read our contributor guidelines:
https://kyuubi.readthedocs.io/en/latest/community/contributions.html
2. If the PR is related to an issue in
https://github.com/apache/incubator-kyuubi/issues, add '[KYUUBI #XXXX]' in your
PR title, e.g., '[KYUUBI #XXXX] Your PR title ...'.
3. If the PR is unfinished, add '[WIP]' in your PR title, e.g.,
'[WIP][KYUUBI #XXXX] Your PR title ...'.
-->
### _Why are the changes needed?_
<!--
Please clarify why the changes are needed. For instance,
1. If you add a feature, you can talk about the use case of it.
2. If you fix a bug, you can clarify why it is a bug.
-->
### _How was this patch tested?_
- [ ] Add some test cases that check the changes thoroughly including
negative and positive cases if possible
- [ ] Add screenshots for manual tests if appropriate
- [ ] [Run
test](https://kyuubi.readthedocs.io/en/latest/develop_tools/testing.html#running-tests)
locally before make a pull request
Closes #1339 from yanghua/KYUUBI-1329.
Closes #1329
895815fb [yanghua] address review suggestion
eddf0605 [yanghua] [KYUUBI #1329] Support download Flink binary package in
kyuubi-download module
Authored-by: yanghua <[email protected]>
Signed-off-by: ulysses-you <[email protected]>
---
docs/develop_tools/building.md | 11 ++++++-----
externals/kyuubi-download/pom.xml | 25 +++++++++++++++++++++++++
pom.xml | 12 ++++++++++++
3 files changed, 43 insertions(+), 5 deletions(-)
diff --git a/docs/develop_tools/building.md b/docs/develop_tools/building.md
index e24ae97..1b936d7 100644
--- a/docs/develop_tools/building.md
+++ b/docs/develop_tools/building.md
@@ -73,11 +73,12 @@ Profile | Default | Since
-Pspark-3.1 | No | 1.1.0
-## Defining the Apache Mirror for Spark
+## Defining the Apache Mirror for Spark or Flink
-By default, we use `https://archive.apache.org/dist/spark/` to download the
built-in Spark release package,
+By default, we use `https://archive.apache.org/dist/spark/` to download the
built-in Spark or
+use `https://archive.apache.org/dist/flink/` to download the built-in Flink
release package,
but if you find it hard to reach, or the downloading speed is too slow, you
can define the `spark.archive.mirror`
-property to a suitable Apache mirror site. For instance,
+or `flink.archive.mirror` property to a suitable Apache mirror site. For
instance,
```bash
build/mvn clean package
-Dspark.archive.mirror=https://mirrors.bfsu.edu.cn/apache/spark/spark-3.0.1
@@ -85,8 +86,8 @@ build/mvn clean package
-Dspark.archive.mirror=https://mirrors.bfsu.edu.cn/apach
Visit [Apache Mirrors](http://www.apache.org/mirrors/) and choose a mirror
based on your region.
-Specifically for developers in China mainland, you can use the pre-defined
profile named `mirror-cn` which use
-`mirrors.bfsu.edu.cn` to speed up Spark Binary downloading. For instance,
+Specifically for developers in China mainland, you can use the pre-defined
profile named `mirror-cn`
+ which use `mirrors.bfsu.edu.cn` to speed up Spark Binary downloading. For
instance,
```bash
build/mvn clean package -Pmirror-cn
diff --git a/externals/kyuubi-download/pom.xml
b/externals/kyuubi-download/pom.xml
index a5613c5..57b7ad0 100644
--- a/externals/kyuubi-download/pom.xml
+++ b/externals/kyuubi-download/pom.xml
@@ -58,6 +58,31 @@
</executions>
</plugin>
+ <!-- Download flink archive -->
+ <plugin>
+ <groupId>com.googlecode.maven-download-plugin</groupId>
+ <artifactId>download-maven-plugin</artifactId>
+ <configuration>
+ <skip>${flink.archive.download.skip}</skip>
+ </configuration>
+ <executions>
+ <execution>
+ <id>download-flink-release</id>
+ <phase>compile</phase>
+ <goals>
+ <goal>wget</goal>
+ </goals>
+ <configuration>
+
<url>${flink.archive.mirror}/${flink.archive.name}</url>
+
<outputDirectory>${project.build.directory}</outputDirectory>
+ <readTimeOut>60000</readTimeOut>
+ <retries>3</retries>
+ <unpack>true</unpack>
+ </configuration>
+ </execution>
+ </executions>
+ </plugin>
+
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-deploy-plugin</artifactId>
diff --git a/pom.xml b/pom.xml
index b9dd3f7..f043588 100644
--- a/pom.xml
+++ b/pom.xml
@@ -97,6 +97,10 @@
<commons-lang3.version>3.10</commons-lang3.version>
<curator.version>2.12.0</curator.version>
<delta.version>1.0.0</delta.version>
+ <flink.version>1.12.5</flink.version>
+
<flink.archive.name>flink-${flink.version}-bin-scala_${scala.binary.version}.tgz</flink.archive.name>
+
<flink.archive.mirror>https://archive.apache.org/dist/flink/flink-${flink.version}</flink.archive.mirror>
+ <flink.archive.download.skip>false</flink.archive.download.skip>
<google.jsr305.version>3.0.2</google.jsr305.version>
<guava.version>30.1-jre</guava.version>
<hadoop.version>3.3.1</hadoop.version>
@@ -1371,6 +1375,7 @@
<id>mirror-cn</id>
<properties>
<spark.archive.mirror>https://mirrors.bfsu.edu.cn/apache/spark/spark-${spark.version}</spark.archive.mirror>
+
<flink.archive.mirror>https://mirrors.bfsu.edu.cn/apache/flink/flink-${flink.version}</flink.archive.mirror>
</properties>
</profile>
@@ -1477,6 +1482,13 @@
</profile>
<profile>
+ <id>flink-provided</id>
+ <properties>
+ <flink.archive.download.skip>true</flink.archive.download.skip>
+ </properties>
+ </profile>
+
+ <profile>
<id>spark-block-cleaner</id>
<modules>
<module>tools/spark-block-cleaner</module>
