This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 3b3debf5a4 [build] Fix nested resource path error when as maven
project from eclipse (#10427)
3b3debf5a4 is described below
commit 3b3debf5a4ae6a49836f84d8e921f385f949c288
Author: Compilation Success <[email protected]>
AuthorDate: Fri Jul 1 18:03:54 2022 +0800
[build] Fix nested resource path error when as maven project from eclipse
(#10427)
1. Fix nested resource path error when as maven project from eclipse
2. Add instructions of "Eclipse import FE as maven project" in developer
guide
---
.../en/developer/developer-guide/fe-eclipse-dev.md | 61 +++++++++++++++++++
.../developer/developer-guide/fe-eclipse-dev.md | 69 +++++++++++++++++++++-
fe/fe-core/pom.xml | 8 +++
3 files changed, 135 insertions(+), 3 deletions(-)
diff --git a/docs/en/developer/developer-guide/fe-eclipse-dev.md
b/docs/en/developer/developer-guide/fe-eclipse-dev.md
index 1445ec24b3..dbbb130a80 100644
--- a/docs/en/developer/developer-guide/fe-eclipse-dev.md
+++ b/docs/en/developer/developer-guide/fe-eclipse-dev.md
@@ -36,6 +36,8 @@ under the License.
The FE module requires part of the generated code, such as Thrift, Protobuf,
Jflex, CUP and other frameworks.
+#### Windows
+
1. Under Linux, enter the source code directory `fe` and execute the following
command:
```
@@ -53,8 +55,43 @@ The FE module requires part of the generated code, such as
Thrift, Protobuf, Jfl
cd /path/to/doris/fe/fe-core/target/ && tar xzf java.tar.gz
```
+#### MacOS
+
+We can use maven to do code generation or in other words build FE project
+directly on MacOS.
+
+1. Install apache thrift from source code or `brew` if you don't have one
+2. Create a directory `thirdparty/installed/bin`, and link command thrift into
+ it.
+
+ ```
+ mkdir -p thirdparty/installed/bin
+ ln -s ${thrift_installed_full_path} thirdparty/installed/bin/thrift
+ ```
+3. Call `maven` to build FE project, if something went wrong, check you
+ `$JAVA_HOME`, java version and newly installed command `thrift` work
+ correctly.
+
+ ```
+ cd fe && mvn package -DskipTests=true -Dos.arch=x86_64
+ ```
+
+Option `-Dos.arch=x86_64` in step 3 is for compatibility of Apple's M series
+CPU.
+
+Note:
+0. cup and jfex use java jar to do code generation, the process is platform
+ independent.
+1. Code generation of protobuf is done by `protoc-jar-maven-plugin`, which
uses
+ precompiled binaries of different archs to make it seems platform
independent.
+2. thrift is the only one relies on `thirdparty/installed`, which needs to be
+ built from source. We will make it independent on
`thirdparty/installed`
+ someday (TODO).
+
## Import FE project
+### Import as eclipse project
+
1. In the `fe/` directory of the development environment, execute the
following command to generate the Eclipse project file:
`cd /path/to/doris/fe/ && mvn -npr eclipse:eclipse -Dskip.plugin=true`
@@ -74,6 +111,18 @@ At this point, FE project import is complete. The project
directory in Eclipse i
+### Import as maven project
+
+We are able to build FE with maven, if we have done all the operations related
+to MacOS. And we are now of course able to import FE project as a maven project
+in eclipse.
+
+In eclipse menu `File` select `Import -> Maven -> Existing Maven Projects`,
+Choose doris fe directory to finish import. It's recommended to use working set
+to manage the all modules of FE.
+
+Enjoy developing and debugging FE in eclipse!
+
## Run Unit Test
Right-click on the unit test file you want to run and select `Run As -> JUnit
Test`. (If you want to debug, select `Debug As -> JUnit Test`).
@@ -128,6 +177,7 @@ To run a UT, a FE service will be started at first. And
then, UT cases execute a
## Code Update
+### Imported as eclipse project
1. Update lexical and grammar files or proto and thrift files
If you modified `fe/src/main/cup/sql_parser.cup` or
`fe/src/main/jflex/sql_scanner.flex` file or proto and thrift files. You need
to execute the following commands in the `fe/` directory:
@@ -146,6 +196,17 @@ To run a UT, a FE service will be started at first. And
then, UT cases execute a
Then refresh the project in Eclipse. If it cannot be updated, it is
recommended to delete the project and import it again according to this
document.
+### Imported as maven project
+
+1. Update lexical and grammar files or proto and thrift files
+ ```
+ cd fe && mvn package -DskipTests=true -Dos.arch=x86_64
+ ```
+2. Update maven dependencies, in eclipse `Package Explorer` right click on the
+ project `maven -> update project...`
+
+3. Refresh project in eclipse
+
## Imports Order
In order to maintain the Imports order of Java, please perform the following
operations to set the Imports Order of the project.
diff --git a/docs/zh-CN/developer/developer-guide/fe-eclipse-dev.md
b/docs/zh-CN/developer/developer-guide/fe-eclipse-dev.md
index 099a2ab970..dc2b155982 100644
--- a/docs/zh-CN/developer/developer-guide/fe-eclipse-dev.md
+++ b/docs/zh-CN/developer/developer-guide/fe-eclipse-dev.md
@@ -36,11 +36,14 @@ under the License.
FE 模块需要部分生成代码，如 Thrift、Protobuf, jflex, cup 等框架的生成代码。这部分需要在 Linux 或者 Mac环境生成。
+#### windows下开发获取生成代码步骤
+
1. 在 Linux 下， 进入 `fe 目录下执行以下命令：
+
```
mvn generate-sources
```
-
+
2. 如果使用window开发 需要将生成的 `fe/fe-core/target/generated-sources` 目录打包：
`fe/fe-core/target/ && tar czf java.tar.gz generated-sources/`
@@ -52,12 +55,47 @@ FE 模块需要部分生成代码，如 Thrift、Protobuf, jflex, cup 等框架
cd /path/to/doris/fe/fe-core/target/ && tar xzf java.tar.gz
```
+#### mac下开发获取生成代码步骤
+
+mac下可以直接使用maven构建的步骤生成代码, 或者说可以直接编译.
+
+1. 安装thrift的解释器(0.13.0), 如果没有的话到官网下载源码进行编译安装或者直接时
+ 使用`brew` 安装一个.
+
+2. 创建一个文件夹`thirdparty/installed/bin`, 然后将thrift 命令建立一个软链到这个
+ 路径下(当然你可以copy二进制).
+
+ ```
+ mkdir -p thirdparty/installed/bin
+ ln -s ${thrift_installed_full_path} thirdparty/installed/bin/thrift
+ ```
+
+3. 调用maven直接进行构建, 如果出现一些错误请检查`$JAVA_HOME`路径以及java版本以及
+ thrift是否能正常正确运行.
+
+ ```
+ cd fe && mvn package -DskipTests=true -Dos.arch=x86_64
+ ```
+
+上述第3步中`-Dos.arch=x86_64` 是为了兼容苹果的m系列处理器(`os.arch=aarch64`),
+protobuf会使用x86_64架构的protoc二进制进行代码生成, 如果是使用m系列处理器的mac,
+有roseta做兼容所以不会有问题.
+
+Note:
+0. cup和jfex均使用java的jar包程序进行编译, 代码生成的流程可以平台无关
+1. protobuf文件使用了现成的开源插件`protoc-jar-maven-plugin`进行跨平台的生成,
+ 本质上是下载已经编译好的对应平台二进制, 进行protobuf代码生成.
+2. thrift是目前(2022-06-26-Sun) FE在maven构建上唯一一个依赖
+ `thirdparty/installed`的工具. 目前还没有使用类似protobuf的生成插件替换(TODO).
+
## 导入 FE 工程
+### 使用eclipse工程导入
+
1. 在开发环境的 `fe/` 目录下，执行以下命令生成 Eclipse 工程文件：
`cd /path/to/doris/fe/ && mvn -npr eclipse:eclipse -Dskip.plugin=true`
-
+
执行完成后，会在 `fe/` 目录下生成 `.project` 和 `.classpath` 文件
2. 导入 FE 工程
@@ -69,10 +107,22 @@ FE 模块需要部分生成代码，如 Thrift、Protobuf, jflex, cup 等框架
* 在 `Java Build Path` 对话框中，选择 `Source` 标签页，点击 `Add Folder`，勾选添加之前拷贝并解压的
`java/` 目录。
* 点击 `Apply and Close` 完成。
+
至此，FE 导入完成。Eclipse 中的工程目录大致如下：
+
+### 使用maven工程导入
+
+经过前边mac相关的操作之后, 我们应该能够直接本地maven构建了. 能够maven构建的项目
+是可以使用eclipse m2e 插件直接导入的.
+
+在eclipse File 菜单中依次选择`Import -> Maven -> Existing Maven Projects`
+然后选择doris fe文件夹即可完成导入. 导入时建议选择working set管理FE的多个module.
+
+至此, 我们已经可以使用eclipse进行FE的开发调试.
+
## 运行单元测试
在想要运行的单元测试文件上右击，选择 `Run As -> JUnit Test`。（如果要单步调试，则选择 `Debug As -> JUnit
Test`）。
@@ -99,7 +149,7 @@ java.lang.Exception: Method xxxx should have no parameters
之后在运行 `Run/Debug` 即可。
-FE的单元测试会首先启动一个FE服务，然后由测试用例作为客户端执行相应的测试逻辑。在UT报错时，UT的日志只会打印相应的客户端日志，如果需要**查看服务端日志**，可以在路径${DORIS_HOME}/fe/mocked下查看。
+FE的单元测试会首先启动一个FE服务，然后由测试用例作为客户端执行相应的测试逻辑。在UT报错时，UT的日志只会打印相应的客户端日志，如果需要**查看服务端日志**，可以在路径`${DORIS_HOME}/fe/mocked`下查看。
## 运行 FE
@@ -127,6 +177,8 @@ FE的单元测试会首先启动一个FE服务，然后由测试用例作为客
## 代码更新
+### eclipse工程
+
1. 更新词法、语法文件或者thrift 和proto 文件
如果修改了 `fe/fe-core/src/main/cup/sql_parser.cup` 或者
`fe/fe-core/src/main/jflex/sql_scanner.flex`文件或者proto 和thrift 文件。则需在 `fe`
目录下执行以下命令：
@@ -145,6 +197,17 @@ FE的单元测试会首先启动一个FE服务，然后由测试用例作为客
之后在 Eclipse 中刷新工程即可。如无法更新，建议删除工程，并按照该文档重新导入一遍即可。
+### maven工程
+
+1. 更新词法、语法文件或者thrift 和proto 文件 在fe目录下命令行执行一次
+ ```
+ cd fe && mvn package -DskipTests=true -Dos.arch=x86_64
+ ```
+2. 更新maven依赖, 直接在eclipse里`Package Explorer` 右键选中maven项目
+ `maven -> update project...`
+
+3. 在eclipse中刷新工程.
+
## Import 顺序
为了保持 Java 的 Import 顺序，请执行如下操作设定项目的 Import Order
diff --git a/fe/fe-core/pom.xml b/fe/fe-core/pom.xml
index 0fd782241f..ccbc7e9c9e 100644
--- a/fe/fe-core/pom.xml
+++ b/fe/fe-core/pom.xml
@@ -652,6 +652,14 @@ under the License.
</dependencies>
<build>
<finalName>doris-fe</finalName>
+ <resources>
+ <resource>
+ <directory>target/generated-sources</directory>
+ <excludes>
+ <exclude>cup</exclude>
+ </excludes>
+ </resource>
+ </resources>
<plugins>
<!--jcup-->
<plugin>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
