This is an automated email from the ASF dual-hosted git repository.
qiaojialin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-iotdb.git
The following commit(s) were added to refs/heads/master by this push:
new 2395075 Add design documents (#757)
2395075 is described below
commit 23950758a5859306aa0ec28cb1c0118ba02260fb
Author: Jialin Qiao <[email protected]>
AuthorDate: Mon Feb 3 15:10:50 2020 +0800
Add design documents (#757)
* add design doc and instructions for inserting figures
Co-authored-by: Zesong Sun <[email protected]>
---
.travis.yml | 2 +-
docs/{ => Community}/Community-History&Vision.md | 0
docs/{ => Community}/Community-Powered By.md | 0
.../Community-Project Committers.md | 0
docs/{ => Development}/Development-Chinese.md | 0
docs/{ => Development}/Development-Contributing.md | 3 +
docs/{ => Development}/Development-Document.md | 3 +
docs/{ => Development}/Development-IDE.md | 0
docs/{ => Development}/Development-VoteRelease.md | 0
.../SystemDesign/0-Architecture/1-Architecture.md | 55 +++++++++++++
.../1-TsFile/1-TsFile.md} | 11 ++-
.../1-TsFile/2-Format.md} | 7 +-
.../SystemDesign/1-TsFile/3-Write.md | 65 +++++++++++++++
.../SystemDesign/1-TsFile/4-Read.md} | 7 +-
.../SystemDesign/2-QueryEngine/1-QueryEngine.md | 64 +++++++++++++++
.../SystemDesign/2-QueryEngine/2-Planner.md | 63 +++++++++++++++
.../SystemDesign/2-QueryEngine/3-PlanExecutor.md} | 6 +-
.../3-SchemaManager/1-SchemaManager.md} | 6 +-
.../4-StorageEngine/1-StorageEngine.md | 92 ++++++++++++++++++++++
.../SystemDesign/4-StorageEngine/2-WAL.md} | 6 +-
.../SystemDesign/4-StorageEngine/3-FlushManager.md | 84 ++++++++++++++++++++
.../4-StorageEngine/4-MergeManager.md} | 6 +-
.../SystemDesign/5-DataQuery/1-DataQuery.md | 67 ++++++++++++++++
.../2-Files.md => 8-Architecture/1-Files.md} | 0
.../2-Writing Data on HDFS.md} | 0
.../3-Shared Nothing Cluster.md} | 0
.../SystemDesign/0-Content.md} | 24 +++++-
.../SystemDesign/1-TsFile/1-TsFile.md} | 10 ++-
.../1-TsFile/2-Format.md} | 6 +-
.../SystemDesign/1-TsFile/3-Write.md} | 4 +-
.../SystemDesign/1-TsFile/4-Read.md} | 4 +-
.../{Developers => SystemDesign}/release.adoc | 0
docs/Documentation/UserGuide/0-Content.md | 9 +--
.../2-Files.md => 8-Architecture/1-Files.md} | 0
.../2-Writing Data on HDFS.md} | 0
.../3-Shared Nothing Cluster.md} | 0
pom.xml | 4 +
37 files changed, 569 insertions(+), 39 deletions(-)
diff --git a/.travis.yml b/.travis.yml
index 93ce19d..9d590e1 100644
--- a/.travis.yml
+++ b/.travis.yml
@@ -175,7 +175,7 @@ matrix:
# - mvn -B apache-rat:check
# - mvn -B clean package -pl
server,grafana,client,example,:kafka-example,:rocketmq-example -am
integration-test
- os: linux
-# if: fork = false #only fork=true (i.e., the committer has permission to
write the repo)
+ if: fork = false #only fork=true (i.e., the committer has permission to
write the repo)
name: sonar-analysis
dist: xenial
jdk: openjdk8
diff --git a/docs/Community-History&Vision.md
b/docs/Community/Community-History&Vision.md
similarity index 100%
copy from docs/Community-History&Vision.md
copy to docs/Community/Community-History&Vision.md
diff --git a/docs/Community-Powered By.md b/docs/Community/Community-Powered
By.md
similarity index 100%
rename from docs/Community-Powered By.md
rename to docs/Community/Community-Powered By.md
diff --git a/docs/Community-Project Committers.md
b/docs/Community/Community-Project Committers.md
similarity index 100%
rename from docs/Community-Project Committers.md
rename to docs/Community/Community-Project Committers.md
diff --git a/docs/Development-Chinese.md
b/docs/Development/Development-Chinese.md
similarity index 100%
rename from docs/Development-Chinese.md
rename to docs/Development/Development-Chinese.md
diff --git a/docs/Development-Contributing.md
b/docs/Development/Development-Contributing.md
similarity index 98%
rename from docs/Development-Contributing.md
rename to docs/Development/Development-Contributing.md
index 650d2fe..247a2e2 100644
--- a/docs/Development-Contributing.md
+++ b/docs/Development/Development-Contributing.md
@@ -95,6 +95,9 @@ Then,
* Step 1: create a new folder named "7-RoadMap", and add document named
`1-B.md` in folder "7-RoadMap";
* Step 2: modify `0-Content.md` file by adding "# Chapter 7: RoadMap" in
the end, and adding `* 1-B.md` in the list of this new chapter.
+If you need to insert **figures** into documents, you can firstly update the
figures in [this issue](https://github.com/thulab/iotdb/issues/543) for storing
pictures in IoTDB website or other MD files.
+Drag a picture and then quote the figure's URL link.
+
### Contributing Bug Reports
If you encounter a problem, try to search the mailing list and JIRA to check
whether other people have faced the same situation. If it is not reported
before, please report an issue.
diff --git a/docs/Development-Document.md
b/docs/Development/Development-Document.md
similarity index 90%
rename from docs/Development-Document.md
rename to docs/Development/Development-Document.md
index 52269b4..56677a0 100644
--- a/docs/Development-Document.md
+++ b/docs/Development/Development-Document.md
@@ -45,4 +45,7 @@ Then,
* Step 1: create a new folder named "7-RoadMap", and add document named
`1-B.md` in folder "7-RoadMap";
* Step 2: modify `0-Content.md` file by adding "# Chapter 7: RoadMap" in
the end, and adding `* 1-B.md` in the list of this new chapter.
+If you need to insert **figures** into documents, you can firstly update the
figures in [this issue](https://github.com/thulab/iotdb/issues/543) for storing
pictures in IoTDB website or other MD files.
+Drag a picture and then quote the figure's URL link.
+
> If you want to contribute more (for example, reviewing Changes, reporting
> bugs, or even being commiters), please refer to [this
> page](/#/Development/Contributing).
\ No newline at end of file
diff --git a/docs/Development-IDE.md b/docs/Development/Development-IDE.md
similarity index 100%
rename from docs/Development-IDE.md
rename to docs/Development/Development-IDE.md
diff --git a/docs/Development-VoteRelease.md
b/docs/Development/Development-VoteRelease.md
similarity index 100%
rename from docs/Development-VoteRelease.md
rename to docs/Development/Development-VoteRelease.md
diff --git
a/docs/Documentation-CHN/SystemDesign/0-Architecture/1-Architecture.md
b/docs/Documentation-CHN/SystemDesign/0-Architecture/1-Architecture.md
new file mode 100644
index 0000000..f026c71
--- /dev/null
+++ b/docs/Documentation-CHN/SystemDesign/0-Architecture/1-Architecture.md
@@ -0,0 +1,55 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+# 应用概览
+
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/19167280/73625222-ddd88680-467e-11ea-9098-e808ed4979c5.png";>
+
+物联网时序数据库 Apache IoTDB 的架构图如上所示，覆盖了对时序数据的采集、存储、查询、分析以及可视化等全生命周期的数据管理功能，其中灰色部分为
IoTDB 组件。
+
+## IoTDB 架构介绍
+
+IoTDB 采用客户端-服务器架构，如下图所示。
+
+<img style="width:100%; max-width:400px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/19167280/73625221-ddd88680-467e-11ea-9cf3-70367e5886f4.png";>
+
+其中服务器端主要包括查询引擎，用来处理用户的所有请求，并分发到对应的管理组件，包括数据写入层、数据查询、元数据管理、权限管理等模块。
+
+* [数据文件](../1-TsFile/1-TsFile.md)
+* [查询引擎](../2-QueryEngine/1-QueryEngine.md)
+* [元数据管理](../3-SchemaManager/1-MManager.md)
+* [存储引擎](../4-StorageEngine/1-StorageEngine.md)
+* [数据查询](../5-DataQuery/1-DataQuery.md)
+* [权限管理](../6-Administration/1-Administration.md)
+
+## 连接器
+
+IoTDB 与大数据系统进行了对接。
+
+* [Hadoop-TsFile](../7-Connector/1-Hadoop-TsFile.md)
+* [Hive-TsFile](../7-Connector/2-Hive-TsFile.md)
+* [Spark-TsFile](../7-Connector/3-Spark-TsFile.md)
+* [Spark-IoTDB](../7-Connector/4-Spark-IoTDB.md)
+* [Grafana](../7-Connector/5-Grafana.md)
+
+## 工具
+
+* [同步工具](../8-Tools/1-Sync.md)
\ No newline at end of file
diff --git a/docs/Documentation-CHN/UserGuide/8-System Design
(Developer)/4-Shared Nothing Cluster.md
b/docs/Documentation-CHN/SystemDesign/1-TsFile/1-TsFile.md
similarity index 79%
copy from docs/Documentation-CHN/UserGuide/8-System Design (Developer)/4-Shared
Nothing Cluster.md
copy to docs/Documentation-CHN/SystemDesign/1-TsFile/1-TsFile.md
index 1c0a79b..241d202 100644
--- a/docs/Documentation-CHN/UserGuide/8-System Design (Developer)/4-Shared
Nothing Cluster.md
+++ b/docs/Documentation-CHN/SystemDesign/1-TsFile/1-TsFile.md
@@ -19,8 +19,13 @@
-->
-# 第8章: 系统设计
+# 数据文件
-## Shared-nothing 架构
+TsFile 是 IoTDB 的底层数据文件，专门为时间序列数据设计的列式文件格式。
-正在开发中，预计2019年底发布，敬请期待。
+
+## 相关文档
+
+* [文件格式](2-Format.md)
+* [写流程](3-Write.md)
+* [读流程](4-Read.md)
diff --git a/docs/Documentation-CHN/UserGuide/8-System Design
(Developer)/1-Hierarchy.md
b/docs/Documentation-CHN/SystemDesign/1-TsFile/2-Format.md
similarity index 99%
rename from docs/Documentation-CHN/UserGuide/8-System Design
(Developer)/1-Hierarchy.md
rename to docs/Documentation-CHN/SystemDesign/1-TsFile/2-Format.md
index 70acc39..3db4efb 100644
--- a/docs/Documentation-CHN/UserGuide/8-System Design
(Developer)/1-Hierarchy.md
+++ b/docs/Documentation-CHN/SystemDesign/1-TsFile/2-Format.md
@@ -7,9 +7,9 @@
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@@ -19,7 +19,8 @@
-->
-# 第8章: 系统设计
+# TsFile 文件格式
+
## 1. TsFile 设计
diff --git a/docs/Documentation-CHN/SystemDesign/1-TsFile/3-Write.md
b/docs/Documentation-CHN/SystemDesign/1-TsFile/3-Write.md
new file mode 100644
index 0000000..fe2fd7f
--- /dev/null
+++ b/docs/Documentation-CHN/SystemDesign/1-TsFile/3-Write.md
@@ -0,0 +1,65 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+# TsFile 写流程
+
+* org.apache.iotdb.tsfile.write.*
+
+TsFile 的写入流程如下图所示：
+
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/19167280/73625238-efba2980-467e-11ea-927e-a7021f8153af.png";>
+
+其中，每个设备对应一个 ChunkGroupWriter，每个传感器对应一个 ChunkWriter。
+
+文件的写入主要分为三种操作，在图上用 1、2、3 标注
+
+* 1、写内存换冲区
+* 2、持久化 ChunkGroup
+* 3、关闭文件
+
+## 1、写内存缓冲区
+
+TsFile 文件层的写入接口有两种
+
+* TsFileWriter.write(TSRecord record)
+
+ 写入一个设备一个时间戳多个测点。
+
+* TsFileWriter.write(RowBatch rowBatch)
+
+ 写入一个设备多个时间戳多个测点。
+
+当调用 write 接口时，这个设备的数据会交给对应的 ChunkGroupWriter，其中的每个测点会交给对应的 ChunkWriter
进行写入。ChunkWriter 完成编码和打包（生成 Page）。
+
+
+## 2、持久化 ChunkGroup
+
+* TsFileWriter.flushAllChunkGroups()
+
+当内存中的数据达到一定阈值，会触发持久化操作。每次持久化会把当前内存中所有设备的数据全部持久化到磁盘的 TsFile 文件中。每个设备对应一个
ChunkGroup，每个测点对应一个 Chunk。
+
+持久化完成后会在内存中缓存对应的元数据信息，以供查询和生成文件尾部 metadata。
+
+## 3、关闭文件
+
+* TsFileWriter.close()
+
+根据内存中缓存的元数据，生成 TsFileMetadata 追加到文件尾部，最后关闭文件。
diff --git a/docs/Community-History&Vision.md
b/docs/Documentation-CHN/SystemDesign/1-TsFile/4-Read.md
similarity index 93%
copy from docs/Community-History&Vision.md
copy to docs/Documentation-CHN/SystemDesign/1-TsFile/4-Read.md
index 64fcf07..0233344 100644
--- a/docs/Community-History&Vision.md
+++ b/docs/Documentation-CHN/SystemDesign/1-TsFile/4-Read.md
@@ -19,6 +19,9 @@
-->
-# History&Vision
+# TsFile 读流程
+
+* org.apache.iotdb.tsfile.read.*
+
+
-Comming Soon.
diff --git a/docs/Documentation-CHN/SystemDesign/2-QueryEngine/1-QueryEngine.md
b/docs/Documentation-CHN/SystemDesign/2-QueryEngine/1-QueryEngine.md
new file mode 100644
index 0000000..b156870
--- /dev/null
+++ b/docs/Documentation-CHN/SystemDesign/2-QueryEngine/1-QueryEngine.md
@@ -0,0 +1,64 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+# 查询引擎
+
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/19167280/73625242-f648a100-467e-11ea-921c-b954a3ecae7a.png";>
+
+## 设计思想
+
+查询引擎负责所有用户命令的解析、生成计划、交给对应的执行器、返回结果集。
+
+## 相关类
+
+* org.apache.iotdb.db.service.TSServiceImpl
+
+ IoTDB 服务器端 RPC 实现，与客户端进行直接交互。
+
+* org.apache.iotdb.db.qp.Planner
+
+ 解析 SQL，生成逻辑计划，逻辑优化，生成物理计划。
+
+* org.apache.iotdb.db.qp.executor.PlanExecutor
+
+ 分发物理计划给对应的执行器，主要包括以下四个具体的执行器。
+
+ * MManager: 元数据操作
+ * StorageEngine: 数据写入
+ * QueryRouter: 数据查询
+ * LocalFileAuthorizer: 权限操作
+
+* org.apache.iotdb.db.query.dataset.*
+
+ 分批构造结果集返回给客户端，包含部分查询逻辑。
+
+## 查询流程
+
+* SQL 解析
+* 生成逻辑计划
+* 生成物理计划
+* 构造结果集生成器
+* 分批返回结果集
+
+## 相关文档
+
+* [查询计划生成器](2-Planner.md)
+* [计划执行器](3-PlanExecutor.md)
\ No newline at end of file
diff --git a/docs/Documentation-CHN/SystemDesign/2-QueryEngine/2-Planner.md
b/docs/Documentation-CHN/SystemDesign/2-QueryEngine/2-Planner.md
new file mode 100644
index 0000000..82640d0
--- /dev/null
+++ b/docs/Documentation-CHN/SystemDesign/2-QueryEngine/2-Planner.md
@@ -0,0 +1,63 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+# 执行计划生成器 Planner
+
+* org.apache.iotdb.db.qp.Planner
+
+将 SQL 解析出的语法树转化成逻辑计划，逻辑优化，物理计划。
+
+## SQL 解析
+
+SQL 解析采用 Antlr4
+
+* server/src/main/antlr4/org/apache/iotdb/db/qp/strategy/SqlBase.g4
+
+mvn clean compile 之后生成代码位置：server/target/generated-sources/antlr4
+
+## 逻辑计划生成器
+
+* org.apache.iotdb.db.qp.strategy.LogicalGenerator
+
+## 逻辑计划优化器
+
+目前有三种逻辑计划优化器
+
+* org.apache.iotdb.db.qp.strategy.optimizer.ConcatPathOptimizer
+
+ 路径优化器，将 SQL 中的查询路径进行拼接，与 MManager 进行交互去掉通配符，进行路径检查。
+
+* org.apache.iotdb.db.qp.strategy.optimizer.RemoveNotOptimizer
+
+ 谓词去非优化器，将谓词逻辑中的非操作符去掉。
+
+* org.apache.iotdb.db.qp.strategy.optimizer.DnfFilterOptimizer
+
+ 将谓词转化为析取范式。
+
+* org.apache.iotdb.db.qp.strategy.optimizer.MergeSingleFilterOptimizer
+
+ 将相同路径的谓词逻辑合并。
+
+## 物理计划生成器
+
+* org.apache.iotdb.db.qp.strategy.PhysicalGenerator
+
diff --git a/docs/Community-History&Vision.md
b/docs/Documentation-CHN/SystemDesign/2-QueryEngine/3-PlanExecutor.md
similarity index 90%
copy from docs/Community-History&Vision.md
copy to docs/Documentation-CHN/SystemDesign/2-QueryEngine/3-PlanExecutor.md
index 64fcf07..529ef45 100644
--- a/docs/Community-History&Vision.md
+++ b/docs/Documentation-CHN/SystemDesign/2-QueryEngine/3-PlanExecutor.md
@@ -19,6 +19,8 @@
-->
-# History&Vision
+# 计划执行器
-Comming Soon.
+* org.apache.iotdb.db.qp.executor.PlanExecutor
+
+执行物理计划。
\ No newline at end of file
diff --git a/docs/Documentation-CHN/UserGuide/8-System Design
(Developer)/4-Shared Nothing Cluster.md
b/docs/Documentation-CHN/SystemDesign/3-SchemaManager/1-SchemaManager.md
similarity index 70%
copy from docs/Documentation-CHN/UserGuide/8-System Design (Developer)/4-Shared
Nothing Cluster.md
copy to docs/Documentation-CHN/SystemDesign/3-SchemaManager/1-SchemaManager.md
index 1c0a79b..08e9971 100644
--- a/docs/Documentation-CHN/UserGuide/8-System Design (Developer)/4-Shared
Nothing Cluster.md
+++ b/docs/Documentation-CHN/SystemDesign/3-SchemaManager/1-SchemaManager.md
@@ -19,8 +19,8 @@
-->
-# 第8章: 系统设计
+# 元数据管理
-## Shared-nothing 架构
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/19167280/73625246-fc3e8200-467e-11ea-8815-67b9c4ab716e.png";>
-正在开发中，预计2019年底发布，敬请期待。
+IoTDB 的元数据管理采用目录树的形式，倒数第二层为设备层，最后一层为传感器层。
\ No newline at end of file
diff --git
a/docs/Documentation-CHN/SystemDesign/4-StorageEngine/1-StorageEngine.md
b/docs/Documentation-CHN/SystemDesign/4-StorageEngine/1-StorageEngine.md
new file mode 100644
index 0000000..1d073b4
--- /dev/null
+++ b/docs/Documentation-CHN/SystemDesign/4-StorageEngine/1-StorageEngine.md
@@ -0,0 +1,92 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+# 存储引擎
+
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/19167280/73625255-03fe2680-467f-11ea-91ae-64407ef1125c.png";>
+
+## 设计思想
+
+存储引擎基于 LSM 设计。数据首先写入内存缓冲区 memtable
中，再刷到磁盘。内存中为每个设备维护当前持久化的（包括已经落盘的和正在持久化的）最大时间戳，根据这个时间戳将数据区分为顺序数据和乱序数据，不同种类的数据通过不同的
memtable 和 TsFile 管理。
+
+每个数据文件 TsFile 在内存中对应一个文件索引信息 TsFileResource，供查询使用。
+
+此外，存储引擎还包括异步持久化和文件合并机制。
+
+## 写入流程
+
+### 相关代码
+
+* org.apache.iotdb.db.engine.StorageEngine
+
+ 负责一个 IoTDB 实例的写入和访问，管理所有的 StorageGroupProsessor。
+
+* org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor
+
+ 负责一个存储组一个时间分区内的数据写入和访问。管理所有分区的TsFileProcessor。
+
+* org.apache.iotdb.db.engine.storagegroup.TsFileProcessor
+
+ 负责一个 TsFile 文件的数据写入和访问。
+
+
+### 单行数据（一个设备一个时间戳多个值）写入
+
+* 对应的接口
+ * JDBC 的 execute 和 executeBatch 接口
+ * Session 的 insert 和 insertInBatch
+
+* 总入口: public void insert(InsertPlan insertPlan)
+ * 找到对应的 StorageGroupProsessor
+ * 根据写入数据的时间以及当前设备落盘的最后时间戳，找到对应的 TsFileProcessor
+ * 写入 TsFileProcessor 对应的 memtable 中
+ * 记录写前日志
+ * 根据 memtable 大小，来判断是否触发异步持久化 memtable 操作
+ * 根据当前磁盘 TsFile 的大小，判断是否触发文件关闭操作
+
+### 批量数据（一个设备多个时间戳多个值）写入
+
+* 对应的接口
+ * Session 的 insertBatch
+
+* 总入口: public Integer[] insertBatch(BatchInsertPlan batchInsertPlan)
+ * 找到对应的 StorageGroupProsessor
+ * 根据这批数据的时间以及当前设备落盘的最后时间戳，将这批数据分成小批，分别对应到一个 TsFileProcessor 中
+ * 分别将每小批写入 TsFileProcessor 对应的 memtable 中
+ * 记录写前日志
+ * 根据 memtable 大小，来判断是否触发异步持久化 memtable 操作
+ * 根据当前磁盘 TsFile 的大小，判断是否触发文件关闭操作
+
+
+## 数据访问
+
+* 总入口（StorageEngine）: public QueryDataSource query(SingleSeriesExpression
seriesExpression, QueryContext context,
+ QueryFileManager filePathsManager)
+
+ * 找到所有包含这个时间序列的顺序和乱序的 TsFileResource 进行返回，供查询引擎使用。
+
+## 相关文档
+
+* [写前日志 (WAL)](2-WAL.md)
+
+* [memtable 持久化](3-FlushManager.md)
+
+* [文件合并机制](4-MergeManager.md)
\ No newline at end of file
diff --git a/docs/Community-History&Vision.md
b/docs/Documentation-CHN/SystemDesign/4-StorageEngine/2-WAL.md
similarity index 92%
copy from docs/Community-History&Vision.md
copy to docs/Documentation-CHN/SystemDesign/4-StorageEngine/2-WAL.md
index 64fcf07..08b54c9 100644
--- a/docs/Community-History&Vision.md
+++ b/docs/Documentation-CHN/SystemDesign/4-StorageEngine/2-WAL.md
@@ -19,6 +19,8 @@
-->
-# History&Vision
+# 写前日志
-Comming Soon.
+## 相关代码
+
+* org.apache.iotdb.db.writelog.*
diff --git
a/docs/Documentation-CHN/SystemDesign/4-StorageEngine/3-FlushManager.md
b/docs/Documentation-CHN/SystemDesign/4-StorageEngine/3-FlushManager.md
new file mode 100644
index 0000000..1850742
--- /dev/null
+++ b/docs/Documentation-CHN/SystemDesign/4-StorageEngine/3-FlushManager.md
@@ -0,0 +1,84 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+# Flush Memtable
+
+## 设计思想
+
+内存缓冲区 memtable 达到一定阈值后，会交给 FlushManager 进行异步的持久化，不阻塞正常写入。持久化的过程采用流水线的方式。
+
+## 相关代码
+
+* org.apache.iotdb.db.engine.flush.FlushManager
+
+ Memtable 的 Flush 任务管理器。
+
+* org.apache.iotdb.db.engine.flush.MemtableFlushTask
+
+ 负责持久化一个 Memtable。
+
+## FlushManager: 持久化管理器
+
+FlushManager 可以接受 memtable 的持久化任务，提交者有两个，第一个是 TsFileProcessor，第二个是持久化子线程
FlushThread。
+
+每个 TsFileProcessor 同一时刻只会有一个 flush 任务执行，一个 TsFileProcessor 可能对应多个需要持久化的
memtable
+
+## MemTableFlushTask: 持久化任务
+
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/19167280/73625254-03fe2680-467f-11ea-8197-115f3a749cbd.png";>
+
+背景：每个 memtable 可包含多个 device，每个 device 可包含多个 measurement。
+
+### 三个线程
+
+一个 memtable 的持久化的过程有三个线程，只有当所有任务都完成后，主线程工作才结束。
+
+* MemTableFlushTask 所在线程
+
+ 持久化主线程兼排序线程，负责给每个 measurement 对应的 chunk 排序。
+
+* encodingTask 线程
+
+ 编码线程，负责给每个 Chunk 进行编码，编码成字节数组。
+
+* ioTask 线程
+
+ IO线程，负责将编码好的 Chunk 持久化到磁盘的 TsFile 文件上。
+
+### 两个任务队列
+
+三个线程之间通过两个任务队列交互
+
+* encodingTaskQueue: 排序线程->编码线程，包括三种任务
+
+ * StartFlushGroupIOTask：开始持久化一个 device (ChunkGroup)， encoding
不处理这个命令，直接发给 IO 线程。
+
+ * Pair\<TVList, MeasurementSchema\>：编码一个 Chunk
+
+ * EndChunkGroupIoTask：结束一个 device (ChunkGroup) 的持久化，encoding
不处理这个命令，直接发给 IO 线程。
+
+* ioTaskQueue: 编码线程->IO线程，包括三种任务
+
+ * StartFlushGroupIOTask：开始持久化一个 device (ChunkGroup)。
+
+ * IChunkWriter：持久化一个 Chunk 到磁盘上
+
+ * EndChunkGroupIoTask：结束一个 device (ChunkGroup) 的持久化。
\ No newline at end of file
diff --git a/docs/Community-History&Vision.md
b/docs/Documentation-CHN/SystemDesign/4-StorageEngine/4-MergeManager.md
similarity index 91%
copy from docs/Community-History&Vision.md
copy to docs/Documentation-CHN/SystemDesign/4-StorageEngine/4-MergeManager.md
index 64fcf07..7e68c4f 100644
--- a/docs/Community-History&Vision.md
+++ b/docs/Documentation-CHN/SystemDesign/4-StorageEngine/4-MergeManager.md
@@ -19,6 +19,8 @@
-->
-# History&Vision
+# 文件合并机制
-Comming Soon.
+## 相关代码
+
+* org.apache.iotdb.db.engine.merge.*
\ No newline at end of file
diff --git a/docs/Documentation-CHN/SystemDesign/5-DataQuery/1-DataQuery.md
b/docs/Documentation-CHN/SystemDesign/5-DataQuery/1-DataQuery.md
new file mode 100644
index 0000000..5b0bf78
--- /dev/null
+++ b/docs/Documentation-CHN/SystemDesign/5-DataQuery/1-DataQuery.md
@@ -0,0 +1,67 @@
+<!--
+
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+-->
+
+# 数据查询
+
+* 查询种类
+ * 原始数据查询
+ * 聚合查询
+ * 降采样查询
+ * 单点补空值查询
+ * 最新数据查询等
+
+## 查询计划
+
+主要包括两种查询框架：
+
+* 不带值过滤条件的（如，select f1, f2, f3 from root.d1）
+
+ 可以按照多路归并的方式进行查询，分别查询三列的结果，最后按照时间进行对齐。
+
+* 带值过滤条件的（如，select f1, f2, f3 from root.d1 where f1 = 1 and f2 = 2）
+
+ 先根据过滤条件生成结果集的时间戳，再根据时间戳查询其他列的值。
+
+## 查询组件
+
+为了实现任意过滤条件下的多种查询，有以下查询组件
+
+* SeriesReaderWithoutValueFilter
+
+ 一个时间序列的 reader，不带任何值过滤条件，但是可能带时间过滤条件。
+
+* SeriesReaderWithValueFilter
+
+ 一个时间序列的 reader，带值过滤条件（也可能包含时间过滤条件）。
+
+* SeriesReaderByTimestamp
+
+ 一个时间序列的 reader，每给一个时间戳，返回此时间戳对应的值，若没有则返回空。
+
+* TimeGenerator
+ * 功能：根据过滤条件构造表达式的树形结构，生成结果集的时间戳，如 f1 = 1 and f2 = 2，构造一棵树
AND（leaf(f1=1), leaf(f2=2)）。
+ * 叶子节点：对应一个 SeriesReaderWithValueFilter，返回满足条件的结果的时间戳。
+ * 中间节点：按照此节点的谓词逻辑对时间戳做计算。
+ * 根节点：返回最终结果集的时间戳。
+
+## AbstractSeriesReader
+
+单个时间序列的基础 Reader 类。输入为一个 QueryDataSource，包括顺序和乱序文件列表。
\ No newline at end of file
diff --git a/docs/Documentation-CHN/UserGuide/8-System Design
(Developer)/2-Files.md
b/docs/Documentation-CHN/UserGuide/8-Architecture/1-Files.md
similarity index 100%
rename from docs/Documentation-CHN/UserGuide/8-System Design
(Developer)/2-Files.md
rename to docs/Documentation-CHN/UserGuide/8-Architecture/1-Files.md
diff --git a/docs/Documentation-CHN/UserGuide/8-System Design
(Developer)/3-Writing Data on HDFS.md
b/docs/Documentation-CHN/UserGuide/8-Architecture/2-Writing Data on HDFS.md
similarity index 100%
rename from docs/Documentation-CHN/UserGuide/8-System Design
(Developer)/3-Writing Data on HDFS.md
rename to docs/Documentation-CHN/UserGuide/8-Architecture/2-Writing Data on
HDFS.md
diff --git a/docs/Documentation-CHN/UserGuide/8-System Design
(Developer)/4-Shared Nothing Cluster.md
b/docs/Documentation-CHN/UserGuide/8-Architecture/3-Shared Nothing Cluster.md
similarity index 100%
copy from docs/Documentation-CHN/UserGuide/8-System Design (Developer)/4-Shared
Nothing Cluster.md
copy to docs/Documentation-CHN/UserGuide/8-Architecture/3-Shared Nothing
Cluster.md
diff --git a/docs/Documentation-CHN/UserGuide/8-System Design
(Developer)/4-Shared Nothing Cluster.md
b/docs/Documentation/SystemDesign/0-Content.md
similarity index 70%
copy from docs/Documentation-CHN/UserGuide/8-System Design (Developer)/4-Shared
Nothing Cluster.md
copy to docs/Documentation/SystemDesign/0-Content.md
index 1c0a79b..72b8385 100644
--- a/docs/Documentation-CHN/UserGuide/8-System Design (Developer)/4-Shared
Nothing Cluster.md
+++ b/docs/Documentation/SystemDesign/0-Content.md
@@ -19,8 +19,24 @@
-->
-# 第8章: 系统设计
-## Shared-nothing 架构
-
-正在开发中，预计2019年底发布，敬请期待。
+# Chapter 0: Architecture
+* 1-Architecture
+# Chapter 1: TsFile
+* 1-TsFile
+* 2-Format
+* 3-Write
+* 4-Read
+# Chapter 2: QueryEngine
+* 1-QueryEngine
+* 2-Planner
+* 3-PlanExecutor
+# Chapter 3: SchemaManager
+* 1-SchemaManager
+# Chapter 4: StorageEngine
+* 1-StorageEngine
+* 2-WAL
+* 3-FlushManager
+* 4-MergeManager
+# Chapter 5: DataQuery
+* 1-DataQuery
\ No newline at end of file
diff --git a/docs/Documentation-CHN/UserGuide/8-System Design
(Developer)/4-Shared Nothing Cluster.md
b/docs/Documentation/SystemDesign/1-TsFile/1-TsFile.md
similarity index 85%
rename from docs/Documentation-CHN/UserGuide/8-System Design
(Developer)/4-Shared Nothing Cluster.md
rename to docs/Documentation/SystemDesign/1-TsFile/1-TsFile.md
index 1c0a79b..b8ea7e4 100644
--- a/docs/Documentation-CHN/UserGuide/8-System Design (Developer)/4-Shared
Nothing Cluster.md
+++ b/docs/Documentation/SystemDesign/1-TsFile/1-TsFile.md
@@ -19,8 +19,12 @@
-->
-# 第8章: 系统设计
+# TsFile
-## Shared-nothing 架构
+TsFile is the data file of IoTDB.
-正在开发中，预计2019年底发布，敬请期待。
+## Related
+
+* [File Format](2-Format.md)
+* [Write Process](3-Write.md)
+* [Read Process](4-Read.md)
diff --git a/docs/Documentation/UserGuide/8-System Design
(Developer)/1-Hierarchy.md
b/docs/Documentation/SystemDesign/1-TsFile/2-Format.md
similarity index 99%
rename from docs/Documentation/UserGuide/8-System Design
(Developer)/1-Hierarchy.md
rename to docs/Documentation/SystemDesign/1-TsFile/2-Format.md
index 02aaec3..5e37bc5 100644
--- a/docs/Documentation/UserGuide/8-System Design (Developer)/1-Hierarchy.md
+++ b/docs/Documentation/SystemDesign/1-TsFile/2-Format.md
@@ -7,9 +7,9 @@
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@@ -19,7 +19,7 @@
-->
-# Chapter 7: System Design
+# TsFile Format
## 1. TsFile Design
diff --git a/docs/Community-History&Vision.md
b/docs/Documentation/SystemDesign/1-TsFile/3-Write.md
similarity index 96%
copy from docs/Community-History&Vision.md
copy to docs/Documentation/SystemDesign/1-TsFile/3-Write.md
index 64fcf07..ca90fe8 100644
--- a/docs/Community-History&Vision.md
+++ b/docs/Documentation/SystemDesign/1-TsFile/3-Write.md
@@ -19,6 +19,4 @@
-->
-# History&Vision
-
-Comming Soon.
+# TsFile Write Process
diff --git a/docs/Community-History&Vision.md
b/docs/Documentation/SystemDesign/1-TsFile/4-Read.md
similarity index 96%
rename from docs/Community-History&Vision.md
rename to docs/Documentation/SystemDesign/1-TsFile/4-Read.md
index 64fcf07..d6127f0 100644
--- a/docs/Community-History&Vision.md
+++ b/docs/Documentation/SystemDesign/1-TsFile/4-Read.md
@@ -19,6 +19,4 @@
-->
-# History&Vision
-
-Comming Soon.
+# TsFile Read Process
diff --git a/docs/Documentation/Developers/release.adoc
b/docs/Documentation/SystemDesign/release.adoc
similarity index 100%
rename from docs/Documentation/Developers/release.adoc
rename to docs/Documentation/SystemDesign/release.adoc
diff --git a/docs/Documentation/UserGuide/0-Content.md
b/docs/Documentation/UserGuide/0-Content.md
index dcef5ac..03f5edb 100644
--- a/docs/Documentation/UserGuide/0-Content.md
+++ b/docs/Documentation/UserGuide/0-Content.md
@@ -65,8 +65,7 @@
* 3-Spark TsFile
* 4-Spark IoTDB
* 5-Hive TsFile
-# Chapter 8: System Design (Developer)
-* 1-Hierarchy
-* 2-Files
-* 3-Writing Data on HDFS
-* 4-Shared Nothing Cluster
+# Chapter 8: Architecture
+* 1-Files
+* 2-Writing Data on HDFS
+* 3-Shared Nothing Cluster
diff --git a/docs/Documentation/UserGuide/8-System Design
(Developer)/2-Files.md b/docs/Documentation/UserGuide/8-Architecture/1-Files.md
similarity index 100%
rename from docs/Documentation/UserGuide/8-System Design (Developer)/2-Files.md
rename to docs/Documentation/UserGuide/8-Architecture/1-Files.md
diff --git a/docs/Documentation/UserGuide/8-System Design (Developer)/3-Writing
Data on HDFS.md b/docs/Documentation/UserGuide/8-Architecture/2-Writing Data on
HDFS.md
similarity index 100%
rename from docs/Documentation/UserGuide/8-System Design (Developer)/3-Writing
Data on HDFS.md
rename to docs/Documentation/UserGuide/8-Architecture/2-Writing Data on HDFS.md
diff --git a/docs/Documentation/UserGuide/8-System Design (Developer)/4-Shared
Nothing Cluster.md b/docs/Documentation/UserGuide/8-Architecture/3-Shared
Nothing Cluster.md
similarity index 100%
rename from docs/Documentation/UserGuide/8-System Design (Developer)/4-Shared
Nothing Cluster.md
rename to docs/Documentation/UserGuide/8-Architecture/3-Shared Nothing
Cluster.md
diff --git a/pom.xml b/pom.xml
index 6d58e96..42bb9d7 100644
--- a/pom.xml
+++ b/pom.xml
@@ -507,6 +507,10 @@
<exclude>licenses/*</exclude>
<!-- generated by Github -->
<exclude>.github/**</exclude>
+ <!-- figures -->
+ <exclude>**/.eps</exclude>
+ <exclude>**/.png</exclude>
+ <exclude>**/.jpg</exclude>
<!--Generated by Apache Release -->
<exclude>local-snapshots-dir/**</exclude>
<!-- JSON can't contain comments and therefore no
Apache header -->
![https://user-images.githubusercontent.com/19167280/73625222-ddd88680-467e-11ea-9098-e808ed4979c5.png"](https://user-images.githubusercontent.com/19167280/73625222-ddd88680-467e-11ea-9098-e808ed4979c5.png"){kind=link}
![https://user-images.githubusercontent.com/19167280/73625221-ddd88680-467e-11ea-9cf3-70367e5886f4.png"](https://user-images.githubusercontent.com/19167280/73625221-ddd88680-467e-11ea-9cf3-70367e5886f4.png"){kind=link}
![https://user-images.githubusercontent.com/19167280/73625238-efba2980-467e-11ea-927e-a7021f8153af.png"](https://user-images.githubusercontent.com/19167280/73625238-efba2980-467e-11ea-927e-a7021f8153af.png"){kind=link}
![https://user-images.githubusercontent.com/19167280/73625242-f648a100-467e-11ea-921c-b954a3ecae7a.png"](https://user-images.githubusercontent.com/19167280/73625242-f648a100-467e-11ea-921c-b954a3ecae7a.png"){kind=link}
![https://user-images.githubusercontent.com/19167280/73625246-fc3e8200-467e-11ea-8815-67b9c4ab716e.png"](https://user-images.githubusercontent.com/19167280/73625246-fc3e8200-467e-11ea-8815-67b9c4ab716e.png"){kind=link}
![https://user-images.githubusercontent.com/19167280/73625255-03fe2680-467f-11ea-91ae-64407ef1125c.png"](https://user-images.githubusercontent.com/19167280/73625255-03fe2680-467f-11ea-91ae-64407ef1125c.png"){kind=link}
![https://user-images.githubusercontent.com/19167280/73625254-03fe2680-467f-11ea-8197-115f3a749cbd.png"](https://user-images.githubusercontent.com/19167280/73625254-03fe2680-467f-11ea-8197-115f3a749cbd.png"){kind=link}