This is an automated email from the ASF dual-hosted git repository.
chaow pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/iotdb.git
The following commit(s) were added to refs/heads/master by this push:
new 6c086fa Update documents in SystemDesign (#4950)
6c086fa is described below
commit 6c086fa63ca2384b58900bcd200a1eafd9bde551
Author: 任宇华 <[email protected]>
AuthorDate: Tue Feb 8 09:34:57 2022 +0800
Update documents in SystemDesign (#4950)
Co-authored-by: renyuhua <[email protected]>
---
docs/SystemDesign/DataQuery/AggregationQuery.md | 111 ++++++++++--------
docs/SystemDesign/DataQuery/SeriesReader.md | 31 +++--
docs/SystemDesign/QueryEngine/QueryEngine.md | 2 +-
.../QueryEngine/ResultSetConstruction.md | 10 +-
docs/SystemDesign/SchemaManager/SchemaManager.md | 33 +++---
.../SystemDesign/StorageEngine/DataManipulation.md | 14 +--
docs/zh/SystemDesign/DataQuery/AggregationQuery.md | 125 ++++++++++++---------
.../zh/SystemDesign/DataQuery/QueryFundamentals.md | 6 +-
docs/zh/SystemDesign/DataQuery/SeriesReader.md | 57 +++++-----
docs/zh/SystemDesign/QueryEngine/QueryEngine.md | 2 +-
.../QueryEngine/ResultSetConstruction.md | 10 +-
.../zh/SystemDesign/SchemaManager/SchemaManager.md | 33 +++---
.../SystemDesign/StorageEngine/DataManipulation.md | 12 +-
13 files changed, 243 insertions(+), 203 deletions(-)
diff --git a/docs/SystemDesign/DataQuery/AggregationQuery.md
b/docs/SystemDesign/DataQuery/AggregationQuery.md
index 4d096d1..05e6c6f 100644
--- a/docs/SystemDesign/DataQuery/AggregationQuery.md
+++ b/docs/SystemDesign/DataQuery/AggregationQuery.md
@@ -27,63 +27,83 @@ The main logic of the aggregation query is in
AggregateExecutor
## Aggregation query without value filter
-For aggregate queries without value filters, the results are obtained by the
`executeWithoutValueFilter()` method and a dataSet is constructed. First use
the `mergeSameSeries()` method to merge aggregate queries for the same time
series. For example: if you need to calculate count(s1), sum(s2), count(s3),
sum(s1), you need to calculate two aggregation values of s1, then the
pathToAggrIndexesMap result will be: s1-> 0, 3; s2-> 1; s3-> 2.
+For aggregate queries without value filters, the results are obtained by the
`executeWithoutValueFilter()` method and a dataSet is constructed. First use
the `groupAggregationsBySeries` method to merge aggregate queries for the same
time series. For example: if you need to calculate count(s1), sum(s2),
count(s3), sum(s1), you need to calculate two aggregation values of s1, then
the pathToAggrIndexesMap result will be: s1-> 0, 3; s2-> 1; s3-> 2.
-Then you will get `pathToAggrIndexesMap`, where each entry is an aggregate
query of series, so you can calculate its aggregate value `aggregateResults` by
calling the `groupAggregationsBySeries()` method. Before you finally create
the result set, you need to restore its order to the order of the user query.
Finally use the `constructDataSet()` method to create a result set and return
it.
+Then you will get `pathToAggrIndexesMap`, where each entry is an aggregate
query of series, so you can calculate its aggregate value `aggregateResult` by
calling the `aggregateOneSeries` method. Before you finally create the result
set, you need to restore its order to the order of the user query. Finally use
the `constructDataSet()` method to create a result set and return it.
-The `groupAggregationsBySeries ()` method is explained in detail below. First
create an `IAggregateReader`:
+The `aggregateOneSeries()` method is explained in detail below. First create
an `IAggregateReader`:
```
-IAggregateReader seriesReader = new SeriesAggregateReader(
- pathToAggrIndexes.getKey(), tsDataType, context,
QueryResourceManager.getInstance()
- .getQueryDataSource(seriesPath, context, timeFilter), timeFilter,
null);
+if (ascAggregateResultList != null && !ascAggregateResultList.isEmpty()) {
+ IAggregateReader seriesReader = new SeriesAggregateReader(
+ seriesPath, measurements, tsDataType, context, queryDataSource,
timeFilter,
+ null, null, true);
+ aggregateFromReader(seriesReader, ascAggregateResultList);
+}
+if (descAggregateResultList != null && !descAggregateResultList.isEmpty()) {
+ IAggregateReader seriesReader = new SeriesAggregateReader(
+ seriesPath, measurements, tsDataType, context, queryDataSource,
timeFilter,
+ null, null, false);
+ aggregateFromReader(seriesReader, descAggregateResultList);
+}
```
-For each entry (that is, series), first create an aggregate result
`AggregateResult` for each aggregate query. Maintain a boolean list
`isCalculatedList`, corresponding to whether each `AggregateResult` has been
calculated. Record the remaining number of functions to be calculated in
`remainingToCalculate`. The list of boolean values and this count value will
make some aggregate functions (such as `FIRST_VALUE`) not need to continue the
entire loop process after obtaining the result.
+For each entry (that is, series), first create an aggregate result
`AggregateResult` for each aggregate query. Maintain a boolean array
`isCalculatedArray`, corresponding to whether each `AggregateResult` has been
calculated. Record the remaining number of functions to be calculated in
`remainingToCalculate`. The array of boolean values and this count value will
make some aggregate functions (such as `FIRST_VALUE`) not need to continue the
entire loop process after obtaining the result.
Next, update `AggregateResult` according to the usage method of
`aggregateReader` introduced in Section 5.2:
```
-while (aggregateReader.hasNextChunk()) {
- if (aggregateReader.canUseCurrentChunkStatistics()) {
- Statistics chunkStatistics = aggregateReader.currentChunkStatistics();
-
- // do some aggregate calculation using chunk statistics
- ...
-
- aggregateReader.skipCurrentChunk();
- continue;
- }
-
- while (aggregateReader.hasNextPage()) {
- if (aggregateReader.canUseCurrentPageStatistics()) {
- Statistics pageStatistic = aggregateReader.currentPageStatistics();
-
- // do some aggregate calculation using page statistics
- ...
-
- aggregateReader.skipCurrentPage();
- continue;
- } else {
- BatchData batchData = aggregateReader.nextPage();
- // do some aggregate calculation using batch data
- ...
- }
- }
+while (seriesReader.hasNextFile()) {
+ // cal by file statistics
+ if (seriesReader.canUseCurrentFileStatistics()) {
+ Statistics fileStatistics = seriesReader.currentFileStatistics();
+ remainingToCalculate =
+ aggregateStatistics(
+ aggregateResultList, isCalculatedArray, remainingToCalculate,
fileStatistics);
+ if (remainingToCalculate == 0) {
+ return;
+ }
+ seriesReader.skipCurrentFile();
+ continue;
+ }
+
+ while (seriesReader.hasNextChunk()) {
+ // cal by chunk statistics
+ if (seriesReader.canUseCurrentChunkStatistics()) {
+ Statistics chunkStatistics = seriesReader.currentChunkStatistics();
+ remainingToCalculate =
+ aggregateStatistics(
+ aggregateResultList, isCalculatedArray, remainingToCalculate,
chunkStatistics);
+ if (remainingToCalculate == 0) {
+ return;
+ }
+ seriesReader.skipCurrentChunk();
+ continue;
+ }
+
+ remainingToCalculate =
+ aggregatePages(
+ seriesReader, aggregateResultList, isCalculatedArray,
remainingToCalculate);
+ if (remainingToCalculate == 0) {
+ return;
+ }
+ }
}
```
-It should be noted that before updating each result, you need to first
determine whether it has been calculated (using the isCalculatedList list);
after each update, call the isCalculatedAggregationResult () method to update
the boolean values in the list . If all values in the list are true, that is,
the value of `remainingToCalculate` is 0, it proves that all aggregate function
results have been calculated and can be returned.
+It should be noted that before updating each result, you need to first
determine whether it has been calculated (using the `isCalculatedArray` array);
after each update, call the `hasFinalResult()` method to update the boolean
values in the array . If all values in the list are true, that is, the value
of `remainingToCalculate` is 0, it proves that all aggregate function results
have been calculated and can be returned.
```
-if (Boolean.FALSE.equals(isCalculatedList.get(i))) {
- AggregateResult aggregateResult = aggregateResultList.get(i);
- ... // update
- if (aggregateResult.isCalculatedAggregationResult()) {
- isCalculatedList.set(i, true);
- remainingToCalculate--;
- if (remainingToCalculate == 0) {
- return aggregateResultList;
+for (int i = 0; i < aggregateResultList.size(); i++) {
+ if (!isCalculatedArray[i]) {
+ AggregateResult aggregateResult = aggregateResultList.get(i);
+ aggregateResult.updateResultFromStatistics(statistics);
+ if (aggregateResult.hasFinalResult()) {
+ isCalculatedArray[i] = true;
+ newRemainingToCalculate--;
+ if (newRemainingToCalculate == 0) {
+ return newRemainingToCalculate;
+ }
+ }
}
- }
}
```
@@ -122,8 +142,9 @@ each node at the given level in current Metadata Tree.
The logic is in the `AggregationExecutor` class.
1. In the beginning, get the final paths group by level and the origin path
index to final path.
- > For example, we could get final path `root.sg1` by
`root.sg1.d1.s0,root.sg1.d2.s1` and `level=1`.
-
+
+> For example, we could get final path `root.sg1` by
`root.sg1.d1.s0,root.sg1.d2.s1` and `level=1`.
+
2. Then, get the aggregated query result: RowRecord.
3. Finally, merge each RowRecord to NewRecord, which has fields like <final
path, count>.
diff --git a/docs/SystemDesign/DataQuery/SeriesReader.md
b/docs/SystemDesign/DataQuery/SeriesReader.md
index 4cc83eb..9efc1e0 100644
--- a/docs/SystemDesign/DataQuery/SeriesReader.md
+++ b/docs/SystemDesign/DataQuery/SeriesReader.md
@@ -165,50 +165,47 @@ First introduce some important fields in SeriesReader
/*
* File layer
*/
-private final List<TsFileResource> seqFileResource;
- Sequential file list, because the sequential file itself is guaranteed
to be ordered, and the timestamps do not overlap each other, just use List to
store
-
-private final PriorityQueue<TsFileResource> unseqFileResource;
- Out-of-order file list, because out-of-order files do not guarantee
order between each other, and may overlap
+protected final QueryDataSource dataSource;
+ The QueryDataSource contains all the seq and unseq TsFileResources for
one timeseries in one query
/*
* chunk layer
*
* The data between the three fields is never duplicated, and first is always
the first (minimum start time)
*/
-private ChunkMetaData firstChunkMetaData;
+protected IChunkMetadata firstChunkMetadata;
This field is filled first when filling the chunk layer to ensure that
this chunk has the current minimum start time
-private final List<ChunkMetaData> seqChunkMetadatas;
- The ChunkMetaData obtained after the sequential files are unpacked is
stored here. It is ordered and does not overlap with each other, so the List is
used for storage.
-
-private final PriorityQueue<ChunkMetaData> unseqChunkMetadatas;
- ChunkMetaData obtained after unordered files are stored is stored here,
there may be overlap between each other, in order to ensure order, priority
queue is used for storage
+protected final PriorityQueue<IChunkMetadata> cachedChunkMetadata;
+ ChunkMetaData obtained after files are stored is stored here, there may
be overlap between each other, in order to ensure order, priority queue is used
for storage
/*
* page layer
*
* The data between the two fields is never duplicated, and first is always
the first (minimum start time)
*/
-private VersionPageReader firstPageReader;
+protected VersionPageReader firstPageReader;
Page reader with the smallest start time
-private PriorityQueue<VersionPageReader> cachedPageReaders;
- All page readers currently acquired, sorted by the start time of each
page
+protected final List<VersionPageReader> seqPageReaders = new LinkedList<>();
+ The page reader obtained after the sequential chunk are unpacked is
stored here. It is ordered and does not overlap with each other, so the List is
used for storage.
+
+protected final PriorityQueue<VersionPageReader> unSeqPageReaders;
+ page reader obtained after unordered chunk are stored is stored here,
there may be overlap between each other, in order to ensure order, priority
queue is used for storage
/*
* Intersecting data point layer
*/
-private PriorityMergeReader mergeReader;
+protected final PriorityMergeReader mergeReader;
Essentially, there are multiple pages with priority, and the data
points are output from low to high according to the timestamp. When the
timestamps are the same, the high priority page is retained.
/*
* Caching of results from intersecting data points
*/
-private boolean hasCachedNextOverlappedPage;
+protected boolean hasCachedNextOverlappedPage;
Whether the next batch is cached
-private BatchData cachedBatchData;
+protected BatchData cachedBatchData;
Cached reference to the next batch
```
diff --git a/docs/SystemDesign/QueryEngine/QueryEngine.md
b/docs/SystemDesign/QueryEngine/QueryEngine.md
index 9086ba0..7d96158 100644
--- a/docs/SystemDesign/QueryEngine/QueryEngine.md
+++ b/docs/SystemDesign/QueryEngine/QueryEngine.md
@@ -29,7 +29,7 @@ The query engine is responsible for parsing all user
commands, generating plans,
## Related classes
-* org.apache.iotdb.db.service.TSServiceImpl
+* org.apache.iotdb.db.service.thrift.impl.TSServiceImpl.java
IoTDB server-side RPC implementation, which directly interacts with the
client.
diff --git a/docs/SystemDesign/QueryEngine/ResultSetConstruction.md
b/docs/SystemDesign/QueryEngine/ResultSetConstruction.md
index f9442cb..15bb466 100644
--- a/docs/SystemDesign/QueryEngine/ResultSetConstruction.md
+++ b/docs/SystemDesign/QueryEngine/ResultSetConstruction.md
@@ -33,7 +33,7 @@ Next Introduce the first part: including the result set
header construction way
The result set table header construction logic for the raw data query is
mainly in the `getWideQueryHeaders()` method.
-- org.apache.iotdb.db.service.TSServiceImpl.getWideQueryHeaders
+- org.apache.iotdb.db.qp.physical.crud.QueryPlan.getWideQueryHeaders
For the construction of each header, you need to provide the column name and
the corresponding data type of the column.
@@ -61,7 +61,7 @@ SQL2：`SELECT count(s1), max_time(s1) FROM root.sg.d1;` ->
The result set table header construction logic for the Align by device query
is mainly in the `getAlignByDeviceQueryHeaders()` method.
-- org.apache.iotdb.db.service.TSServiceImpl.getAlignByDeviceQueryHeaders
+- org.apache.iotdb.db.qp.physical.crud.AlignByDevicePlan.java
The result set construction of the AlignByDeviceQuery depends on the list of
**measurements not deduplicated** generated in the physical query plan. For a
brief introduction, the measurements list is a list generated by the suffix
path (including wildcards) in the SELECT clause, including three types, namely
constant, exist and nonexist. For details, please refer to [Align by device
query](../DataQuery/AlignByDeviceQuery.md)
@@ -110,7 +110,7 @@ Unlike the header construction, we do not need to query
duplicate data when exec
In addition to AlignByDeviceQuery, the deduplication logic of **RawDataQuery,
AggregateQuery, LastQuery** etc. is in the `duplicate()` method.
-- org.apache.iotdb.db.qp.strategy.PhysicalGenerator.deduplicate()
+- org.apache.iotdb.db.qp.physical.crud.QueryPlan.deduplicate()
The deduplication logic is relatively simple: first, get the path not
deduplicated from the query plan, and then create a `Set` structure to
deduplicate during traversal.
@@ -121,7 +121,7 @@ Because only one set of data needs to be calculated for the
LastQuery, there is
The deduplication logic of **AlignByDeviceQuery** is in the
`hasNextWithoutConstraint()` method of its result set.
--
org.apache.iotdb.db.query.dataset.AlignByDeviceDataSet.hasNextWithoutConstraint()
+- org.apache.iotdb.db.query.dataset.QueryDataSet.hasNextWithoutConstraint()
Because AlignByDeviceQuery need to organize their query plans by device, each
device query may not have the same path, and it is allowed to contain constant
columns and nonexistent timeseries, so it cannot simply be deduplicated with
other queries. Deduplication requires **removing not only the repeated
timeseries path, but also the constant columns appearing in the query and the
timeseries that do not exist in the current device**.
The implementation logic can be referred to [Align by device
query](../DataQuery/AlignByDeviceQuery.md).
@@ -152,7 +152,7 @@ Then query result set is：
To restore the final result set, we need to construct a mapping set
`columnOrdinalMap` with the column name to its position in the query result
set, which is aimed at fetching the corresponding result of a column from the
query result set. This part of logic is completed in the constructor of the new
result set `IoTDBQueryResultSet`.
-- org.apache.iotdb.jdbc.AbstractIoTDBResultSet.AbstractIoTDBResultSet()
+- org.apache.iotdb.jdbc.AbstractIoTDBResultSet.AbstractIoTDBJDBCResultSet()
In order to construct metadata information in final result set, a complete
list of column names needs to be constructed. The `columnnamelist` given above
does not contain a timestamp. Therefore, it's necessary to determine whether a
timestamp needs to be printed. If so, add the `Time` column to the header to
form a complete header.
diff --git a/docs/SystemDesign/SchemaManager/SchemaManager.md
b/docs/SystemDesign/SchemaManager/SchemaManager.md
index 9846e7e..f1c2cdc 100644
--- a/docs/SystemDesign/SchemaManager/SchemaManager.md
+++ b/docs/SystemDesign/SchemaManager/SchemaManager.md
@@ -7,9 +7,9 @@
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@@ -30,7 +30,7 @@ Metadata of IoTDB is managed by MManger, including:
## MManager
-* Maintain an inverted index for tag: `Map<String, Map<String,
Set<LeafMNode>>> tagIndex`
+* Maintain an inverted index for tag: `Map<String, Map<String,
Set<IMeasurementMNode>>> tagIndex`
> tag key -> tag value -> timeseries LeafMNode
@@ -63,7 +63,6 @@ In the process of initializing, MManager will replay the mlog
to load the metada
* persist log into mlog
* currently, we won't delete the tag/attribute info of that
timeseries in tlog
-
* Set Storage Group
* add StorageGroupMNode in MTree
* If dynamic parameters are turned on, check the memory is satisfied or
not
@@ -91,7 +90,7 @@ In the process of initializing, MManager will replay the mlog
to load the metada
In addition to these seven operation that are needed to be logged, there are
another six alter operation to tag/attribute info of timeseries.
-
+
Same as above, at the beginning of each operation, it will try to obtain the
write lock of MManager, and release it after operation.
* Rename Tag/Attribute
@@ -145,13 +144,13 @@ Same as above, at the beginning of each operation, it
will try to obtain the wri
* org.apache.iotdb.db.metadata.mtree.MTree
-There three types of nodes in MTree: StorageGroupMNode、InternalMNode(Non-leaf
node)、LeafMNode(leaf node), they all extend to MNode.
+There three types of nodes in MTree: StorageGroupMNode、 IMeasurementMNode
(Non-leaf node)、LeafMNode(leaf node), they all extend to MNode.
-Each InternalMNode has a read-write lock. When querying metadata information,
you need to obtain a read lock for each InternalMNode on the path. When
modifying metadata information, if you modify the LeafMNode, you need to obtain
the write lock of its parent node. If you modify a non-leaf node, only need to
obtain its own write lock. If the InternalMNode is located in the device layer,
it also contains a `Map <String, MNode> aliasChildren`, which is used to store
alias information.
+Each InternalMNode has a read-write lock. When querying metadata information,
you need to obtain a read lock for each InternalMNode on the path. When
modifying metadata information, if you modify the IMeasurementMNode , you need
to obtain the write lock of its parent node. If you modify a non-leaf node,
only need to obtain its own write lock. If the InternalMNode is located in the
device layer, it also contains a `Map <String, MNode> aliasChildren`, which is
used to store alias information.
StorageGroupMNode extends to InternalMNode, containing metadata information
for storage groups, such as TTL.
-LeafMNode contains the schema information of the corresponding time series,
its alias(if it doesn't have, it is null) and the offset of the time series
tag/attribute information in the tlog file(if there is no tag/attribute, it is
-1)
+ IMeasurementMNode contains the schema information of the corresponding time
series, its alias(if it doesn't have, it is null) and the offset of the time
series tag/attribute information in the tlog file(if there is no tag/attribute,
it is -1)
example：
@@ -249,7 +248,7 @@ Schema operation examples and the corresponding parsed mlog
record:
* set ttl to root.turbine 10
> mlog: 10,root.turbine,10
-
+
> format: 10,path,ttl
* alter timeseries root.turbine.d1.s1 add tags(tag1=v1)
@@ -260,36 +259,36 @@ Schema operation examples and the corresponding parsed
mlog record:
> format: 10,path,[change offset]
* alter timeseries root.turbine.d1.s1 UPSERT ALIAS=newAlias
-
+
> mlog: 13,root.turbine.d1.s1,newAlias
> format: 13,path,[new alias]
-
+
* create schema template temp1(
s1 INT32 with encoding=Gorilla and compression SNAPPY,
s2 FLOAT with encoding=RLE and compression=SNAPPY
)
-
+
> mlog:5,temp1,0,s1,1,8,1
-
+
> mlog:5,temp1,0,s2,3,2,1
-
+
> format: 5,template name,is Aligned
Timeseries,measurementId,TSDataType,TSEncoding,CompressionType
* set schema template temp1 to root.turbine
-
+
> mlog: 6,temp1,root.turbine
> format: 6,template name,path
* Auto create device root.turbine.d1 (after set a template to a prefix path,
create a device path in mtree automatically when insert data to the device)
-
+
> mlog: 4,root.turbine.d1
> format: 4,path
* set root.turbine.d1 is using template (after set a template to a device
path, this log shows the device is using template)
-
+
> mlog: 61,root.turbine.d1
> format: 61,path
diff --git a/docs/SystemDesign/StorageEngine/DataManipulation.md
b/docs/SystemDesign/StorageEngine/DataManipulation.md
index 2b82820..5394110 100644
--- a/docs/SystemDesign/StorageEngine/DataManipulation.md
+++ b/docs/SystemDesign/StorageEngine/DataManipulation.md
@@ -31,7 +31,7 @@ The following describes four common data manipulation
operations, which are inse
* JDBC's execute and executeBatch interfaces
* Session's insertRecord and insertRecords
* Main entrance: ```public void insert(InsertRowPlan insertRowPlan)```
StorageEngine.java
- * Find the corresponding StorageGroupProcessor
+ * Find the corresponding VirtualStorageGroupProcessor
* Find the corresponding TsFileProcessor according to the time of writing
the data and the last time stamp of the current device order
* Write to the corresponding memtable of TsFileProcessor
* If the file is out of order, update the endTimeMap in tsfileResource
@@ -47,7 +47,7 @@ The following describes four common data manipulation
operations, which are inse
* Session‘s insertTablet
* Main entrance: ```public void insertTablet(InsertTabletPlan
insertTabletPlan)``` StorageEngine.java
- * Find the corresponding StorageGroupProcessor
+ * Find the corresponding VirtualStorageGroupProcessor
* According to the time of this batch of data and the last timestamp of
the current device order, this batch of data is divided into small batches,
which correspond to a TsFileProcessor
* Write each small batch to the corresponding memtable of
TsFileProcessor
* If the file is out of order, update the endTimeMap in
tsfileResource
@@ -70,15 +70,15 @@ Old data is automatically deleted by merging, see:
* Corresponding interface
* JDBC's execute interface, using delete SQL statements
-Each StorageGroupProcessor maintains a ascending version for each partition,
which is managed by SimpleFileVersionController.
+Each VirtualStorageGroupProcessor maintains a ascending version for each
partition, which is managed by SimpleFileVersionController.
Each memtable will apply a version when submitted to flush. After flushing to
TsFile, a current position-version will added to TsFileMetadata.
This information will be used to set version to ChunkMetadata when query.
Main entrance in StorageEngine.java:
-
+
```public void delete(String deviceId, String measurementId, long startTime,
long endTime)```
- * Find the corresponding StorageGroupProcessor
+ * Find the corresponding VirtualStorageGroupProcessor
* Find all impacted working TsFileProcessors to write WAL
* Find all impacted TsFileResources to record a Modification in its mods
file, the Modification format is: path，version, startTime, endTime
* If the TsFile is not closed，get its TsFileProcessor
@@ -94,8 +94,8 @@ For the following mods file, data of d1.s1 falls in range
[100, 200], [180, 300]
* JDBC's execute interface, using the SET TTL statement
* Main entrance: ```public void setTTL(String storageGroup, long dataTTL)
```StorageEngine.java
- * Find the corresponding StorageGroupProcessor
- * Set new data ttl in StorageGroupProcessor
+ * Find the corresponding VirtualStorageGroupProcessor
+ * Set new data ttl in VirtualStorageGroupProcessor
* TTL check on all TsfileResource
* If a file expires under the current TTL, delete the file
diff --git a/docs/zh/SystemDesign/DataQuery/AggregationQuery.md
b/docs/zh/SystemDesign/DataQuery/AggregationQuery.md
index b69fe91..fdcf1da 100644
--- a/docs/zh/SystemDesign/DataQuery/AggregationQuery.md
+++ b/docs/zh/SystemDesign/DataQuery/AggregationQuery.md
@@ -7,9 +7,9 @@
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@@ -27,72 +27,95 @@
## 不带值过滤条件的聚合查询
-对于不带值过滤条件的聚合查询，通过 `executeWithoutValueFilter()` 方法获得结果并构建 dataSet。首先使用
`mergeSameSeries()` 方法将对于相同时间序列的聚合查询合并，例如：如果需要计算 count(s1), sum(s2), count(s3),
sum(s1)，即需要计算 s1 的两个聚合值，那么将会得到 pathToAggrIndexesMap 结果为：s1 -> 0, 3; s2 -> 1; s3
-> 2。
+对于不带值过滤条件的聚合查询，通过 `executeWithoutValueFilter()` 方法获得结果并构建 dataSet。首先使用
`groupAggregationsBySeries` 方法将对于相同时间序列的聚合查询合并，例如：如果需要计算 count(s1), sum(s2),
count(s3), sum(s1)，即需要计算 s1 的两个聚合值，那么将会得到 pathToAggrIndexesMap 结果为：s1 -> 0, 3;
s2 -> 1; s3 -> 2。
-那么将会得到 `pathToAggrIndexesMap`，其中每一个 entry 都是一个 series 的聚合查询，因此可以通过调用
`groupAggregationsBySeries()` 方法计算出其聚合值
`aggregateResults`。在最后创建结果集之前，需要将其顺序还原为用户查询的顺序。最后使用 `constructDataSet()`
方法创建结果集并返回。
+那么将会得到 `pathToAggrIndexesMap`，其中每一个 entry 都是一个 series 的聚合查询，因此可以通过调用
`aggregateOneSeries` 方法计算出其聚合值
`aggregateResult`。在最后创建结果集之前，需要将其顺序还原为用户查询的顺序。最后使用 `constructDataSet()`
方法创建结果集并返回。
-下面详细讲解 `groupAggregationsBySeries()` 方法。首先创建一个 `IAggregateReader`：
-```
-IAggregateReader seriesReader = new SeriesAggregateReader(
- pathToAggrIndexes.getKey(), tsDataType, context,
QueryResourceManager.getInstance()
- .getQueryDataSource(seriesPath, context, timeFilter), timeFilter,
null);
+下面详细讲解 `aggregateOneSeries` 方法。首先创建一个 `IAggregateReader`：
+```java
+if (ascAggregateResultList != null && !ascAggregateResultList.isEmpty()) {
+ IAggregateReader seriesReader = new SeriesAggregateReader(
+ seriesPath, measurements, tsDataType, context, queryDataSource,
timeFilter,
+ null, null, true);
+ aggregateFromReader(seriesReader, ascAggregateResultList);
+}
+if (descAggregateResultList != null && !descAggregateResultList.isEmpty()) {
+ IAggregateReader seriesReader = new SeriesAggregateReader(
+ seriesPath, measurements, tsDataType, context, queryDataSource,
timeFilter,
+ null, null, false);
+ aggregateFromReader(seriesReader, descAggregateResultList);
+}
```
-对于每一个 entry（即 series），首先为其每一种聚合查询创建一个聚合结果 `AggregateResult`，同时维护一个布尔值列表
`isCalculatedList`，对应每一个 `AggregateResult`是否已经计算完成，并记录需要剩余计算的聚合函数数目
`remainingToCalculate`。布尔值列表和这个计数值将会使得某些聚合函数（如
`FIRST_VALUE`）在获得结果后，不需要再继续进行整个循环过程。
+对于每一个 entry（即 series），首先为其每一种聚合查询创建一个聚合结果 `AggregateResult`，同时维护一个布尔值数组
`isCalculatedArray`，对应每一个 `AggregateResult`是否已经计算完成，并记录需要剩余计算的聚合函数数目
`remainingToCalculate`。布尔值列表和这个计数值将会使得某些聚合函数（如
`FIRST_VALUE`）在获得结果后，不需要再继续进行整个循环过程。
接下来，按照 5.2 节所介绍的 `aggregateReader` 使用方法，更新 `AggregateResult`：
-```
-while (aggregateReader.hasNextChunk()) {
- if (aggregateReader.canUseCurrentChunkStatistics()) {
- Statistics chunkStatistics = aggregateReader.currentChunkStatistics();
-
- // do some aggregate calculation using chunk statistics
- ...
-
- aggregateReader.skipCurrentChunk();
- continue;
- }
-
- while (aggregateReader.hasNextPage()) {
- if (aggregateReader.canUseCurrentPageStatistics()) {
- Statistics pageStatistic = aggregateReader.currentPageStatistics();
-
- // do some aggregate calculation using page statistics
- ...
-
- aggregateReader.skipCurrentPage();
- continue;
- } else {
- BatchData batchData = aggregateReader.nextPage();
- // do some aggregate calculation using batch data
- ...
- }
- }
+```java
+while (seriesReader.hasNextFile()) {
+ // cal by file statistics
+ if (seriesReader.canUseCurrentFileStatistics()) {
+ Statistics fileStatistics = seriesReader.currentFileStatistics();
+ remainingToCalculate =
+ aggregateStatistics(
+ aggregateResultList, isCalculatedArray, remainingToCalculate,
fileStatistics);
+ if (remainingToCalculate == 0) {
+ return;
+ }
+ seriesReader.skipCurrentFile();
+ continue;
+ }
+
+ while (seriesReader.hasNextChunk()) {
+ // cal by chunk statistics
+ if (seriesReader.canUseCurrentChunkStatistics()) {
+ Statistics chunkStatistics = seriesReader.currentChunkStatistics();
+ remainingToCalculate =
+ aggregateStatistics(
+ aggregateResultList, isCalculatedArray, remainingToCalculate,
chunkStatistics);
+ if (remainingToCalculate == 0) {
+ return;
+ }
+ seriesReader.skipCurrentChunk();
+ continue;
+ }
+
+ remainingToCalculate =
+ aggregatePages(
+ seriesReader, aggregateResultList, isCalculatedArray,
remainingToCalculate);
+ if (remainingToCalculate == 0) {
+ return;
+ }
+ }
}
```
-需要注意的是，在对于每一个 result 进行更新之前，需要首先判断其是否已经被计算完（利用 `isCalculatedList`
列表）；每一次更新后，调用 `isCalculatedAggregationResult()` 方法同时更新列表中的布尔值。如果列表中所有值均为 true，即
`remainingToCalculate` 值为 0，证明所有聚合函数结果均已计算完，可以返回。
+需要注意的是，在对于每一个 result 进行更新之前，需要首先判断其是否已经被计算完（利用 `isCalculatedArray`
数组）；每一次更新后，调用 `hasFinalResult` 方法同时更新数组中的布尔值。如果列表中所有值均为 true，即
`remainingToCalculate` 值为 0，证明所有聚合函数结果均已计算完，可以返回。
+
```
-if (Boolean.FALSE.equals(isCalculatedList.get(i))) {
- AggregateResult aggregateResult = aggregateResultList.get(i);
- ... // 更新
- if (aggregateResult.isCalculatedAggregationResult()) {
- isCalculatedList.set(i, true);
- remainingToCalculate--;
- if (remainingToCalculate == 0) {
- return aggregateResultList;
+for (int i = 0; i < aggregateResultList.size(); i++) {
+ if (!isCalculatedArray[i]) {
+ AggregateResult aggregateResult = aggregateResultList.get(i);
+ aggregateResult.updateResultFromStatistics(statistics);
+ if (aggregateResult.hasFinalResult()) {
+ isCalculatedArray[i] = true;
+ newRemainingToCalculate--;
+ if (newRemainingToCalculate == 0) {
+ return newRemainingToCalculate;
+ }
+ }
}
- }
}
```
-在使用 `overlapedPageData` 进行更新时，由于获得每一个聚合函数结果都会遍历这个 batchData，因此需要调用
`resetBatchData()` 方法将指针指向其开始位置，使得下一个函数可以遍历。
+ 在使用 `overlapedPageData` 进行更新时，由于获得每一个聚合函数结果都会遍历这个 batchData，因此需要调用
`resetBatchData()` 方法将指针指向其开始位置，使得下一个函数可以遍历。
## 带值过滤条件的聚合查询
+
对于带值过滤条件的聚合查询，通过 `executeWithoutValueFilter()` 方法获得结果并构建 dataSet。首先根据表达式创建
`timestampGenerator`，然后为每一个时间序列创建一个 `SeriesReaderByTimestamp`，放到
`readersOfSelectedSeries`列表中；为每一个查询创建一个聚合结果 `AggregateResult`，放到
`aggregateResults`列表中。
初始化完成后，调用 `aggregateWithValueFilter()` 方法更新结果：
+
```
while (timestampGenerator.hasNext()) {
// 生成 timestamps
@@ -120,14 +143,16 @@ while (timestampGenerator.hasNext()) {
这个逻辑在 `AggregationExecutor`类里。
1. 首先，把所有涉及到的时序按 level 来进行汇集，最后的路径。
- > 例如把 root.sg1.d1.s0,root.sg1.d2.s1 按 level=1 汇集成 root.sg1。
+
+ > 例如把 root.sg1.d1.s0,root.sg1.d2.s1 按 level=1 汇集成 root.sg1。
2. 然后调用上述的聚合逻辑求出所有时序的总点数信息，这个会返回 RowRecord 数据结构。
3. 最后，把聚合查询返回的 RowRecord 按上述的 final paths，进行累加，组合成新的 RowRecord。
- > 例如，把《root.sg1.d1.s0,3》，《root.sg1.d2.s1，4》聚合成《root.sg1，7》
+ > 例如，把《root.sg1.d1.s0,3》，《root.sg1.d2.s1，4》聚合成《root.sg1，7》
> 注意：
+>
> 1. 这里只支持 count 操作
> 2. root 的层级 level=0
\ No newline at end of file
diff --git a/docs/zh/SystemDesign/DataQuery/QueryFundamentals.md
b/docs/zh/SystemDesign/DataQuery/QueryFundamentals.md
index 2a42fc8..ad17a4e 100644
--- a/docs/zh/SystemDesign/DataQuery/QueryFundamentals.md
+++ b/docs/zh/SystemDesign/DataQuery/QueryFundamentals.md
@@ -46,8 +46,9 @@ TsFile 各级结构在前面的 [TsFile](../TsFile/TsFile.md) 文档中已有介
## 顺序和乱序文件的数据特点
-对于顺序和乱序文件的数据，其数据在文件中的分部特征有所不同。
-顺序文件的 TimeseriesMetadata 中所包含的 ChunkMetadata 也是有序的，也就是说如果按照 chunkMetadata1,
chunkMetadata2 的顺序存储，那么将会保证 chunkMetadata1.endtime <= chunkMetadata2.startTime。
+对于顺序和乱序文件的数据，其数据在文件中的分部特征有所不同。
+
+顺序文件的 TimeseriesMetadata 中所包含的 ChunkMetadata 也是有序的，也就是说如果按照 chunkMetadata1,
chunkMetadata2 的顺序存储，那么将会保证 chunkMetadata1.endtime <= chunkMetadata2.startTime。
乱序文件的 TimeseriesMetadata 中所包含的 ChunkMetadata 是无序的，乱序文件中多个 Chunk
所覆盖的数据可能存在重叠，同时也可能与顺序文件中的 Chunk 数据存在重叠。
@@ -67,6 +68,7 @@ Modification
文件：org.apache.iotdb.db.engine.modification.ModificationFile
删除区间的内部表示：org.apache.iotdb.tsfile.read.common.TimeRange
### Modification 文件
+
IoTDB 通过为包含数据的 TsFile 写入一个 Modification 文件来完成删除操作。
在 0.11.0 版本的 IoTDB 中对 Modification 文件中的删除记录格式进行了修改，每一行的删除记录包含删除的开始时间和结束时间。
diff --git a/docs/zh/SystemDesign/DataQuery/SeriesReader.md
b/docs/zh/SystemDesign/DataQuery/SeriesReader.md
index 9946d02..7091c68 100644
--- a/docs/zh/SystemDesign/DataQuery/SeriesReader.md
+++ b/docs/zh/SystemDesign/DataQuery/SeriesReader.md
@@ -7,9 +7,9 @@
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@@ -163,55 +163,52 @@ Object[] values =
readerByTimestamp.getValueInTimestamp(timestamps, length);
```
/*
- * 文件层
- */
-private final List<TsFileResource> seqFileResource;
- 顺序文件列表，因为顺序文件本身就保证有序，且时间戳互不重叠，只需使用 List 进行存储
-
-private final PriorityQueue<TsFileResource> unseqFileResource;
- 乱序文件列表，因为乱序文件互相之间不保证顺序性，且可能有重叠，为了保证顺序，使用优先队列进行存储
+* 文件层
+*/
+protected final QueryDataSource dataSource;
+ QueryDataSource在一个查询中包含一个timeseries的所有seq和unseq TsFileResources
/*
- * chunk 层
- *
- * 三个字段之间数据永远不重复，first 永远是第一个（开始时间最小）
- */
-private ChunkMetaData firstChunkMetaData;
+* chunk 层
+*
+* 三个字段之间数据永远不重复，first 永远是第一个（开始时间最小）
+*/
+protected IChunkMetadata firstChunkMetadata;
填充 chunk 层时优先填充此字段，保证这个 chunk 具有当前最小开始时间
-private final List<ChunkMetaData> seqChunkMetadatas;
- 顺序文件解开后得到的 ChunkMetaData 存放在此，本身有序且互不重叠，所以使用 List 存储
-
-private final PriorityQueue<ChunkMetaData> unseqChunkMetadatas;
- 乱序文件解开后得到的 ChunkMetaData 存放在此，互相之间可能有重叠，为了保证顺序，使用优先队列进行存储
+protected final PriorityQueue<IChunkMetadata> cachedChunkMetadata;
+ 文件解开后得到的 ChunkMetaData 存放在此，互相之间可能有重叠，为了保证顺序，使用优先队列进行存储
/*
- * page 层
- *
- * 两个字段之间数据永远不重复，first 永远是第一个（开始时间最小）
- */
-private VersionPageReader firstPageReader;
+* page cache
+*
+* 两个字段之间数据永远不重复，first 永远是第一个（开始时间最小）
+*/
+protected VersionPageReader firstPageReader;
开始时间最小的 page reader
-private PriorityQueue<VersionPageReader> cachedPageReaders;
- 当前获得的所有 page reader，按照每个 page 的起始时间进行排序
+protected final List<VersionPageReader> seqPageReaders = new LinkedList<>();
+ 顺序 chunk 解开后得到的 page reader 存放在此，本身有序且互不重叠，所以使用 List 存储
+
+protected final PriorityQueue<VersionPageReader> unSeqPageReaders;
+ 乱序 chunk 解开后得到的 page reader 存放在此，互相之间可能有重叠，为了保证顺序，使用优先队列进行存储
/*
* 相交数据点层
*/
-private PriorityMergeReader mergeReader;
+protected final PriorityMergeReader mergeReader;
本质上是多个带优先级的 page，按时间戳从低到高输出数据点，时间戳相同时，保留优先级高的
/*
* 相交数据点产出结果的缓存
*/
-private boolean hasCachedNextOverlappedPage;
+protected boolean hasCachedNextOverlappedPage;
是否缓存了下一个 batch
-private BatchData cachedBatchData;
+protected BatchData cachedBatchData;
缓存的下一个 batch 的引用
```
-
+
下面介绍一下 SeriesReader 里的重要方法
#### hasNextChunk()
diff --git a/docs/zh/SystemDesign/QueryEngine/QueryEngine.md
b/docs/zh/SystemDesign/QueryEngine/QueryEngine.md
index 04b43f0..94a7eef 100644
--- a/docs/zh/SystemDesign/QueryEngine/QueryEngine.md
+++ b/docs/zh/SystemDesign/QueryEngine/QueryEngine.md
@@ -29,7 +29,7 @@
## 相关类
-* org.apache.iotdb.db.service.TSServiceImpl
+* org.apache.iotdb.db.service.thrift.impl.TSServiceImpl.java
IoTDB 服务器端 RPC 实现，与客户端进行直接交互。
diff --git a/docs/zh/SystemDesign/QueryEngine/ResultSetConstruction.md
b/docs/zh/SystemDesign/QueryEngine/ResultSetConstruction.md
index 8836e05..0a6b25a 100644
--- a/docs/zh/SystemDesign/QueryEngine/ResultSetConstruction.md
+++ b/docs/zh/SystemDesign/QueryEngine/ResultSetConstruction.md
@@ -33,7 +33,7 @@
原始数据查询的结果集表头构造逻辑主要在 `getWideQueryHeaders()` 方法中。
-- org.apache.iotdb.db.service.TSServiceImpl.getWideQueryHeaders
+- org.apache.iotdb.db.qp.physical.crud.QueryPlan.getWideQueryHeaders
对于每个结果集表头的构造，需要提供列名及该列对应的数据类型。
@@ -61,7 +61,7 @@ SQL2：`SELECT count(s1), max_time(s1) FROM root.sg.d1;` ->
原始数据查询的结果集表头构造逻辑主要在 `getAlignByDeviceQueryHeaders()` 方法中。
-- org.apache.iotdb.db.service.TSServiceImpl.getAlignByDeviceQueryHeaders
+- org.apache.iotdb.db.qp.physical.crud.AlignByDevicePlan.java
按设备对齐查询的结果集构造依赖于物理查询计划中生成的**未去重**的度量（Measurements）列表。在此作简单介绍，度量列表是由 SELECT
子句中的后缀路径（包括通配符）生成的列表，其中共有三种类型，分别为常量（Constant）、存在的时间序列（Exist）以及不存在的时间序列（NonExist）。详细可以参考
[Align by device query](../DataQuery/AlignByDeviceQuery.md)
@@ -110,7 +110,7 @@ SQL：`SELECT last s1, s2 FROM root.sg.d1;`
除按设备对齐查询外，**原始数据查询、聚合查询、最新数据查询** 等查询的去重逻辑均在 `deduplicate()` 方法中。
-- org.apache.iotdb.db.qp.strategy.PhysicalGenerator.deduplicate()
+- org.apache.iotdb.db.qp.physical.crud.QueryPlan.deduplicate()
去重逻辑比较简单：首先从查询计划中取得未去重的路径，然后在遍历时创建一个 Set 集合用于去重即可。
@@ -120,7 +120,7 @@ SQL：`SELECT last s1, s2 FROM root.sg.d1;`
**按设备对齐查询**的去重逻辑在其结果集的 `hasNextWithoutConstraint()` 方法中。
--
org.apache.iotdb.db.query.dataset.AlignByDeviceDataSet.hasNextWithoutConstraint()
+- org.apache.iotdb.db.query.dataset.QueryDataSet.hasNextWithoutConstraint()
由于按设备对齐查询需要按设备依次组织其查询计划，每个设备查询的路径未必相同，且允许包含常量列以及不存在的时间序列，因此不能简单地与其他查询一起去重。去重时**不仅需要去除重复查询的时间序列路径，还需要去除查询中出现的常量列以及当前设备中不存在的时间序列**。实现方法可以参考
[Align by device query](../DataQuery/AlignByDeviceQuery.md).
@@ -150,7 +150,7 @@ SQL: `SELECT s2, s1, s2 FROM root.sg.d1;`
为了还原最终结果集，需要构造一个列名到其在查询结果集中位置的映射集
`columnOrdinalMap`，方便从查询结果集中取出某一列对应的结果，该部分逻辑在新建最终结果集 `IoTDBQueryResultSet`
的构造函数内完成。
-- org.apache.iotdb.jdbc.AbstractIoTDBResultSet.AbstractIoTDBResultSet()
+- org.apache.iotdb.jdbc.AbstractIoTDBResultSet.AbstractIoTDBJDBCResultSet()
为了构造最终结果集中的元数据信息，需要构造完整的列名列表，由于上面给出的 `columnNameList`
中不包含时间戳，因此，如果需要打印时间戳则在表头中加入 `Time` 列构成完整的表头。
diff --git a/docs/zh/SystemDesign/SchemaManager/SchemaManager.md
b/docs/zh/SystemDesign/SchemaManager/SchemaManager.md
index 596dd3d..f99b7f0 100644
--- a/docs/zh/SystemDesign/SchemaManager/SchemaManager.md
+++ b/docs/zh/SystemDesign/SchemaManager/SchemaManager.md
@@ -7,9 +7,9 @@
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
-
+
http://www.apache.org/licenses/LICENSE-2.0
-
+
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@@ -29,7 +29,7 @@ IoTDB 的元数据统一由 MManger 管理，包括以下几个部分：
## MManager
-* 维护 tag 倒排索引：`Map<String, Map<String, Set<LeafMNode>>> tagIndex`
+* 维护 tag 倒排索引：`Map<String, Map<String, Set<IMeasurementMNode>>> tagIndex`
> tag key -> tag value -> timeseries LeafMNode
@@ -61,7 +61,6 @@ IoTDB 的元数据统一由 MManger 管理，包括以下几个部分：
* 将所删除的时间序列信息记录到 mlog 中
* 目前并不会删除 tlog 中关于此时间序列的标签/属性信息。
-
* 设置存储组
* 在 MTree 中创建存储组
* 如果开启动态参数，检查内存是否满足
@@ -138,13 +137,13 @@ IoTDB 的元数据统一由 MManger 管理，包括以下几个部分：
* org.apache.iotdb.db.metadata.mtree.MTree
-树中包括三种节点：StorageGroupMNode、InternalMNode（非叶子节点）、LeafMNode（叶子节点），他们都是 MNode 的子类。
+树中包括三种节点：StorageGroupMNode、InternalMNode（非叶子节点）、IMeasurementMNode（叶子节点），他们都是
MNode 的子类。
-每个 InternalMNode 中都有一个读写锁，查询元数据信息时，需要获得路径上每一个 InternalMNode
的读锁，修改元数据信息时，如果修改的是 LeafMNode，需要获得其父节点的写锁，若修改的是 InternalMNode，则只需获得本身的写锁。若该
InternalMNode 位于 Device 层，则还包含了一个`Map<String, MNode> aliasChildren`，用于存储别名信息。
+每个 InternalMNode 中都有一个读写锁，查询元数据信息时，需要获得路径上每一个 InternalMNode
的读锁，修改元数据信息时，如果修改的是 IMeasurementMNode，需要获得其父节点的写锁，若修改的是
InternalMNode，则只需获得本身的写锁。若该 InternalMNode 位于 Device 层，则还包含了一个`Map<String,
MNode> aliasChildren`，用于存储别名信息。
StorageGroupMNode 继承 InternalMNode，包含存储组的元数据信息，如 TTL。
-LeafMNode 中包含了对应时间序列的 Schema 信息，其别名（若没有别名，则为 null) 以及该时间序列的标签/属性信息在 tlog 文件中的
offset（若没有标签/属性，则为-1)
+IMeasurementMNode中包含了对应时间序列的 Schema 信息，其别名（若没有别名，则为 null) 以及该时间序列的标签/属性信息在
tlog 文件中的 offset（若没有标签/属性，则为-1)
示例：
@@ -176,7 +175,7 @@ IoTDB 的元数据管理采用目录树的形式，倒数第二层为设备层
1. 后台线程检查自动创建：每隔 10 分钟，后台线程检查 MTree 的最后修改时间，需要同时满足
* 用户超过 1 小时（可配置）没修改 MTree，即`mlog.bin` 文件超过 1 小时没有修改
* `mlog.bin` 中积累了 100000 行日志（可配置）
-
+
2. 手动创建：使用`create snapshot for schema`命令手动触发创建 MTree 快照
### 创建过程
@@ -188,7 +187,7 @@ IoTDB 的元数据管理采用目录树的形式，倒数第二层为设备层
* 普通节点：0, 名字，子节点个数
* 存储组节点：1, 名字，TTL, 子节点个数
* 传感器节点：2, 名字，别名，数据类型，编码，压缩方式，属性，偏移量，子节点个数
-
+
3. 序列化结束后，将临时文件重命名为正式文件（`mtree.snapshot`），防止在序列化过程中出现服务器人为或意外关闭，导致序列化失败的情况。
4. 调用`MLogWriter.clear()`方法，清空 `mlog.bin`：
* 关闭 BufferedWriter，删除`mlog.bin`文件；
@@ -242,7 +241,7 @@ mlog.bin 存储二进制编码。我们可以使用 [MlogParser Tool](https://io
* 给 root.turbine 设置时间为 10 秒的 ttl
> mlog: 10,root.turbine,10
-
+
> 格式：10,path,ttl
* alter timeseries root.turbine.d1.s1 add tags(tag1=v1)
@@ -253,7 +252,7 @@ mlog.bin 存储二进制编码。我们可以使用 [MlogParser Tool](https://io
> 格式：10,path,[change offset]
* alter timeseries root.turbine.d1.s1 UPSERT ALIAS=newAlias
-
+
> mlog: 13,root.turbine.d1.s1,newAlias
> 格式：13,path,[new alias]
@@ -262,27 +261,27 @@ mlog.bin 存储二进制编码。我们可以使用 [MlogParser Tool](https://io
s1 INT32 with encoding=Gorilla and compression SNAPPY,
s2 FLOAT with encoding=RLE and compression=SNAPPY
)
-
+
> mlog:5,temp1,0,s1,1,8,1
-
+
> mlog:5,temp1,0,s2,3,2,1
-
+
> 格式： 5,template name,is Aligned
Timeseries,measurementId,TSDataType,TSEncoding,CompressionType
* 在某前缀路径上设置元数据模版 set schema template temp1 to root.turbine
-
+
> mlog: 6,temp1,root.turbine
> 格式：6,template name,path
* 自动创建设备 （应用场景为在某个前缀路径上设置模版之后，写入时会自动创建设备）
-
+
> mlog: 4,root.turbine.d1
> 格式：4,path
* 设置某设备正在使用模版 （应用场景为在某个设备路径上设置模版之后，表示该设备正在应用模版）
-
+
> mlog: 61,root.turbine.d1
> 格式：61,path
diff --git a/docs/zh/SystemDesign/StorageEngine/DataManipulation.md
b/docs/zh/SystemDesign/StorageEngine/DataManipulation.md
index 5a84036..812b4b1 100644
--- a/docs/zh/SystemDesign/StorageEngine/DataManipulation.md
+++ b/docs/zh/SystemDesign/StorageEngine/DataManipulation.md
@@ -32,7 +32,7 @@
* Session 的 insertRecord 和 insertRecords
* 总入口：public void insert(InsertRowPlan insertRowPlan) StorageEngine.java
- * 找到对应的 StorageGroupProcessor
+ * 找到对应的 VirtualStorageGroupProcessor
* 根据写入数据的时间以及当前设备落盘的最后时间戳，找到对应的 TsFileProcessor
* 写入 TsFileProcessor 对应的 memtable 中
* 如果是乱序文件，则更新 tsfileResource 中的 endTimeMap
@@ -48,7 +48,7 @@
* Session 的 insertTablet
* 总入口：public void insertTablet(InsertTabletPlan insertTabletPlan)
StorageEngine.java
- * 找到对应的 StorageGroupProcessor
+ * 找到对应的 VirtualStorageGroupProcessor
* 根据这批数据的时间以及当前设备落盘的最后时间戳，将这批数据分成小批，分别对应到一个 TsFileProcessor 中
* 分别将每小批写入 TsFileProcessor 对应的 memtable 中
* 如果是乱序文件，则更新 tsfileResource 中的 endTimeMap
@@ -71,14 +71,14 @@
* JDBC 的 execute 接口，使用 delete SQL 语句
-每个 StorageGroupProsessor 中针对每个分区会维护一个自增的版本号，由 SimpleFileVersionController 管理。
+每个 VirtualStorageGroupProcessor 中针对每个分区会维护一个自增的版本号，由
SimpleFileVersionController 管理。
每个内存缓冲区 memtable 在持久化的时候会申请一个版本号。持久化到 TsFile 后，会在 TsFileMetadata 中记录此 memtable
对应的 多个 ChunkGroup 的终止位置和版本号。
查询时会根据此信息对 ChunkMetadata 赋 version。
StorageEngine.java 中的 delete 入口：
```public void delete(String deviceId, String measurementId, long timestamp)```
- * 找到对应的 StorageGroupProcessor
+ * 找到对应的 VirtualStorageGroupProcessor
* 找到受影响的所有 working TsFileProcessor 记录写前日志
* 找到受影响的所有 TsFileResource，在其对应的 mods 文件中记录一条记录：path，version，startTime，endTime
* 如果存在 working memtable：则删除内存中的数据
@@ -94,8 +94,8 @@ Mods 文件用来存储所有的删除记录。下图的 mods 文件中，d1.s1
* JDBC 的 execute 接口，使用 SET TTL 语句
* 总入口：public void setTTL(String storageGroup, long dataTTL) StorageEngine.java
- * 找到对应的 StorageGroupProcessor
- * 在 StorageGroupProcessor 中设置新的 data ttl
+ * 找到对应的 VirtualStorageGroupProcessor
+ * 在 VirtualStorageGroupProcessor 中设置新的 data ttl
* 对所有 TsfileResource 进行 TTL 检查
* 如果某个文件在当前 TTL 下失效，则删除文件
