This is an automated email from the ASF dual-hosted git repository.
qiaojialin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-iotdb.git
The following commit(s) were added to refs/heads/master by this push:
new a7b40b2 [IOTDB-655] Update documents of new version TsFile (#1186)
a7b40b2 is described below
commit a7b40b29afc56634ef3202732da3a06ea90b0b4b
Author: Zesong Sun <[email protected]>
AuthorDate: Sun May 17 23:11:02 2020 +0800
[IOTDB-655] Update documents of new version TsFile (#1186)
* [IOTDB-655] Update documents of new version TsFile
---
docs/SystemDesign/1-TsFile/2-Format.md | 217 ++++++++------------
docs/UserGuide/3-Server/4-Config Manual.md | 9 +
docs/zh/SystemDesign/1-TsFile/2-Format.md | 222 ++++++++-------------
docs/zh/UserGuide/3-Server/4-Config Manual.md | 9 +
.../tsfile/file/metadata/TimeseriesMetadata.java | 2 +-
5 files changed, 182 insertions(+), 277 deletions(-)
diff --git a/docs/SystemDesign/1-TsFile/2-Format.md
b/docs/SystemDesign/1-TsFile/2-Format.md
index 205ef16..74a48b7 100644
--- a/docs/SystemDesign/1-TsFile/2-Format.md
+++ b/docs/SystemDesign/1-TsFile/2-Format.md
@@ -66,7 +66,7 @@
Here is a graph about the TsFile structure.
-
+
This TsFile contains two devices: d1, d2. Each device contains three
measurements: s1, s2, s3. 6 timeseries in total, d1 is blue, d2 is purple. Each
timeseries contains 2 Chunks.
@@ -78,8 +78,8 @@ There are three parts of metadata
Query Process：e.g., read d1.s1
-* deserialize TsFileMetadata，get the position and length of all
TimeseriesMetadata of d1
-* deserialize all TimeseriesMetadata of d1，get the TimeseriesMetadata of d1.s1
+* deserialize TsFileMetadata，get the position of TimeseriesMetadata of d1.s1
+* deserialize and get the TimeseriesMetadata of d1.s1
* according to TimeseriesMetadata of d1.s1，deserialize all ChunkMetadata of
d1.s1
* according to each ChunkMetadata of d1.s1，read its Chunk
@@ -103,16 +103,14 @@ The `ChunkGroup` has an array of `Chunk`, a following
byte `0x00` as the marker,
A `Chunk` represents the data of a *measurement* in a time range, data points
in Chunks are in time ascending order. There is a byte `0x01` as the marker,
following a `ChunkHeader` and an array of `Page`.
##### ChunkHeader
-
-| Member Description | Member Type |
-| :------------------------------------: | :---------: |
-| The name of this sensor(measurementID) | String |
-| Size of this chunk | int |
-| Data type of this chuck | short |
-| Number of pages | int |
-| Compression Type | short |
-| Encoding Type | short |
-| Max Tombstone Time(unused) | long |
+| Member | Type | Description |
+| :--------------------------: | :----: | :----: |
+| measurementID | String | Name of measurement |
+| dataSize | int | Size of this chunk |
+| dataType | TSDataType | Data type of this chuck |
+| compressionType | CompressionType | Compression Type |
+| encodingType | TSEncoding | Encoding Type |
+| numOfPages | int | Number of pages |
##### Page
@@ -120,159 +118,102 @@ A `Page` represents some data in a `Chunk`. It contains
a `PageHeader` and the a
PageHeader Structure
-| Member Description | Member Type |
-| :----------------------------------------: | :--------------: |
-| Data size before compressing | int |
-| Data size after compressing(if use SNAPPY) | int |
-| Number of values | int |
-| Maximum time stamp | long |
-| Minimum time stamp | long |
-| Maximum value of the page | Type of the page |
-| Minimum value of the page | Type of the page |
-| First value of the page | Type of the page |
-| Sum of the Page | double |
-| Last value of the page | Type of the page |
+| Member | Type | Description |
+| :----------------------------------: | :--------------: | :----: |
+| uncompressedSize | int | Data size before compressing |
+| compressedSize | int | Data size after compressing(if use
SNAPPY) |
+| statistics | Statistics | Statistics values |
##### ChunkGroupFooter
-| Member Description | Member Type |
-| :-------------------------: | :---------: |
-| DeviceId | String |
-| Data size of the ChunkGroup | long |
-| Number of chunks | int |
+| Member | Type | Description |
+| :--------------------------------: | :----: | :----: |
+| deviceID | String | Name of device |
+| dataSize | long | Data size of the ChunkGroup |
+| numberOfChunks | int | Number of chunks |
#### 1.2.3 Metadata
-##### 1.2.3.1 TsDeviceMetaData
-
-The first part of metadata is `TsDeviceMetaData`
+##### 1.2.3.1 ChunkMetadata
-| Member Description | Member Type |
-| :------------------------: | :---------: |
-| Start time | long |
-| End time | long |
-| Number of chunk groups | int |
-| List of ChunkGroupMetaData | list |
+The first part of metadata is `ChunkMetadata`
-###### ChunkGroupMetaData
+| Member | Type | Description |
+| :------------------------------------------------: | :------: | :----: |
+| measurementUid | String | Name of measurement |
+| offsetOfChunkHeader | long | Start offset of ChunkHeader |
+| tsDataType | TSDataType | Data type |
+| statistics | Statistics | Statistic values |
-| Member Description | Member Type |
-| :----------------------------: | :---------: |
-| DeviceId | String |
-| Start offset of the ChunkGroup | long |
-| End offset of the ChunkGroup | long |
-| Version | long |
-| Number of ChunkMetaData | int |
-| List of ChunkMetaData | list |
+As for the five statistics (min, max, first, last and sum), `ChunkMetadata` of
Binary and Boolean type only has two values: first and last.
-###### ChunkMetaData
+##### 1.2.3.2 TimeseriesMetadata
-| Member Description | Member Type |
-| :--------------------------: | :---------: |
-| MeasurementId | String |
-| Start offset of ChunkHeader | long |
-| Number of data points | long |
-| Start time | long |
-| End time | long |
-| Data type | short |
-| The statistics of this chunk | TsDigest |
+The second part of metadata is `TimeseriesMetadata`.
-###### TsDigest
+| Member | Type | Description |
+| :------------------------------------------------: | :------: | :------: |
+| measurementUid | String | Name of measurement |
+| tsDataType | short | Data type |
+| startOffsetOfChunkMetadataList | long | Start offset of ChunkMetadata list
|
+| chunkMetaDataListDataSize | int | ChunkMetadata list size |
+| statistics | Statistics | Statistic values |
-Right now there are five statistics: `min_value, max_value, first_value,
last_value, sum_value`.
+As for the five statistics (min, max, first, last and sum),
`TimeseriesMetadata` of Binary and Boolean type only has two values: first and
last.
-In v0.8.0, the storage format of statistics is a name-value pair. That is,
`Map<String, ByteBuffer> statistics`. The name is a string (remember the length
is before the literal). But for the value, there is also an integer byteLength
acting as the self description length of the following value because the value
may be of various type. For example, if the `min_value` is an integer 0, then
it will be stored as [9 "min_value" 4 0] in the TsFile.
+##### 1.2.3.3 TsFileMetaData
-The figure below shows an example of `TsDigest.deserializeFrom(buffer)`. In
v0.8.0, we will get
+The third part of metadata is `TsFileMetaData`.
-```
-Map<String, ByteBuffer> statistics = {
- "min_value" -> ByteBuffer of int value 0,
- "last" -> ByteBuffer of int value 19,
- "sum" -> ByteBuffer of double value 1093347116,
- "first" -> ByteBuffer of int value 0,
- "max_value" -> ByteBuffer of int value 99
-}
-```
+| Member | Type | Description |
+| :-------------------------------------------------: |
:---------------------: | :---: |
+| MetadataIndex | MetadataIndexNode | MetadataIndex
node |
+| totalChunkNum | int |
total chunk num |
+| invalidChunkNum | int |
invalid chunk num |
+| versionInfo | List<Pair<Long, Long>>
| version information |
+| metaOffset | long | offset of
MetaMarker.SEPARATOR |
+| bloomFilter | BloomFilter
| bloom filter |
-<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/33376433/63765352-664a4280-c8fb-11e9-869e-859edf6d00bb.png";>
+MetadataIndexNode has members as below:
-In v0.9.x, the storage format is changed to an array for space and time
efficiency. That is, `ByteBuffer[] statistics`. Each position of the array has
a fixed association with a specific type of statistic, following the order
defined in StatisticType:
+| Member | Type | Description |
+| :------------------------------------: | :----: | :---: |
+| children | List<MetadataIndexEntry> | MetadataIndexEntry list |
+| endOffset | long | EndOffset of this MetadataIndexNode |
+| nodeType | MetadataIndexNodeType | MetadataIndexNode type |
-```
-enum StatisticType {
- min_value, max_value, first_value, last_value, sum_value
-}
-```
+MetadataIndexEntry has members as below:
-Therefore, in the above example, we will get
+| Member | Type | Description |
+| :------------------------------------: | :----: | :---: |
+| name | String | Name of related device or measurement |
+| offset | long | offset |
-```
-ByteBuffer[] statistics = [
- ByteBuffer of int value 0, // associated with "min_value"
- ByteBuffer of int value 99, // associated with "max_value"
- ByteBuffer of int value 0, // associated with "first_value"
- ByteBuffer of int value 19, // associated with "last_value"
- ByteBuffer of double value 1093347116 // associated with "sum_value"
-]
-```
+All MetadataIndexNode forms a **metadata index tree**, which consists of no
more than two levels: device index level and measurement index level. In
different situation, the tree could have different components. The
MetadataIndexNodeType has four enums: `INTERNAL_DEVICE`, `LEAF_DEVICE`,
`INTERNAL_MEASUREMENT`, `LEAF_MEASUREMENT`, which indicates the internal or
leaf node of device index level and measurement index level respectively. Only
the `LEAF_MEASUREMENT` nodes point to `Timeseries [...]
-As another example in v0.9.x, when deserializing a TsDigest from buffer [3,
0,4,0, 1,4,99, 3,4,19], we get
+To describe the structure of metadata index tree more clearly, we will give
four examples here in details.
-```
-ByteBuffer[] statistics = [
- ByteBuffer of int value 0, // associated with "min_value"
- ByteBuffer of int value 99, // associated with "max_value"
- null, // associated with "first_value"
- ByteBuffer of int value 19, // associated with "last_value"
- null // associated with "sum_value"
-]
-```
+The max degree of the metadata index tree (that is, the max number of each
node's children) could be configured by users, and is 1024 by default. In the
examples below, we assume `max_degree_of_index_node = 10` in the following
examples.
-##### 1.2.3.2 TsFileMetaData
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/19167280/81935219-de3fd080-9622-11ea-9aa1-a59bef1c0001.png";>
-`TsFileMetaData` follows after `TsDeviceMetadatas`.
+5 devices with 5 measurements each: Since the numbers of devices and
measurements are both no more than `max_degree_of_index_node`, the tree has
only measurement index level by default. In this level, each MetadataIndexNode
is composed of no more than 10 MetadataIndex entries. The root nonde is
`INTERNAL_MEASUREMENT` type, and the 5 MetadataIndex entries point to
MetadataIndex nodes of related devices. These nodes point to
`TimeseriesMetadata` directly, as they are `LEAF_MEASUREMENT` type.
-| Member Description | Member Type
|
-| :------------------------------------------: |
:--------------------------------: |
-| Number of devices | int
|
-| Pairs of device name and deviceMetadataIndex | String, TsDeviceMetadataIndex
pair |
-| Number of measurements | int
|
-| Pairs of measurement name and schema | String, MeasurementSchema
pair |
-| Author byte | byte
|
-| Author(if author byte is 0x01) | String
|
-| totalChunkNum | int
|
-| invalidChunkNum | int
|
-| Bloom filter size | int
|
-| Bloom filter bit vector | byte[Bloom filter size]
|
-| Bloom filter capacity | int
|
-| Bloom filter hash functions size | int
|
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/19167280/81935210-d97b1c80-9622-11ea-8a69-2c2c5f05a876.png";>
-###### TsDeviceMetadataIndex
+1 device with 150 measurements: The number of measurements exceeds
`max_degree_of_index_node`, so the tree has only measurement index level by
default. In this level, each MetadataIndexNode is composed of no more than 10
MetadataIndex entries. The nodes that point to `TimeseriesMetadata` directly
are `LEAF_MEASUREMENT` type. Other nodes and root node of index tree are not
leaf nodes of measurement index level, so they are `INTERNAL_MEASUREMENT` type.
-| Member Description | Member Type |
-| :------------------------------: | :---------: |
-| DeviceId | String |
-| Start offset of TsDeviceMetaData | long |
-| length | int |
-| Start time | long |
-| End time | long |
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/19167280/81935182-cd8f5a80-9622-11ea-8e41-661a5219974b.png";>
-###### MeasurementSchema
+150 device with 1 measurement each: The number of devices exceeds
`max_degree_of_index_node`, so the device index level and measurement index
level of the tree are both formed. In these two levels, each MetadataIndexNode
is composed of no more than 10 MetadataIndex entries. The nodes that point to
`TimeseriesMetadata` directly are `LEAF_MEASUREMENT` type. The root nodes of
measurement index level are also the leaf nodes of device index level, which
are `LEAF_DEVICE` type. Other nodes and [...]
-| Member Description | Member Type |
-| :----------------: | :---------: |
-| MeasurementId | String |
-| Data type | short |
-| Encoding | short |
-| Compressor | short |
-| Size of props | int |
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/19167280/81935138-b6e90380-9622-11ea-94f9-c97bd2b5d050.png";>
-If size of props is greater than 0, there is an array of <String, String> pair
as properties of this measurement.
+150 device with 150 measurements each: The numbers of devices and measurements
both exceed `max_degree_of_index_node`, so the device index level and
measurement index level are both formed. In these two levels, each
MetadataIndexNode is composed of no more than 10 MetadataIndex entries. As is
described before, from the root node to the leaf nodes of device index level,
their types are `INTERNAL_DEVICE` and `LEAF_DEVICE`; each leaf node of device
index level can be seen as the root node o [...]
-Such as "max_point_number""2".
+The MetadataIndex is designed as tree structure so that not all the
`TimeseriesMetadata` need to be read when the number of devices or measurements
is too large. Only reading specific MetadataIndex nodes according to
requirement and reducing I/O could speed up the query. More reading process of
TsFile in details will be described in the last section of this chapter.
-##### 1.2.3.3 TsFileMetadataSize
+##### 1.2.3.4 TsFileMetadataSize
After the TsFileMetaData, there is an int indicating the size of the
TsFileMetaData.
@@ -617,10 +558,14 @@ You can also use
`example/tsfile/org/apache/iotdb/tsfile/TsFileSequenceRead` to
### 1.4 A TsFile Visualization Example
-#### v0.8.0
+#### v0.8
<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/33376433/65209576-2bd36000-dacb-11e9-9e43-49e0dd01274e.png";>
-#### v0.9.x
+#### v0.9 / 000001
+
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/33376433/69341240-26012300-0ca4-11ea-91a1-d516810cad44.png";>
+
+#### v0.10 / 000002
-<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/33376433/69341240-26012300-0ca4-11ea-91a1-d516810cad44.png";>
\ No newline at end of file
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/19167280/82010604-299ac300-96a5-11ea-996d-013c0017f669.png";>
diff --git a/docs/UserGuide/3-Server/4-Config Manual.md
b/docs/UserGuide/3-Server/4-Config Manual.md
index 5163315..2059bb4 100644
--- a/docs/UserGuide/3-Server/4-Config Manual.md
+++ b/docs/UserGuide/3-Server/4-Config Manual.md
@@ -141,6 +141,15 @@ The permission definitions are in
${IOTDB\_CONF}/conf/jmx.access.
|Default| 1048576 |
|Effective|Trigger|
+* max\_degree\_of\_index\_node
+
+|Name| max\_degree\_of\_index\_node |
+|:---:|:---|
+|Description|The maximum degree of the metadata index tree (that is, the max
number of each node's children)|
+|Type|Int32|
+|Default| 1024 |
+|Effective|Trigger|
+
* max\_string\_length
|Name| max\_string\_length |
diff --git a/docs/zh/SystemDesign/1-TsFile/2-Format.md
b/docs/zh/SystemDesign/1-TsFile/2-Format.md
index 803034d..749366b 100644
--- a/docs/zh/SystemDesign/1-TsFile/2-Format.md
+++ b/docs/zh/SystemDesign/1-TsFile/2-Format.md
@@ -65,7 +65,7 @@
下图是关于TsFile的结构图。
-
+
此文件包括两个设备 d1、d2，每个设备包含三个测点 s1、s2、s3，共 6 个时间序列，d1为蓝色，d2为紫色。每个时间序列包含两个 Chunk。
@@ -77,8 +77,8 @@
查询流程：以查 d1.s1 为例
-* 反序列化 TsFileMetadata，得到 d1 的所有 TimeseriesMetadata 的位置
-* 反序列化 d1 的所有 TimeseriesMetadata，并得到 d1.s1 的 TimeseriesMetadata
+* 反序列化 TsFileMetadata，得到 d1.s1 的 TimeseriesMetadata 的位置
+* 反序列化得到 d1.s1 的 TimeseriesMetadata
* 根据 d1.s1 的 TimeseriesMetadata，反序列化其所有 ChunkMetadata
* 根据 d1.s1 的每一个 ChunkMetadata，读取其 Chunk 数据
@@ -102,15 +102,14 @@ TsFile文件的内容可以划分为两个部分: 数据（Chunk）和元数据
##### ChunkHeader
-| 成员 | 类型 |
-| :--------------------------: | :----: |
-| 传感器名称(measurementID) | String |
-| chunk大小(dataSize) | int |
-| chunk的数据类型(dataType) | short |
-| 包含的page数量(numOfPages) | int |
-| 压缩类型(compressionType) | short |
-| 编码类型(encodingType) | short |
-| Max Tombstone Time(暂时没用) | long |
+| 成员 | 类型 | 解释 |
+| :--------------------------: | :----: | :----: |
+| measurementID | String | 传感器名称 |
+| dataSize | int | chunk 大小 |
+| dataType | TSDataType | chunk的数据类型 |
+| compressionType | CompressionType | 压缩类型 |
+| encodingType | TSEncoding | 编码类型 |
+| numOfPages | int | 包含的page数量 |
##### Page
@@ -118,163 +117,103 @@ TsFile文件的内容可以划分为两个部分: 数据（Chunk）和元数据
PageHeader 结构
-| 成员 | 类型 |
-| :----------------------------------: | :--------------: |
-| 压缩前数据大小(uncompressedSize) | int |
-| SNAPPY压缩后数据大小(compressedSize) | int |
-| 包含的values的数量(numOfValues) | int |
-| 最大时间戳(maxTimestamp) | long |
-| 最小时间戳(minTimestamp) | long |
-| 该页最大值(max) | Type of the page |
-| 该页最小值(min) | Type of the page |
-| 该页第一个值(first) | Type of the page |
-| 该页值的和(sum) | double |
-| 该页最后一个值(last) | Type of the page |
+| 成员 | 类型 | 解释 |
+| :----------------------------------: | :--------------: | :----: |
+| uncompressedSize | int | 压缩前数据大小 |
+| compressedSize | int | SNAPPY压缩后数据大小 |
+| statistics | Statistics | 统计量 |
##### ChunkGroupFooter
-| 成员 | 类型 |
-| :--------------------------------: | :----: |
-| 设备Id(deviceID) | String |
-| ChunkGroup大小(dataSize) | long |
-| 包含的chunks的数量(numberOfChunks) | int |
+| 成员 | 类型 | 解释 |
+| :--------------------------------: | :----: | :----: |
+| deviceID | String | 设备名称 |
+| dataSize | long | ChunkGroup 大小 |
+| numberOfChunks | int | 包含的 chunks 的数量 |
#### 1.2.3 元数据
-##### 1.2.3.1 TsDeviceMetaData
+##### 1.2.3.1 ChunkMetadata
-第一部分的元数据是 `TsDeviceMetaData`
+第一部分的元数据是 `ChunkMetadata`
-| 成员 | 类型 |
-| :----------------------------------------------: | :--: |
-| 开始时间(startTime) | long |
-| 结束时间(endTime) | long |
-| 包含的ChunkGroup的数量 | int |
-| 所有的ChunkGroupMetaData(chunkGroupMetadataList) | list |
+| 成员 | 类型 | 解释 |
+| :------------------------------------------------: | :------: | :----: |
+| measurementUid | String | 传感器名称 |
+| offsetOfChunkHeader | long | 文件中 ChunkHeader 开始的偏移量 |
+| tsDataType | TSDataType | 数据类型 |
+| statistics | Statistics | 统计量 |
-###### ChunkGroupMetaData
+其中，对于五个统计值(min、max、first、last、sum)，Binary 和 Boolean 类型的 `ChunkMetadata` 只有
first 和 last 两个值。
-| 成员 | 类型 |
-| :-----------------------------------------------------: | :----: |
-| 设备Id(deviceID) | String |
-| 在文件中ChunkGroup开始的偏移量(startOffsetOfChunkGroup) | long |
-| 在文件中ChunkGroup结束的偏移量(endOffsetOfChunkGroup) | long |
-| 版本(version) | long |
-| 包含的ChunkMetaData的数量 | int |
-| 所有的ChunkMetaData(chunkMetaDataList) | list |
+##### 1.2.3.2 TimeseriesMetadata
-###### ChunkMetaData
+第二部分的元数据是 `TimeseriesMetadata`。
-| 成员 | 类型 |
-| :------------------------------------------------: | :------: |
-| 传感器名称(measurementUid) | String |
-| 文件中ChunkHeader开始的偏移量(offsetOfChunkHeader) | long |
-| 数据的总数(numOfPoints) | long |
-| 开始时间(startTime) | long |
-| 结束时间(endTime) | long |
-| 数据类型(tsDataType) | short |
-| chunk的统计信息 | TsDigest |
+| 成员 | 类型 | 解释 |
+| :------------------------------------------------: | :------: | :------: |
+| measurementUid | String | 传感器名称 |
+| tsDataType | TSDataType | 数据类型 |
+| startOffsetOfChunkMetadataList | long | 文件中 ChunkMetadata 列表开始的偏移量 |
+| chunkMetaDataListDataSize | int | ChunkMetadata 列表的大小 |
+| statistics | Statistics | 统计量 |
-###### TsDigest
+其中，对于五个统计值(min、max、first、last、sum)，Binary 和 Boolean 类型的 `TimeseriesMetadata`
只有 first 和 last 两个值。
-目前有五项统计数据: `min_value, max_value, first_value, last_value, sum_value`。
+##### 1.2.3.3 TsFileMetaData
-在 v0.8.0 版本中, 统计数据使用 name-value 编码的键值对。 也就是 `Map<String, ByteBuffer>
statistics`。 name使用的一个字符串类型(需要注意的是字符串前有个长度标识)。 对于值来讲，它有可能是很多种类型，所以需要用 integer
类型用来描述值的长度。 比如, 如果 `min_value` 是一个 integer 类型的 0, 那么在 TsFile 中将被存储为 [9
"min_value" 4 0]。
+第三部分的元数据是 `TsFileMetaData`。
-下面是一个调用 `TsDigest.deserializeFrom(buffer)` 方法后的数据示例。在 v0.8.0 版本中, 我们会得到
+| 成员 | 类型 | 解释 |
+| :-------------------------------------------------: |
:---------------------: | :---:|
+| MetadataIndex | MetadataIndexNode |元数据索引节点 |
+| totalChunkNum | int |
包含的 Chunk 总数 |
+| invalidChunkNum | int |
失效的 Chunk 总数 |
+| versionInfo | List<Pair<Long, Long>>
| 版本信息映射 |
+| metaOffset | long |
MetaMarker.SEPARATOR偏移量 |
+| bloomFilter | BloomFilter
| 布隆过滤器 |
-```
-Map<String, ByteBuffer> statistics = {
- "min_value" -> ByteBuffer of int value 0,
- "last" -> ByteBuffer of int value 19,
- "sum" -> ByteBuffer of double value 1093347116,
- "first" -> ByteBuffer of int value 0,
- "max_value" -> ByteBuffer of int value 99
-}
-```
-
-<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/33376433/63765352-664a4280-c8fb-11e9-869e-859edf6d00bb.png";>
+元数据索引节点 (MetadataIndexNode) 的成员和类型具体如下：
-在 v0.9.0 版本中, 为了提高空间和时间的效率，存储的结构被修改为数组的形式。也就是 `ByteBuffer[]
statistics`。用固定的位置代表某一个具体的统计信息, 在 StatisticType 中定义的顺序如下:
+| 成员 | 类型 | 解释 |
+| :------------------------------------: | :----: | :---: |
+| children | List<MetadataIndexEntry> | 节点元数据索引项列表 |
+| endOffset | long | 此元数据索引节点的结束偏移量 |
+| nodeType | MetadataIndexNodeType | 节点类型 |
-```
-enum StatisticType {
- min_value, max_value, first_value, last_value, sum_value
-}
-```
+元数据索引项 (MetadataIndexEntry) 的成员和类型具体如下：
-修改存储形式后,在上面的示例中,我们将得到
+| 成员 | 类型 | 解释 |
+| :------------------------------------: | :----: | :---: |
+| name | String | 对应设备或传感器的名字 |
+| offset | long | 偏移量 |
-```
-ByteBuffer[] statistics = [
- ByteBuffer of int value 0, // associated with "min_value"
- ByteBuffer of int value 99, // associated with "max_value"
- ByteBuffer of int value 0, // associated with "first_value"
- ByteBuffer of int value 19, // associated with "last_value"
- ByteBuffer of double value 1093347116 // associated with "sum_value"
-]
-```
+所有的元数据索引节点构成一棵**元数据索引树**，这棵树最多由两个层级组成：设备索引层级和传感器索引层级，在不同的情况下会有不同的组成方式。元数据索引节点类型有四种，分别是`INTERNAL_DEVICE`、`LEAF_DEVICE`、`INTERNAL_MEASUREMENT`、`LEAF_MEASUREMENT`，分别对应设备索引层级的中间节点和叶子节点，和传感器索引层级的中间节点和叶子节点。
+只有传感器索引层级的叶子节点(`LEAF_MEASUREMENT`) 指向 `TimeseriesMetadata`。
-另一个关于 v0.9.0 的示例数据, 当我们从 buffer [3, 0,4,0, 1,4,99, 3,4,19] 反序列化为 TsDigest 结构时,
我们将得到
+为了更清楚的说明元数据索引树的结构，这里我们使用四个例子来加以详细说明。
-```
-//这里可能会有些难理解，读取顺序为：1.读取一个int类型的数据总数(3) 2.读取short类型的位于数组中的位置(0)
3.读取int类型的数据长度(4) 4.根据第3步的长度读取数据(0)
-//因为示例数据中，索引值只出现了(0,1,3),所以 first_value sum_value 的值为null
-
-ByteBuffer[] statistics = [
- ByteBuffer of int value 0, // associated with "min_value"
- ByteBuffer of int value 99, // associated with "max_value"
- null, // associated with "first_value"
- ByteBuffer of int value 19, // associated with "last_value"
- null // associated with "sum_value"
-]
-```
+元数据索引树的最大度（即每个节点的最大子节点个数）是可以由用户进行配置的，配置项为`max_degree_of_index_node`，其默认值为1024。在以下例子中，为了简化，我们假定
`max_degree_of_index_node = 10`。
-##### 1.2.3.2 TsFileMetaData
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/19167280/81935219-de3fd080-9622-11ea-9aa1-a59bef1c0001.png";>
-上节讲到的是 `TsDeviceMetadatas` 紧跟其后的数据是 `TsFileMetaData`。
+在5个设备，每个设备有5个传感器的情况下，由于设备数和传感器树均不超过
`max_degree_of_index_node`，因此元数据索引树只有默认的传感器层级。在这个层级里，每个 MetadataIndexNode
最多由10个 MetadataIndexEntry 组成。根节点的 MetadataIndexNode 是 `INTERNAL_MEASUREMENT`
类型，其中的5个 MetadataIndexEntry 指向对应的设备的 MetadataIndexNode，这些节点直接指向
`TimeseriesMetadata`，是 `LEAF_MEASUREMENT`。
-| 成员 | 类型
|
-| :-------------------------------------------------: |
:--------------------------------: |
-| 包含的设备个数 | int
|
-| 设备名称和设备元数据索引的键值对(deviceIndexMap) | String, TsDeviceMetadataIndex pair |
-| 包含的传感器个数 | int
|
-| 传感器名称和传感器元数据的键值对(measurementSchema) | String, MeasurementSchema pair |
-| 水印标识 | byte
|
-| 当标识为0x01时的水印信息(createdBy) | String
|
-| 包含的Chunk总数(totalChunkNum) | int
|
-| 失效的Chunk总数(invalidChunkNum) | int
|
-| 布隆过滤器序列化大小 | int
|
-| 布隆过滤器所有数据 | byte[Bloom filter size]
|
-| 布隆过滤器容量 | int
|
-| 布隆过滤器容量包含的HashFunction数量 | int
|
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/19167280/81935210-d97b1c80-9622-11ea-8a69-2c2c5f05a876.png";>
-###### TsDeviceMetadataIndex
+在1个设备，设备中有150个传感器的情况下，传感器个数超过了
`max_degree_of_index_node`，元数据索引树有默认的传感器层级。在这个层级里，每个 MetadataIndexNode 最多由10个
MetadataIndexEntry 组成。直接指向 `TimeseriesMetadata`的节点类型均为
`LEAF_MEASUREMENT`；而后续产生的中间节点和根节点不是传感器索引层级的叶子节点，这些节点是 `INTERNAL_MEASUREMENT`。
-| 成员 | 类型 |
-| :------------------------------------: | :----: |
-| 设备名 | String |
-| 文件中TsDeviceMetaData的偏移量(offset) | long |
-| 序列化后数据大小(len) | int |
-| 存储的设备最小时间(startTime) | long |
-| 存储的设备最大时间(endTime) | long |
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/19167280/81935182-cd8f5a80-9622-11ea-8e41-661a5219974b.png";>
-###### MeasurementSchema
+在150个设备，每个设备中有1个传感器的情况下，设备个数超过了
`max_degree_of_index_node`，形成元数据索引树的传感器层级和设备索引层级。在这两个层级里，每个 MetadataIndexNode
最多由10个 MetadataIndexEntry 组成。直接指向 `TimeseriesMetadata` 的节点类型为
`LEAF_MEASUREMENT`，传感器索引层级的根节点同时作为设备索引层级的叶子节点，其节点类型为
`LEAF_DEVICE`；而后续产生的中间节点和根节点不是设备索引层级的叶子节点，因此节点类型为 `INTERNAL_DEVICE`。
-| 成员 | 类型 |
-| :-----------------------: | :-----------------: |
-| 传感器名称(measurementId) | String |
-| 数据类型(type) | short |
-| 编码方式(encoding) | short |
-| 压缩方式(compressor) | short |
-| 附带参数的数量 | int |
-| 所有附带的参数(props) | String, String pair |
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/19167280/81935138-b6e90380-9622-11ea-94f9-c97bd2b5d050.png";>
-如果附带的参数数量大于 0, 传感器的附带参数会以一个数组形式的 <String, String> 键值对存储。
+在150个设备，每个设备中有150个传感器的情况下，传感器和设备个数均超过了
`max_degree_of_index_node`，形成元数据索引树的传感器层级和设备索引层级。在这两个层级里，每个 MetadataIndexNode
均最多由10个 MetadataIndexEntry 组成。如前所述，从根节点到设备索引层级的叶子节点，类型分别为`INTERNAL_DEVICE` 和
`LEAF_DEVICE`，而每个设备索引层级的叶子节点都是传感器索引层级的根节点，从这里到传感器索引层级的叶子节点，类型分别为`INTERNAL_MEASUREMENT`
和 `LEAF_MEASUREMENT`。
-比如说: "max_point_number""2".
+元数据索引采用树形结构进行设计的目的是在设备数或者传感器数量过大时，可以不用一次读取所有的
`TimeseriesMetadata`，只需要根据所读取的传感器定位对应的节点，从而减少 I/O，加快查询速度。有关 TsFile
的读流程将在本章最后一节加以详细说明。
-##### 1.2.3.3 TsFileMetadataSize
+##### 1.2.3.4 TsFileMetadataSize
在TsFileMetaData之后，有一个int值用来表示TsFileMetaData的大小。
@@ -607,7 +546,6 @@ file length: 33436
33436| END of TsFile
---------------------------------- TsFile Sketch End
----------------------------------
-
````````````````````````
#### 1.3.4 TsFileSequenceRead
@@ -616,10 +554,14 @@ file length: 33436
### 1.4 TsFile 的总览图
-#### v0.8.0
+#### v0.8
<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/33376433/65209576-2bd36000-dacb-11e9-9e43-49e0dd01274e.png";>
-#### v0.9.0
+#### v0.9 / 000001
+
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/33376433/69341240-26012300-0ca4-11ea-91a1-d516810cad44.png";>
+
+#### v0.10 / 000002
-<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/33376433/69341240-26012300-0ca4-11ea-91a1-d516810cad44.png";>
\ No newline at end of file
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto;
margin-right:auto; display:block;"
src="https://user-images.githubusercontent.com/19167280/82010604-299ac300-96a5-11ea-996d-013c0017f669.png";>
diff --git a/docs/zh/UserGuide/3-Server/4-Config Manual.md
b/docs/zh/UserGuide/3-Server/4-Config Manual.md
index 7e77992..1361cf5 100644
--- a/docs/zh/UserGuide/3-Server/4-Config Manual.md
+++ b/docs/zh/UserGuide/3-Server/4-Config Manual.md
@@ -112,6 +112,15 @@
|默认值| 1048576 |
|改后生效方式|触发生效|
+* max\_degree\_of\_index\_node
+
+|名字| max\_degree\_of\_index\_node |
+|:---:|:---|
+|描述|元数据索引树的最大度（即每个节点的最大子节点个数）|
+|类型|Int32|
+|默认值| 1024 |
+|改后生效方式|触发生效|
+
* max\_string\_length
|名字| max\_string\_length |
diff --git
a/tsfile/src/main/java/org/apache/iotdb/tsfile/file/metadata/TimeseriesMetadata.java
b/tsfile/src/main/java/org/apache/iotdb/tsfile/file/metadata/TimeseriesMetadata.java
index 5c7e1aa..4893563 100644
---
a/tsfile/src/main/java/org/apache/iotdb/tsfile/file/metadata/TimeseriesMetadata.java
+++
b/tsfile/src/main/java/org/apache/iotdb/tsfile/file/metadata/TimeseriesMetadata.java
@@ -64,7 +64,7 @@ public class TimeseriesMetadata implements Accountable {
timeseriesMetaData.setTSDataType(ReadWriteIOUtils.readDataType(buffer));
timeseriesMetaData.setOffsetOfChunkMetaDataList(ReadWriteIOUtils.readLong(buffer));
timeseriesMetaData.setDataSizeOfChunkMetaDataList(ReadWriteIOUtils.readInt(buffer));
- timeseriesMetaData.statistics = Statistics.deserialize(buffer,
timeseriesMetaData.tsDataType);
+ timeseriesMetaData.setStatistics(Statistics.deserialize(buffer,
timeseriesMetaData.tsDataType));
return timeseriesMetaData;
}
![https://user-images.githubusercontent.com/33376433/63765352-664a4280-c8fb-11e9-869e-859edf6d00bb.png"](https://user-images.githubusercontent.com/33376433/63765352-664a4280-c8fb-11e9-869e-859edf6d00bb.png"){kind=link}
![https://user-images.githubusercontent.com/19167280/81935219-de3fd080-9622-11ea-9aa1-a59bef1c0001.png"](https://user-images.githubusercontent.com/19167280/81935219-de3fd080-9622-11ea-9aa1-a59bef1c0001.png"){kind=link}
![https://user-images.githubusercontent.com/19167280/81935210-d97b1c80-9622-11ea-8a69-2c2c5f05a876.png"](https://user-images.githubusercontent.com/19167280/81935210-d97b1c80-9622-11ea-8a69-2c2c5f05a876.png"){kind=link}
![https://user-images.githubusercontent.com/19167280/81935182-cd8f5a80-9622-11ea-8e41-661a5219974b.png"](https://user-images.githubusercontent.com/19167280/81935182-cd8f5a80-9622-11ea-8e41-661a5219974b.png"){kind=link}
![https://user-images.githubusercontent.com/19167280/81935138-b6e90380-9622-11ea-94f9-c97bd2b5d050.png"](https://user-images.githubusercontent.com/19167280/81935138-b6e90380-9622-11ea-94f9-c97bd2b5d050.png"){kind=link}
![https://user-images.githubusercontent.com/33376433/65209576-2bd36000-dacb-11e9-9e43-49e0dd01274e.png"](https://user-images.githubusercontent.com/33376433/65209576-2bd36000-dacb-11e9-9e43-49e0dd01274e.png"){kind=link}
![https://user-images.githubusercontent.com/33376433/69341240-26012300-0ca4-11ea-91a1-d516810cad44.png"](https://user-images.githubusercontent.com/33376433/69341240-26012300-0ca4-11ea-91a1-d516810cad44.png"){kind=link}
![https://user-images.githubusercontent.com/19167280/82010604-299ac300-96a5-11ea-996d-013c0017f669.png"](https://user-images.githubusercontent.com/19167280/82010604-299ac300-96a5-11ea-996d-013c0017f669.png"){kind=link}