This is an automated email from the ASF dual-hosted git repository. qiaojialin pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-iotdb.git
commit 3565f133a4322e55911a964d16128959a6b69308 Author: qiaojialin <[email protected]> AuthorDate: Tue Apr 7 12:00:59 2020 +0800 update delete doc --- docs/SystemDesign/5-DataQuery/8-ModificationHandle.md | 18 ++++++++++++++---- .../SystemDesign/5-DataQuery/8-ModificationHandle.md | 18 ++++++++++++++---- 2 files changed, 28 insertions(+), 8 deletions(-) diff --git a/docs/SystemDesign/5-DataQuery/8-ModificationHandle.md b/docs/SystemDesign/5-DataQuery/8-ModificationHandle.md index 1166250..2871332 100644 --- a/docs/SystemDesign/5-DataQuery/8-ModificationHandle.md +++ b/docs/SystemDesign/5-DataQuery/8-ModificationHandle.md @@ -21,25 +21,35 @@ # Modification handling in query +Background: + Data deletion only record a mods file for disk data, the data is not really deleted. Therefore, we need to consider the modifications in query. -Each timeseries is treated independently in query process. For each timeseries, there are 5 levels: TsFileResource -> TimeseriesMetadata -> ChunkMetadata -> IPageReader -> BatchData +If a TsFile is influenced by deletion, a deletion operation will be recorded in its mods file. The log contains 3 parts: path, deleted time, version + +## Related class + +mods file: org.apache.iotdb.db.engine.modification.ModificationFile -Query resource: TsFileResource and possibly exist mods file. If a TsFile is influenced by deletion, a modification log will be recorded in its mods file. The log contains 3 parts: path, deleted time, version +deletion operation: org.apache.iotdb.db.engine.modification.Modification + +## Query process  +Each timeseries is treated independently in query process. For each timeseries, there are 5 levels: TsFileResource -> TimeseriesMetadata -> ChunkMetadata -> IPageReader -> BatchData + * TsFileResource -> TimeseriesMetadata ``` -// Set the statistics in TimeseriesMetadata unusable if the timeseries contains modifications +// Set the statistics in TimeseriesMetadata unusable if the timeseries contains deletion operations FileLoaderUtils.loadTimeseriesMetadata() ``` * TimeseriesMetadata -> List\<ChunkMetadata\> ``` -// For each ChunkMetadata, find the largest timestamp in all modifications whose version is larger than it. Set deleted time to ChunkMetadata. +// For each ChunkMetadata, find the largest timestamp in all deletion operations whose version is larger than it. Set deleted time to ChunkMetadata. // set the statistics in ChunkMetadata is unusable if it is affected by deletion FileLoaderUtils.loadChunkMetadataList() ``` diff --git a/docs/zh/SystemDesign/5-DataQuery/8-ModificationHandle.md b/docs/zh/SystemDesign/5-DataQuery/8-ModificationHandle.md index ce4ff7b..d3357ed 100644 --- a/docs/zh/SystemDesign/5-DataQuery/8-ModificationHandle.md +++ b/docs/zh/SystemDesign/5-DataQuery/8-ModificationHandle.md @@ -21,25 +21,35 @@ # 查询中的数据修改处理 +背景介绍： + 数据删除操作对磁盘数据只记录了 mods 文件，并未真正执行删除逻辑，因此查询时需要考虑数据删除的逻辑。 -查询时每个时间序列会单独处理。针对一个时间序列，由大到小有 5 个层次：TsFileResource -> TimeseriesMetadata -> ChunkMetadata -> IPageReader -> BatchData +如果一个文件中有数据被删除了，将删除操作记录到 mods 文件中。记录三列：删除的时间序列，删除范围的最大时间点，删除操作对应的版本。 + +## 相关类 + +mods 文件: org.apache.iotdb.db.engine.modification.ModificationFile -查询资源：TsFileResource 以及可能存在的 mods 文件，如果一个文件中有数据被删除了，将删除操作记录到 mods 文件中。记录三列：删除的时间序列，删除范围的最大时间点，删除操作对应的版本。 +删除操作: org.apache.iotdb.db.engine.modification.Modification + +## 查询流程  +查询时每个时间序列会单独处理。针对一个时间序列，由大到小有 5 个层次：TsFileResource -> TimeseriesMetadata -> ChunkMetadata -> IPageReader -> BatchData + * TsFileResource -> TimeseriesMetadata ``` -// 只要这个时间序列有对应的 modification，就标记 TimeseriesMetadata 中的统计信息不可用 +// 只要这个时间序列有对应的删除操作，就标记 TimeseriesMetadata 中的统计信息不可用 FileLoaderUtils.loadTimeseriesMetadata() ``` * TimeseriesMetadata -> List\<ChunkMetadata\> ``` -// 对于每个 ChunkMetadata，找到比其 version 大的所有 modification 中最大时间戳, 设置到 ChunkMetadata 的 deleteAt 中，并标记 统计信息不可用 +// 对于每个 ChunkMetadata，找到比其 version 大的所有删除操作中最大时间戳, 设置到 ChunkMetadata 的 deleteAt 中，并标记 统计信息不可用 FileLoaderUtils.loadChunkMetadataList() ```
