A1: Yes. However, in IoTDB queries, the order of chunks is not that important as a simple sort by startTime can make them ordered again once they are loaded into memory.
A2: Yes, but not necessarily. We handle time series one by one to minimize the memory burden, but if memory is abundant, we can handle several time series at the same time. Nevertheless, handing multiple time series at the same time does not seem to bring any significant advantages. Eventually all data in the unsequential file is merged, so no marks need leaving. In case of system failure, I also design a merge log to fast recover and I will explain it in the next mail. > 在 2019年7月8日，下午4:10，Xiangdong Huang <saint...@gmail.com> 写道： > > Hi, > > Q1: you change the order of Chunks in a TsFile. Does that break the > time-ordering characteristic? > > Q2: Do you mean handling time series one by one (An unseq file may consists > of many devices and measurements)? Do we need to make some marks on an unseq > file if only the data of a part of devices are merged? > > Best. > ----------------------------------- > Xiangdong Huang > School of Software, Tsinghua University > > 黄向东 > 清华大学 软件学院 > > > 江天 <jt2594...@163.com <mailto:jt2594...@163.com>> 于2019年7月8日周一 下午3:49写道： > I have worked out a simple solution of merge, the following picture explains > the basic procedure of a merge, if you cannot see the picture, please visit > http://assets.processon.com/chart_image/5d22a081e4b0878e40a8a4ae.png > <http://assets.processon.com/chart_image/5d22a081e4b0878e40a8a4ae.png>. The > procedure consists of 6 steps: > 1. Build a MergeReader over the unsequential files. > 2. Read data from the MergeReader by time-ascending order. > 3. For each unsequential data point, find its corresponding chunk in > sequential file. > 4. Merge unsequantial data points with its corresponding sequential chunk to > generate a new chunk in a new file. > 5.1 If unmerged chunks are the minority, append unmerged chunks to the new > file and generate metadata for it. > 5.2 If merged chunks are the minority, remove metadata in the old file, > append merged chunks to the old file and re-generate metadata for it, > ignoring the chunks that have been merged in the old file. > 6. Use the new file to replace the old file and remove useless files. > > >
