Sicheng Yu created IOTDB-6209:
---------------------------------
Summary: Pipe: Solving the topological order of the progress index
in the historical data collection phase
Key: IOTDB-6209
URL: https://issues.apache.org/jira/browse/IOTDB-6209
Project: Apache IoTDB
Issue Type: Bug
Reporter: Sicheng Yu
Assignee: Sicheng Yu
Problems:
When the pipe performs historical data collection, it currently sends
sequential data first and then disorganized data, which is obviously wrong
because some of the disorganized data may have a smaller progress index than
the sequential file, causing the pipe to record the wrong progress index as
progress information, resulting in some of the disorganized files not being
sent.
Solution:
After collecting all the historical data, construct a DAG according to the
progress index range contained in the TsFile to solve the topological order.
Use the topological order of the files as the order in which the historical
data is collected.
问题：
在 pipe 进行历史数据收集的时候，目前是先发送顺序数据，再发送乱序数据，这明显是错误的，因为部分乱序数据的 progress index
可能比顺序文件更小，导致 pipe 会记录错误的 progress index 作为进度信息，导致部分乱序文件没有被发送。
解决方案：
在收集完所有历史数据之后，按照 TsFile 包含的 progress index 范围构造 DAG，求解拓扑序。使用文件的拓扑序作为历史数据收集的顺序。
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
