I think both avg_series_point_number_threshold and memtable_size_threshold parameters can be calculated by IoTDB itself. On one hand, the time series number of one memtable should be considered for parameter avg_series_point_number_threshold. On the other hand, the memory size allocated for write and the number of storage groups should be considered for parameter memtable_size_threshold. Besides, it is an interesting and challenging issue to manage high frequency time series and low frequency time series, hot data and cold data.
Best, Haiming Zhu Haonan Hou <[email protected]> 于2022年5月17日周二 14:46写道： > Hi, > > As we know, we have a parameter avg_series_point_number_threshold=10000, > which controls the maximum point number of one series in memtable can not > exceed the threshold. > The main purpose of this configuration is solve the problem that if the > number of point in one TVList is too large, it will take too much sorting > time on query and flush. > > However, we recently got many feedback about when import historical data > series by series, each time we insert a series with 10,000 points, then the > 10,000 points will form a tsfile. > As a result, there would be a lot of small tsfiles after the data imported > and the query efficiency is not as good as we expected. > > Therefore, I would like to change the default value of this parameter to > 100,000. Two reasons about it. One, change the threshold larger can improve > the efficiency of the scenario > above. Besides, it may not have any side effect to enlarge the number on > other application scenario. Someone may remember, this configuration was > actually introduced to IoTDB > before the memory control mechanism added. After we have the memory > control, this threshold almost cannot be reached except the importing > historical data scenario. > > This is the first step to solve the problem. We may need to find a more > smart strategy to replace this parameter in the future. > > How do you think? > > BR, > Haonan Hou > > >
