[
https://issues.apache.org/jira/browse/IOTDB-586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17083276#comment-17083276
]
Xiangdong Huang commented on IOTDB-586:
---------------------------------------
Hi [~Jason Chao], how do you think of Tian Jiang's idea?
As he thinks we do not need some search like getting the time-series names that
match "root.a.*.b", so Tree structure is not needed.
I think it is correct.
The only concern is that does Map structure spend too much memory?
> [Distributed] Eliminate some timeseries schemas When memory is limited
> ----------------------------------------------------------------------
>
> Key: IOTDB-586
> URL: https://issues.apache.org/jira/browse/IOTDB-586
> Project: Apache IoTDB
> Issue Type: Improvement
> Components: Core/Cluster
> Reporter: WangChao
> Priority: Minor
>
> Some businesses have 400 million time series. It will be very memory
> consuming to stored all of them. We need to eliminate some unusual time
> series schemas.
>
> The possible way is,
> 1. The time sequence pulled from the remote is stored in a new MTREE. The
> retrieval can retrieve two mtrees at the same time, and only this remote
> MTREE will be eliminated
> 2. All schemas exist in one MTREE. Scan the whole MTREE to find the remote
> schema for elimination
>
> There are also two ways to elimination:
> 1. Direct discard
> 2. Save to local file
>
> When does it need to be eliminated?
> 1. Limit the number of time series of an MTREE, exceeding that of elimination
> 2. Limit memory usage of MTREE (for example, 500MB), which will be eliminated
> if exceeded
>
> I tend to maintain one MTREE, discard it directly, and limit the number of
> time series of MTREE.
> The advantages are:
> 1. Under normal circumstances, it takes less time to read metadata from the
> remote memory than from the disk randomly. If it is loaded from the disk, it
> means that the memory is full, and it needs to eliminate other schemas when
> reading from the disk, which is troublesome
> 2. Query performance may be poor when maintain two MTree
> 3. It is easy to maintain the number of time series
>
> Please leave your opinions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
