[
https://issues.apache.org/jira/browse/IOTDB-6036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17738464#comment-17738464
]
Zhijia Cao commented on IOTDB-6036:
-----------------------------------
add the mods file settle tool, when mods file size greater than 1M, this tool
will execute compact,merge and sort the same path time series.
> [delete timeseries] the mods file is too large, causing Query very slow even
> OOM problem
> ----------------------------------------------------------------------------------------
>
> Key: IOTDB-6036
> URL: https://issues.apache.org/jira/browse/IOTDB-6036
> Project: Apache IoTDB
> Issue Type: Bug
> Components: Core/Server
> Reporter: Zhijia Cao
> Priority: Blocker
> Fix For: master branch
>
> Attachments: Snipaste_2023-06-29_17-52-27.png, mods.png
>
>
> In some scenarios, users perform ttl operations by deleting the time series.
> There is a user every 5 seconds to delete 14 days old data, pseudo-SQL is as
> follows:
> {code:java}
> delete timeseres root.cisdi.im.steel.sentry.** where time < systemTime - 14d
> {code}
> Each execution of this SQL will append a line of records to the mods file,
> and after a while, the file will become very large, as shown in the
> attachment, the mods file has reached 200 of megabytes, and the deleted
> records in the file have been about 3 million lines.
>
> When you query the contents related to this sequence, the query engine
> filters out the invalid contents in the mods file. Therefore, all the
> contents of the mods file are loaded into the memory. This operation may
> cause that the query result cannot be loaded for a long time or even the OOM
> result is displayed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
