Re: Introducing a memory control mechanism during the query planning stage #12573

[Yuan Tian]

Hi lanyu,

Nice work!

On Thu, May 23, 2024 at 11:09 AM Liao Lanyu <1435078...@qq.com.invalid>
wrote:

> Hi,
> Currently, the IoTDB query engine does not implement memory control at the
> FE (Frontend) stage. In scenarios with massive series queries (e.g., select
> * from root.**), the query plans generated at the FE stage can become
> excessively large. Roughly estimating, the size of a single SeriesScanNode
> is about 1/2 KB, which means that two million series corresponding to two
> million SeriesScanNodes would occupy 1 GB, posing a potential risk of
> Out-Of-Memory (OOM). In high concurrency scenarios, even if a single query
> plan is not large, the total memory occupied by multiple query plans can
> still lead to OOM.
> Therefore, it is now desired to introduce memory size control for FE query
> plans within the query engine.
> The PR is:&nbsp;https://github.com/apache/iotdb/pull/12573
>
>
>
>
> 1435078631
> 1435078...@qq.com
>
>
>
> &nbsp;