Hi hongying, looking forward to your unified memory management framework for IoTDB.
On Thu, May 23, 2024 at 12:11 PM Hongyin Zhang <[email protected]> wrote: > +1, brilliant job! This work allowed me to learn a lot, which laid a solid > foundation for subsequent system memory control and improved the system's > robustness in query scenarios. > > Best regards, > Hongyin Zhang > > > > > > > > > > > > At 2024-05-23 11:08:35, "Liao Lanyu" <[email protected]> wrote: > >Hi, > >Currently, the IoTDB query engine does not implement memory control at > the FE (Frontend) stage. In scenarios with massive series queries (e.g., > select * from root.**), the query plans generated at the FE stage can > become excessively large. Roughly estimating, the size of a single > SeriesScanNode is about 1/2 KB, which means that two million series > corresponding to two million SeriesScanNodes would occupy 1 GB, posing a > potential risk of Out-Of-Memory (OOM). In high concurrency scenarios, even > if a single query plan is not large, the total memory occupied by multiple > query plans can still lead to OOM. > >Therefore, it is now desired to introduce memory size control for FE > query plans within the query engine. > >The PR is: https://github.com/apache/iotdb/pull/12573 > > > > > > > > > >1435078631 > >[email protected] > > > > > > > > >
