Z-SWEI commented on issue #59889: URL: https://github.com/apache/doris/issues/59889#issuecomment-3753229999
> How many tablets and how much memory is consumed? To add more context with concrete numbers: In our current cluster, the tablet count reaches tens of millions. During FE metadata checkpoint, we observe a significant memory spike, with peak memory usage reaching around 300 GB. It appears that during the checkpoint process, a large amount of metadata objects are held in memory at the same time, which causes this sharp increase in memory usage. This is why we’re particularly interested in potential optimizations around FE metadata footprint and checkpoint behavior. 当前集群规模下，tablet 数量达到几千万级别。在 FE 执行 metadata checkpoint 过程中，内存占用会出现明显峰值，最高可达到 300GB。 从现象看，checkpoint 阶段会同时持有大量元数据对象，导致瞬时内存压力非常大，这也是我们比较关心 FE 元数据进一步瘦身和 checkpoint 行为优化的原因。 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
