GitHub user yatriks added a comment to the discussion: Memory leak and confusing logs
@git-hulk yes, the second test was with the `max_compaction_bytes` as well. I left the load running for a few more days, and had another memory-related oom kill (with both kvrocks and docker reported memory going beyond the limit when kill happens). I think I've realized one other thing, since I'm trying to use 'realistic' load testing, I set up very large batches of inserts, but when the batches run out after ~4 days or so, the select loads run unimpeded, hammering the server and potentially triggering the large memory usage. I'm now going to run two tests: 1. intensive select workload only (large sunions), see if that causes the memory spike to happen any faster / at the time of a compaction, and after that, 2. try disabling auto compactions and seeing if that changes anything. Thanks for your guidance. I was surprised to see this last test also run into a memory spike, and operationally I'm sure this kind of workload would only happen in bursts at first but I would like to continue investigating why it happens at all. Appreciate any additional thoughts and insights :pray: GitHub link: https://github.com/apache/kvrocks/discussions/2984#discussioncomment-13273848 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
