caipengbo commented on PR #2047:
URL: https://github.com/apache/kvrocks/pull/2047#issuecomment-1907634891
Your idea is similar to what I did on our personalized version kvrocks.
@jihuayu
It's just that we don't want to save disk space, we want to avoid OOM,
because if we delete a lot of data, we can create a very large WAL, which
caches a lot of data in memory, which is very easy to cause OOM(we have
encountered it), and even worse, we can't restart our program, because we need
to replay the WAL at startup. At this point, all unflushed data will be loaded
into memory, again leading to an OOM!
So we find all the places in the program where kvrocks can generate a large
WAL, check it, and if it's larger than `max-write-batch-size`, we return an
error so the user can't do it.
> This is a feature I plan to implement in some subsequent PRs (about memory
control).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
