Alex, I think offheap memory preallocation will not be helpful as you expect: 1. We still will not have any guarantee JVM process won't be killed by OOM-killer due to memory overcommiting e.g. for GC needs. as JVM allocates memory for internal needs in same process memory space. 2. Startup process will costs as we'll need to touch every offheap page.
We can't try to add some checks on startup to warn a user in case of low physical memory (based on ram size + swap size + overcommit settings), but I have no idea how we can estimate memory for JVM internal needs (threads, GC and others). On Wed, Oct 24, 2018 at 11:25 AM Andrey Mashenkov < andrey.mashen...@gmail.com> wrote: > Docs [1] says, that OOM can also be thrown when native library can't > allocate memory chunk if physical memory (ram + swap). > > >> Shouldn't OOM happen if you're trying to allocate beyond the virtual > memory capacity (beyond X GB)? > With 64-bit addressing you have some exabytes of virtual memory, is it > possible to have such amount of physical memory? > > I'd think JVM process should be killed by OS much earlier due to memory > overcommit. See [2]. > > [1] > https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/memleaks002.html > [2] > http://engineering.pivotal.io/post/virtual_memory_settings_in_linux_-_the_problem_with_overcommit/ > > On Wed, Oct 24, 2018 at 7:12 AM Denis Magda <dma...@apache.org> wrote: > >> Alex, >> >> Correct me if I'm wrong, but even if an OS runs out of physical memory (X >> GB in total) an Ignite node process still can request the X GB from >> virtual >> memory. Yes, virtual memory can involve swapping and disk to satisfy your >> request but this shouldn't be a reason of the OOM. Shouldn't OOM happen if >> you're trying to allocate beyond the virtual memory capacity (beyond X >> GB)? >> >> Denis >> >> On Tue, Oct 23, 2018 at 12:08 PM Gerus <age...@gridgain.com> wrote: >> >> > Hi *Igniters*, >> > Some time ago I've raised a suggestion for product improvement >> > https://issues.apache.org/jira/browse/IGNITE-9112 >> > <https://issues.apache.org/jira/browse/IGNITE-9112> . It's all about >> > off-heap memory allocation. Current implementation can have some >> > improvements for failure critical systems. Ignite can have OOM in >> runtime, >> > because RAM can be used by OS, if it will not be pre-booked by operation >> > system and this proposal is to address that. Common case is offheap and >> > thats why memory segment cannot be managed by JVM that has >> +AlwaysPreTouch >> > option >> > Obviously this implementation will make startup longer and thats why it >> is >> > proposed to use configuration flag to manage this feature >> > I think, it will be useful to have this option. Are you supporting this? >> > >> > >> > >> > -- >> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/ >> > >> > > > -- > Best regards, > Andrey V. Mashenkov > -- Best regards, Andrey V. Mashenkov
