Re: OFFHEAP_TIERED and memory fragmentation

Hi Alexey, I see. That sounds reasonable on that level (and is something different). What is your plan to limit the overhead/impact of detecting and moving around such small chunks? Kind regards, Peter 2017-02-03 14:52 GMT+01:00 Alexey Goncharuk : > Peter, > > A

Re: OFFHEAP_TIERED and memory fragmentation

Peter, A page is a minimal entity that is allowed to be moved on OS level. If I have a chunk of memory allocated in the middle of a page, OS won't be able to move it during defragmentation (at least this is how things worked last time I checked). In the new approach, we are able to move this

Re: OFFHEAP_TIERED and memory fragmentation

Hi Alexey, a modern OS won't give you a physical address. Instead you get virtual memory which allows the OS to support features like defragmentation. "fine-grained per-entry" sounds like a byte-array for every entry. We tested something like that and the result is again higher GC pauses. In our

Re: OFFHEAP_TIERED and memory fragmentation

Hi Peter, Leaving defragmentation to Ignite is one of the reasons we are trying PageMemory approach. In Ignite 1.x we basically use OS memory allocator to place a value off-heap. Once the OS has given us a pointer, the memory cannot be moved around unless we free this region, thus the

Re: OFFHEAP_TIERED and memory fragmentation

Hi Val, fair enough :-) However, in the source-code it looks like "Flexible and precise per-cache memory limit" is supported already. At least each GridUnsafeMemory instance (in each GridCacheContext) receives the value from CacheConfiguration.getOffHeapMaxMemory). I haven't debugged it, but the

Re: OFFHEAP_TIERED and memory fragmentation

Hi Val, that means it depends a lot on the OS and if/how it handles virtual memory-pages. So why do you plan to change that? Thanks, Peter 2017-01-25 0:14 GMT+01:00 vkulichenko : > Random means anywhere in empty memory space. When you create an entry, a > new >

Re: OFFHEAP_TIERED and memory fragmentation

Random means anywhere in empty memory space. When you create an entry, a new block is create for this value only. If you remove an entry, memory is released. Once removed, it can be used by any application including Ignite. Basically, the actual location where memory is allocated is defined by OS.

OFFHEAP_TIERED and memory fragmentation

Hello Ignite-Community! I just found IGNITE-3477 and esp. "Continuous put/remove operations with OFFHEAP_TIERED mode lead to uncontrollable memory fragmentation" How does Ignite handle OFFHEAP_TIERED caches currently (= how does it store the data)? I just tried to create some tests to get an