Peter, A page is a minimal entity that is allowed to be moved on OS level. If I have a chunk of memory allocated in the middle of a page, OS won't be able to move it during defragmentation (at least this is how things worked last time I checked). In the new approach, we are able to move this chunk within the page, this is the key difference compared to the old off-heap approach.
Note, that fine-grained per-entry locking does not imply a byte array per entry. We will still have our old GridCacheMapEntry object, but it will be created only during a cache read or update, and will be GC-ed immediately after cache operation completes. A value itself, on the other hand, will be stored off-heap. Your measurements confirm that pure off-heap cache outperforms both options for on-heap caches in terms of GC, so I do not see any problems here. Thanks, Alexey 2017-01-26 23:23 GMT+03:00 Peter Schmitt <[email protected]>: > Hi Alexey, > > a modern OS won't give you a physical address. Instead you get virtual > memory which allows the OS to support features like defragmentation. > "fine-grained per-entry" sounds like a byte-array for every entry. > We tested something like that and the result is again higher GC pauses. > In our case a FGC with off-heap caches takes e.g. up to 1s. > With the same data and using on-heap caches it takes up to 25s. > With a byte-array per entry we get about 4s for a FGC. > With some optimizations you might get down a bit, however, in view of the > FGC time I fear that such an approach causes a significant impact for > applications which need to keep huge caches. > > Kind regards, > Peter > > > > 2017-01-26 15:57 GMT+01:00 Alexey Goncharuk <[email protected]>: > >> Hi Peter, >> >> Leaving defragmentation to Ignite is one of the reasons we are trying >> PageMemory approach. In Ignite 1.x we basically use OS memory allocator to >> place a value off-heap. Once the OS has given us a pointer, the memory >> cannot be moved around unless we free this region, thus the fragmentation. >> On the other hand, with PageMemory approach we can define the page layout >> ourselves and, given fine-grained per-entry locking, move values around in >> memory. This makes memory management more complicated and usually defines >> some restrictions on the data structures, but we expect this will give us >> more control over the data. >> >> For example, one of the tricks we can do in the future is cluster-wide >> cache snapshots: we simply dump the continuous region of memory on disk and >> get a node-local caches snapshot. This is impossible to do using current >> off-heap approach. >> >> --AG >> >> 2017-01-25 12:32 GMT+03:00 Peter Schmitt <[email protected]>: >> >>> Hi Val, >>> >>> fair enough :-) >>> However, in the source-code it looks like "Flexible and precise >>> per-cache memory limit" is supported already. >>> At least each GridUnsafeMemory instance (in each GridCacheContext) >>> receives the value from CacheConfiguration.getOffHeapMaxMemory). >>> I haven't debugged it, but the instances are created per cache. >>> >>> Furthermore, I'm not sure about the memory-page concept, because you >>> would get pages on top of pages (used by a modern OS). >>> Sounds like even more fragmentation and in this case Ignite would need >>> to do the defragmentation. >>> >>> Kind regards, >>> Peter >>> >>> >>> >>> 2017-01-25 1:56 GMT+01:00 vkulichenko <[email protected]>: >>> >>>> I don't think I will be able to explain better than it's done in the >>>> ticket >>>> :) Current approach is naive and not always effective for multiple >>>> reasons. >>>> >>>> -Val >>>> >>>> >>>> >>>> -- >>>> View this message in context: http://apache-ignite-users.705 >>>> 18.x6.nabble.com/OFFHEAP-TIERED-and-memory-fragmentation-tp1 >>>> 0218p10235.html >>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com. >>>> >>> >>> >> >
