Re: [PROPOSAL] - Java OffHeap Memory Pool
2013/4/18 Olivier Lamy ol...@apache.org Hi, Could that be part of directmemory as sub project ? +1 or either see if there's interest in merging directly into DM (probably requiring an IP clearance). Tommaso 2013/4/17 serkan özal serkanoza...@hotmail.com: Project Name: Jillegal 1. Abstract: GC is one of the time taken operations in Java. GC run anytime, marks, swaps and compacts objects at memory. If there are so many live objects, managing them by GC leads to overhead. If objects can be allocated outside of GC, there will be no overhead for the application. The session will go through the new method of creating and using object off-heap with no additional serialization/deserialization or any other overheads. 2. Proposal: For off-heap memory, I propose a solution that objects are allocated and initialized at off-heap instead of heap. Not only object attributes are allocated at off-heap, but also object header and metadata are allocated at off-heap. So while a reference to this object at off-heap is being interpreted, JVM knows which class this object references to. You can get your off-heap object from an object pool instead of with new operator. This object pool allocates a fixed size memory region from off-heap and create empty objects with given class on there. These empty objects can be created as lazy or eager. Another advantage off this technique is that objects in pool are layout in memory as sequential. While accessing an object, its neighbour objects are also fetched to CPU cache, so CPU cache hit rate for sequential object accesses are increased. On the other hand, freeing unused objects is responsibility of developer by calling free method of object pool which means there is no dead object detection mechanism like GC. Therefore, getting all objects from off-heap for whole application is dangerous and not recommended. Because, this will cause memory leaks. In addition, this technique can be combined with Java Instrumentation API. With @FromOffHeap annotation, developer can sign classes these must be allocated from off-heap instead of heap. With Java Instrumentation API, all new operators for signed classes with @FromOffHeap annotation can ben transformed to code that allocates objects of signed class from off-heap via object pool. So developer doesn't change all new keywords for getting from object pool in code. Instread of this, just sign class with @FromOffHeap annotation and all new keywords transformed for getting from object pool at class load time. This technique was used at a real time CDR processing system for Turk Telekom. There were billions of objects were used. Managing these objects by GC caused to performance overhead. For some most used classes, we allocated these objects from off-heap instead of new keyword. After some processings on them (takes 4-5 hours), we release these allocated memory regions to operating system by freeing them. Allocating objects from off-heap pool helps us to gain significant execution time performance. 3. Rationale: In general, off-heap pool implementations are implemented by serialization/deserialization to allocated off-heap memory region via ByteBuffer class. But this technique leads to extra execution overhead. Because while reading from an object, the target object must be created by deserializing all primitive fields eagerly or only required fields on demand and while writing to an object, the attribute has been set by application, must be deserialized to allocated off-heap memory region. In addition, objects itself is created at heap, so GC knows and tracks it. With my solution, all of these overheads are overcomed. 4. Initial Goals: * Allocating objects from off-heap and using them as normal on-heap Java object. * Allocating arrays for object types from off-heap and using them as normal on-heap Java object arrays. * Allocating arrays for primitive types from off-heap and using them as normal on-heap Java primitive type arrays. * Allocating strings from off-heap and using them as normal on-heap strings. * Implementing auto expandable off-heap pool that expands when its delegated off-heap pool implementation is full. * All features must be supported for 32 bit and 64 bit JVM. * All features must be supported for Sun HotSpot JVM, Oracle JRockit, IBM J9. 5. Currently Implemented Features: * Allocating objects from off-heap and using them as normal on-heap Java object * Allocating arrays for object types from off-heap and using them as normal on-heap Java object array * Allocating arrays for primitive types from off-heap and using them as normal on-heap Java primitive type arrays * Implementing auto expandable off-heap pool that expands when its delegated off-heap pool implementation is full. * All features are supported for 32 bit and 64 bit
Re: [PROPOSAL] - Java OffHeap Memory Pool
+1 of course these efforts are pretty much related and the IP clearance for contributed code is needed in any case. Merging with DM would avoid most part of the incubation phase On Thu, Apr 18, 2013 at 8:07 AM, Tommaso Teofili tommaso.teof...@gmail.comwrote: 2013/4/18 Olivier Lamy ol...@apache.org Hi, Could that be part of directmemory as sub project ? +1 or either see if there's interest in merging directly into DM (probably requiring an IP clearance). Tommaso 2013/4/17 serkan özal serkanoza...@hotmail.com: Project Name: Jillegal 1. Abstract: GC is one of the time taken operations in Java. GC run anytime, marks, swaps and compacts objects at memory. If there are so many live objects, managing them by GC leads to overhead. If objects can be allocated outside of GC, there will be no overhead for the application. The session will go through the new method of creating and using object off-heap with no additional serialization/deserialization or any other overheads. 2. Proposal: For off-heap memory, I propose a solution that objects are allocated and initialized at off-heap instead of heap. Not only object attributes are allocated at off-heap, but also object header and metadata are allocated at off-heap. So while a reference to this object at off-heap is being interpreted, JVM knows which class this object references to. You can get your off-heap object from an object pool instead of with new operator. This object pool allocates a fixed size memory region from off-heap and create empty objects with given class on there. These empty objects can be created as lazy or eager. Another advantage off this technique is that objects in pool are layout in memory as sequential. While accessing an object, its neighbour objects are also fetched to CPU cache, so CPU cache hit rate for sequential object accesses are increased. On the other hand, freeing unused objects is responsibility of developer by calling free method of object pool which means there is no dead object detection mechanism like GC. Therefore, getting all objects from off-heap for whole application is dangerous and not recommended. Because, this will cause memory leaks. In addition, this technique can be combined with Java Instrumentation API. With @FromOffHeap annotation, developer can sign classes these must be allocated from off-heap instead of heap. With Java Instrumentation API, all new operators for signed classes with @FromOffHeap annotation can ben transformed to code that allocates objects of signed class from off-heap via object pool. So developer doesn't change all new keywords for getting from object pool in code. Instread of this, just sign class with @FromOffHeap annotation and all new keywords transformed for getting from object pool at class load time. This technique was used at a real time CDR processing system for Turk Telekom. There were billions of objects were used. Managing these objects by GC caused to performance overhead. For some most used classes, we allocated these objects from off-heap instead of new keyword. After some processings on them (takes 4-5 hours), we release these allocated memory regions to operating system by freeing them. Allocating objects from off-heap pool helps us to gain significant execution time performance. 3. Rationale: In general, off-heap pool implementations are implemented by serialization/deserialization to allocated off-heap memory region via ByteBuffer class. But this technique leads to extra execution overhead. Because while reading from an object, the target object must be created by deserializing all primitive fields eagerly or only required fields on demand and while writing to an object, the attribute has been set by application, must be deserialized to allocated off-heap memory region. In addition, objects itself is created at heap, so GC knows and tracks it. With my solution, all of these overheads are overcomed. 4. Initial Goals: * Allocating objects from off-heap and using them as normal on-heap Java object. * Allocating arrays for object types from off-heap and using them as normal on-heap Java object arrays. * Allocating arrays for primitive types from off-heap and using them as normal on-heap Java primitive type arrays. * Allocating strings from off-heap and using them as normal on-heap strings. * Implementing auto expandable off-heap pool that expands when its delegated off-heap pool implementation is full. * All features must be supported for 32 bit and 64 bit JVM. * All features must be supported for Sun HotSpot JVM, Oracle JRockit, IBM J9. 5. Currently Implemented Features: * Allocating objects from off-heap and using them as normal on-heap Java object * Allocating arrays for object types from off-heap and
[PROPOSAL] - Java OffHeap Memory Pool
Project Name: Jillegal 1. Abstract: GC is one of the time taken operations in Java. GC run anytime, marks, swaps and compacts objects at memory. If there are so many live objects, managing them by GC leads to overhead. If objects can be allocated outside of GC, there will be no overhead for the application. The session will go through the new method of creating and using object off-heap with no additional serialization/deserialization or any other overheads. 2. Proposal: For off-heap memory, I propose a solution that objects are allocated and initialized at off-heap instead of heap. Not only object attributes are allocated at off-heap, but also object header and metadata are allocated at off-heap. So while a reference to this object at off-heap is being interpreted, JVM knows which class this object references to. You can get your off-heap object from an object pool instead of with new operator. This object pool allocates a fixed size memory region from off-heap and create empty objects with given class on there. These empty objects can be created as lazy or eager. Another advantage off this technique is that objects in pool are layout in memory as sequential. While accessing an object, its neighbour objects are also fetched to CPU cache, so CPU cache hit rate for sequential object accesses are increased. On the other hand, freeing unused objects is responsibility of developer by calling free method of object pool which means there is no dead object detection mechanism like GC. Therefore, getting all objects from off-heap for whole application is dangerous and not recommended. Because, this will cause memory leaks. In addition, this technique can be combined with Java Instrumentation API. With @FromOffHeap annotation, developer can sign classes these must be allocated from off-heap instead of heap. With Java Instrumentation API, all new operators for signed classes with @FromOffHeap annotation can ben transformed to code that allocates objects of signed class from off-heap via object pool. So developer doesn't change all new keywords for getting from object pool in code. Instread of this, just sign class with @FromOffHeap annotation and all new keywords transformed for getting from object pool at class load time. This technique was used at a real time CDR processing system for Turk Telekom. There were billions of objects were used. Managing these objects by GC caused to performance overhead. For some most used classes, we allocated these objects from off-heap instead of new keyword. After some processings on them (takes 4-5 hours), we release these allocated memory regions to operating system by freeing them. Allocating objects from off-heap pool helps us to gain significant execution time performance. 3. Rationale: In general, off-heap pool implementations are implemented by serialization/deserialization to allocated off-heap memory region via ByteBuffer class. But this technique leads to extra execution overhead. Because while reading from an object, the target object must be created by deserializing all primitive fields eagerly or only required fields on demand and while writing to an object, the attribute has been set by application, must be deserialized to allocated off-heap memory region. In addition, objects itself is created at heap, so GC knows and tracks it. With my solution, all of these overheads are overcomed. 4. Initial Goals: * Allocating objects from off-heap and using them as normal on-heap Java object. * Allocating arrays for object types from off-heap and using them as normal on-heap Java object arrays. * Allocating arrays for primitive types from off-heap and using them as normal on-heap Java primitive type arrays. * Allocating strings from off-heap and using them as normal on-heap strings. * Implementing auto expandable off-heap pool that expands when its delegated off-heap pool implementation is full. * All features must be supported for 32 bit and 64 bit JVM. * All features must be supported for Sun HotSpot JVM, Oracle JRockit, IBM J9. 5. Currently Implemented Features: * Allocating objects from off-heap and using them as normal on-heap Java object * Allocating arrays for object types from off-heap and using them as normal on-heap Java object array * Allocating arrays for primitive types from off-heap and using them as normal on-heap Java primitive type arrays * Implementing auto expandable off-heap pool that expands when its delegated off-heap pool implementation is full. * All features are supported for 32 bit and 64 bit JVM. * All features are supported for Sun HotSpot JVM, Oracle JRockit (IBM J9 support will be added). 6. Roadmap * Automatic detection and binding for complex off-heap objects will be implemented. * All allocated objects with new operator will be automatically allocated
Re: [PROPOSAL] - Java OffHeap Memory Pool
Is code for this available for review anywhere? -- Brane On 16.04.2013 19:46, serkan özal wrote: Project Name: Jillegal 1. Abstract: GC is one of the time taken operations in Java. GC run anytime, marks, swaps and compacts objects at memory. If there are so many live objects, managing them by GC leads to overhead. If objects can be allocated outside of GC, there will be no overhead for the application. The session will go through the new method of creating and using object off-heap with no additional serialization/deserialization or any other overheads. 2. Proposal: For off-heap memory, I propose a solution that objects are allocated and initialized at off-heap instead of heap. Not only object attributes are allocated at off-heap, but also object header and metadata are allocated at off-heap. So while a reference to this object at off-heap is being interpreted, JVM knows which class this object references to. You can get your off-heap object from an object pool instead of with new operator. This object pool allocates a fixed size memory region from off-heap and create empty objects with given class on there. These empty objects can be created as lazy or eager. Another advantage off this technique is that objects in pool are layout in memory as sequential. While accessing an object, its neighbour objects are also fetched to CPU cache, so CPU cache hit rate for sequential object accesses are increased. On the other hand, freeing unused objects is responsibility of developer by calling free method of object pool which means there is no dead object detection mechanism like GC. Therefore, getting all objects from off-heap for whole application is dangerous and not recommended. Because, this will cause memory leaks. In addition, this technique can be combined with Java Instrumentation API. With @FromOffHeap annotation, developer can sign classes these must be allocated from off-heap instead of heap. With Java Instrumentation API, all new operators for signed classes with @FromOffHeap annotation can ben transformed to code that allocates objects of signed class from off-heap via object pool. So developer doesn't change all new keywords for getting from object pool in code. Instread of this, just sign class with @FromOffHeap annotation and all new keywords transformed for getting from object pool at class load time. This technique was used at a real time CDR processing system for Turk Telekom. There were billions of objects were used. Managing these objects by GC caused to performance overhead. For some most used classes, we allocated these objects from off-heap instead of new keyword. After some processings on them (takes 4-5 hours), we release these allocated memory regions to operating system by freeing them. Allocating objects from off-heap pool helps us to gain significant execution time performance. 3. Rationale: In general, off-heap pool implementations are implemented by serialization/deserialization to allocated off-heap memory region via ByteBuffer class. But this technique leads to extra execution overhead. Because while reading from an object, the target object must be created by deserializing all primitive fields eagerly or only required fields on demand and while writing to an object, the attribute has been set by application, must be deserialized to allocated off-heap memory region. In addition, objects itself is created at heap, so GC knows and tracks it. With my solution, all of these overheads are overcomed. 4. Initial Goals: * Allocating objects from off-heap and using them as normal on-heap Java object. * Allocating arrays for object types from off-heap and using them as normal on-heap Java object arrays. * Allocating arrays for primitive types from off-heap and using them as normal on-heap Java primitive type arrays. * Allocating strings from off-heap and using them as normal on-heap strings. * Implementing auto expandable off-heap pool that expands when its delegated off-heap pool implementation is full. * All features must be supported for 32 bit and 64 bit JVM. * All features must be supported for Sun HotSpot JVM, Oracle JRockit, IBM J9. 5. Currently Implemented Features: * Allocating objects from off-heap and using them as normal on-heap Java object * Allocating arrays for object types from off-heap and using them as normal on-heap Java object array * Allocating arrays for primitive types from off-heap and using them as normal on-heap Java primitive type arrays * Implementing auto expandable off-heap pool that expands when its delegated off-heap pool implementation is full. * All features are supported for 32 bit and 64 bit JVM. * All features are supported for Sun HotSpot JVM, Oracle JRockit (IBM J9 support will be added). 6. Roadmap
Re: [PROPOSAL] - Java OffHeap Memory Pool
Hi, Could that be part of directmemory as sub project ? 2013/4/17 serkan özal serkanoza...@hotmail.com: Project Name: Jillegal 1. Abstract: GC is one of the time taken operations in Java. GC run anytime, marks, swaps and compacts objects at memory. If there are so many live objects, managing them by GC leads to overhead. If objects can be allocated outside of GC, there will be no overhead for the application. The session will go through the new method of creating and using object off-heap with no additional serialization/deserialization or any other overheads. 2. Proposal: For off-heap memory, I propose a solution that objects are allocated and initialized at off-heap instead of heap. Not only object attributes are allocated at off-heap, but also object header and metadata are allocated at off-heap. So while a reference to this object at off-heap is being interpreted, JVM knows which class this object references to. You can get your off-heap object from an object pool instead of with new operator. This object pool allocates a fixed size memory region from off-heap and create empty objects with given class on there. These empty objects can be created as lazy or eager. Another advantage off this technique is that objects in pool are layout in memory as sequential. While accessing an object, its neighbour objects are also fetched to CPU cache, so CPU cache hit rate for sequential object accesses are increased. On the other hand, freeing unused objects is responsibility of developer by calling free method of object pool which means there is no dead object detection mechanism like GC. Therefore, getting all objects from off-heap for whole application is dangerous and not recommended. Because, this will cause memory leaks. In addition, this technique can be combined with Java Instrumentation API. With @FromOffHeap annotation, developer can sign classes these must be allocated from off-heap instead of heap. With Java Instrumentation API, all new operators for signed classes with @FromOffHeap annotation can ben transformed to code that allocates objects of signed class from off-heap via object pool. So developer doesn't change all new keywords for getting from object pool in code. Instread of this, just sign class with @FromOffHeap annotation and all new keywords transformed for getting from object pool at class load time. This technique was used at a real time CDR processing system for Turk Telekom. There were billions of objects were used. Managing these objects by GC caused to performance overhead. For some most used classes, we allocated these objects from off-heap instead of new keyword. After some processings on them (takes 4-5 hours), we release these allocated memory regions to operating system by freeing them. Allocating objects from off-heap pool helps us to gain significant execution time performance. 3. Rationale: In general, off-heap pool implementations are implemented by serialization/deserialization to allocated off-heap memory region via ByteBuffer class. But this technique leads to extra execution overhead. Because while reading from an object, the target object must be created by deserializing all primitive fields eagerly or only required fields on demand and while writing to an object, the attribute has been set by application, must be deserialized to allocated off-heap memory region. In addition, objects itself is created at heap, so GC knows and tracks it. With my solution, all of these overheads are overcomed. 4. Initial Goals: * Allocating objects from off-heap and using them as normal on-heap Java object. * Allocating arrays for object types from off-heap and using them as normal on-heap Java object arrays. * Allocating arrays for primitive types from off-heap and using them as normal on-heap Java primitive type arrays. * Allocating strings from off-heap and using them as normal on-heap strings. * Implementing auto expandable off-heap pool that expands when its delegated off-heap pool implementation is full. * All features must be supported for 32 bit and 64 bit JVM. * All features must be supported for Sun HotSpot JVM, Oracle JRockit, IBM J9. 5. Currently Implemented Features: * Allocating objects from off-heap and using them as normal on-heap Java object * Allocating arrays for object types from off-heap and using them as normal on-heap Java object array * Allocating arrays for primitive types from off-heap and using them as normal on-heap Java primitive type arrays * Implementing auto expandable off-heap pool that expands when its delegated off-heap pool implementation is full. * All features are supported for 32 bit and 64 bit JVM. * All features are supported for Sun HotSpot JVM, Oracle JRockit (IBM J9 support will be added). 6. Roadmap
