Project Name: Jillegal
1. Abstract:
GC is one of the time taken operations in Java. GC run anytime, marks, swaps
and compacts objects at memory. If there are so many live objects, managing
them by GC leads to overhead. If objects can be allocated outside of GC, there
will be no overhead for the application. The session will go through the new
method of creating and using object off-heap with no additional
serialization/deserialization or any other overheads.
2. Proposal:
For off-heap memory, I propose a solution that objects are allocated and
initialized at off-heap instead of heap. Not only object attributes are
allocated at off-heap, but also object header and metadata are allocated at
off-heap. So while a reference to this object at off-heap is being interpreted,
JVM knows which class this object references to. You can get your off-heap
object from an object pool instead of with "new" operator. This object pool
allocates a fixed size memory region from off-heap and create empty objects
with given class on there. These empty objects can be created as lazy or eager.
Another advantage off this technique is that objects in pool are layout in
memory as sequential. While accessing an object, its neighbour objects are also
fetched to CPU cache, so CPU cache hit rate for sequential object accesses are
increased. On the other hand, freeing unused objects is responsibility of
developer by calling "free" method of object pool which means there is no dead
object detection mechanism like GC. Therefore, getting all objects from
off-heap for whole application is dangerous and not recommended. Because, this
will cause memory leaks. In addition, this technique can be combined with "Java
Instrumentation API". With "@FromOffHeap" annotation, developer can sign
classes these must be allocated from off-heap instead of heap. With "Java
Instrumentation API", all "new" operators for signed classes with
"@FromOffHeap" annotation can ben transformed to code that allocates objects of
signed class from off-heap via object pool. So developer doesn't change all
"new" keywords for getting from object pool in code. Instread of this, just
sign class with "@FromOffHeap" annotation and all "new" keywords transformed
for getting from object pool at class load time. This technique was used at a
real time CDR processing system for Turk Telekom. There were billions of
objects were used. Managing these objects by GC caused to performance overhead.
For some most used classes, we allocated these objects from off-heap instead of
"new" keyword. After some processings on them (takes 4-5 hours), we release
these allocated memory regions to operating system by freeing them. Allocating
objects from off-heap pool helps us to gain significant execution time
performance.
3. Rationale:
In general, off-heap pool implementations are implemented by
serialization/deserialization to allocated off-heap memory region via
"ByteBuffer" class. But this technique leads to extra execution overhead.
Because while reading from an object, the target object must be created by
deserializing all primitive fields eagerly or only required fields on demand
and while writing to an object, the attribute has been set by application, must
be deserialized to allocated off-heap memory region. In addition, objects
itself is created at heap, so GC knows and tracks it. With my solution, all of
these overheads are overcomed.
4. Initial Goals:
* Allocating objects from off-heap and using them as normal on-heap
Java object. * Allocating arrays for object types from off-heap and
using them as normal on-heap Java object arrays. * Allocating arrays
for primitive types from off-heap and using them as normal on-heap Java
primitive type arrays. * Allocating strings from off-heap and using
them as normal on-heap strings. * Implementing auto expandable off-heap
pool that expands when its delegated off-heap pool implementation is full.
* All features must be supported for 32 bit and 64 bit JVM. * All
features must be supported for Sun HotSpot JVM, Oracle JRockit, IBM J9.
5. Currently Implemented Features:
* Allocating objects from off-heap and using them as normal on-heap
Java object * Allocating arrays for object types from off-heap and
using them as normal on-heap Java object array * Allocating arrays for
primitive types from off-heap and using them as normal on-heap Java primitive
type arrays * Implementing auto expandable off-heap pool that expands
when its delegated off-heap pool implementation is full. * All features
are supported for 32 bit and 64 bit JVM. * All features are supported
for Sun HotSpot JVM, Oracle JRockit (IBM J9 support will be added).
6. Roadmap
* Automatic detection and binding for complex off-heap objects will be
implemented. * All allocated objects with new operator will be
automatically allocated from off-heap without any changing in your
code. Just annotate class whose instances must be allocated from off-heap
with "@FromOffHeap" annotation. This feature can be implemented with Java
Instrumentation API by transforming all "new" byte codes for
specified classes to a code block like "OffHeapPool.getObject(Class clazz)"
