[
https://issues.apache.org/jira/browse/HADOOP-9601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14101342#comment-14101342
]
Todd Lipcon commented on HADOOP-9601:
-------------------------------------
Hey Gopal,
As far as I know, no existing production garbage collector actually fragments
arrays. Instead, if you try to allocate an array larger than the available
contiguous free heap space, it will force a compacting GC inline with the
allocation.
This is confirmed by the JDK7 source:
{code}
JNI_ENTRY(void*, jni_GetPrimitiveArrayCritical(JNIEnv *env, jarray array,
jboolean *isCopy))
JNIWrapper("GetPrimitiveArrayCritical");
DTRACE_PROBE3(hotspot_jni, GetPrimitiveArrayCritical__entry, env, array,
isCopy);
GC_locker::lock_critical(thread);
if (isCopy != NULL) {
*isCopy = JNI_FALSE;
}
oop a = JNIHandles::resolve_non_null(array);
assert(a->is_array(), "just checking");
BasicType type;
if (a->is_objArray()) {
type = T_OBJECT;
} else {
type = typeArrayKlass::cast(a->klass())->element_type();
}
void* ret = arrayOop(a)->base(type);
DTRACE_PROBE1(hotspot_jni, GetPrimitiveArrayCritical__return, ret);
return ret;
JNI_END
{code}
Note that {{isCopy}} is always set to JNI_FALSE. I checked the jdk9 source as
well
(http://hg.openjdk.java.net/jdk9/hs/hotspot/file/7a0fe19ac034/src/share/vm/prims/jni.cpp)
and found the same.
So, not sure why GetPrimitiveArrayCritical would ever slow down. The only
reason it would ever go slow is if the {{GC_locker::lock_critical}} call blocks
- this is the case if there is a pending safepoint. So, maybe in your
application there were other threads which were blocking safepoints for a long
time, and GetPrimitiveArrayCritical was feeling the effects?
> Support native CRC on byte arrays
> ---------------------------------
>
> Key: HADOOP-9601
> URL: https://issues.apache.org/jira/browse/HADOOP-9601
> Project: Hadoop Common
> Issue Type: Improvement
> Components: performance, util
> Affects Versions: 3.0.0
> Reporter: Todd Lipcon
> Assignee: Gopal V
> Labels: perfomance
> Attachments: HADOOP-9601-WIP-01.patch, HADOOP-9601-WIP-02.patch,
> HADOOP-9601-bench.patch, HADOOP-9601-rebase+benchmark.patch,
> HADOOP-9601-trunk-rebase-2.patch, HADOOP-9601-trunk-rebase.patch
>
>
> When we first implemented the Native CRC code, we only did so for direct byte
> buffers, because these correspond directly to native heap memory and thus
> make it easy to access via JNI. We'd generally assumed that accessing byte[]
> arrays from JNI was not efficient enough, but now that I know more about JNI
> I don't think that's true -- we just need to make sure that the critical
> sections where we lock the buffers are short.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
