Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13275 )
Change subject: IMPALA-2029. Implement our own getJNIEnv equivalent ...................................................................... IMPALA-2029. Implement our own getJNIEnv equivalent The libhdfs getJNIEnv function was made non-exported in Hadoop 2. For a while in CDH we were hacking around this with a vendor-specific patch that re-exported it. However, that was always a bit annoying to maintain our own patch each time we rebased to new versions, etc. Earlier attempts to solve this issue turned up strange bugs around coordinating whether we or libhdfs were responsible for attaching and detaching to the JVM/JNI environment. So, this patch takes a new approach: rather than directly creating/attaching to the JVM, we just look for an existing attached environment. If there isn't one, we call some simple libhdfs function which forces it to attach the current thread, and then try again. Performance is maintained (or maybe improved) by adding a thread-local cache of the attached JVM, with an inlined fast-path. I tested this with a CDP build of Hadoop which doesn't have the getJNIEnv workaround. Prior to this fix, I wasn't able to run Java tests against that build because it would fail to link getJNIEnv() at runtime. Now, they pass. Change-Id: I766bcfd70addb00e9fd8a860e89c2a1c5d4c71d5 Reviewed-on: http://gerrit.cloudera.org:8080/13275 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/catalog/catalog-util.cc M be/src/catalog/catalog.cc M be/src/exec/external-data-source-executor.cc M be/src/exec/hbase-scan-node.cc M be/src/exec/hbase-table-scanner.cc M be/src/exec/hbase-table-writer.cc M be/src/exprs/hive-udf-call.cc M be/src/runtime/hbase-table-factory.cc M be/src/runtime/hbase-table.cc M be/src/scheduling/request-pool-service.cc M be/src/service/fe-support.cc M be/src/service/frontend.cc M be/src/util/jni-util.cc M be/src/util/jni-util.h M be/src/util/logging-support.cc M be/src/util/zip-util.cc 16 files changed, 91 insertions(+), 47 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/13275 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I766bcfd70addb00e9fd8a860e89c2a1c5d4c71d5 Gerrit-Change-Number: 13275 Gerrit-PatchSet: 4 Gerrit-Owner: Todd Lipcon <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Todd Lipcon <[email protected]>
