Hello Joe McDonnell,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/13275
to review the following change.
Change subject: IMPALA-2029. Implement our own getJNIEnv equivalent
......................................................................
IMPALA-2029. Implement our own getJNIEnv equivalent
The libhdfs getJNIEnv function was made non-exported in Hadoop 2. For a
while in CDH we were hacking around this with a vendor-specific patch
that re-exported it. However, that was always a bit annoying to maintain
our own patch each time we rebased to new versions, etc.
Earlier attempts to solve this issue turned up strange bugs around
coordinating whether we or libhdfs were responsible for attaching and
detaching to the JVM/JNI environment. So, this patch takes a new
approach: rather than directly creating/attaching to the JVM, we just
look for an existing attached environment. If there isn't one, we call
some simple libhdfs function which forces it to attach the current
thread, and then try again.
Performance is maintained (or maybe improved) by adding a thread-local
cache of the attached JVM, with an inlined fast-path.
I tested this with a CDP build of Hadoop which doesn't have the
getJNIEnv workaround. Prior to this fix, I wasn't able to run Java tests
against that build because it would fail to link getJNIEnv() at runtime.
Now, they pass.
Change-Id: I766bcfd70addb00e9fd8a860e89c2a1c5d4c71d5
---
M be/src/catalog/catalog-util.cc
M be/src/catalog/catalog.cc
M be/src/exec/external-data-source-executor.cc
M be/src/exec/hbase-scan-node.cc
M be/src/exec/hbase-table-scanner.cc
M be/src/exec/hbase-table-writer.cc
M be/src/exprs/hive-udf-call.cc
M be/src/runtime/hbase-table-factory.cc
M be/src/runtime/hbase-table.cc
M be/src/scheduling/request-pool-service.cc
M be/src/service/fe-support.cc
M be/src/service/frontend.cc
M be/src/util/jni-util.cc
M be/src/util/jni-util.h
M be/src/util/logging-support.cc
M be/src/util/zip-util.cc
16 files changed, 91 insertions(+), 47 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/75/13275/1
--
To view, visit http://gerrit.cloudera.org:8080/13275
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I766bcfd70addb00e9fd8a860e89c2a1c5d4c71d5
Gerrit-Change-Number: 13275
Gerrit-PatchSet: 1
Gerrit-Owner: Todd Lipcon <t...@apache.org>
Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com>
