[
https://issues.apache.org/jira/browse/IMPALA-2029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16836934#comment-16836934
]
ASF subversion and git services commented on IMPALA-2029:
---------------------------------------------------------
Commit 1e49b6a6b4ecb756b6d11c73cc479508c86e7028 in impala's branch
refs/heads/master from Todd Lipcon
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=1e49b6a ]
IMPALA-2029. Implement our own getJNIEnv equivalent
The libhdfs getJNIEnv function was made non-exported in Hadoop 2. For a
while in CDH we were hacking around this with a vendor-specific patch
that re-exported it. However, that was always a bit annoying to maintain
our own patch each time we rebased to new versions, etc.
Earlier attempts to solve this issue turned up strange bugs around
coordinating whether we or libhdfs were responsible for attaching and
detaching to the JVM/JNI environment. So, this patch takes a new
approach: rather than directly creating/attaching to the JVM, we just
look for an existing attached environment. If there isn't one, we call
some simple libhdfs function which forces it to attach the current
thread, and then try again.
Performance is maintained (or maybe improved) by adding a thread-local
cache of the attached JVM, with an inlined fast-path.
I tested this with a CDP build of Hadoop which doesn't have the
getJNIEnv workaround. Prior to this fix, I wasn't able to run Java tests
against that build because it would fail to link getJNIEnv() at runtime.
Now, they pass.
Change-Id: I766bcfd70addb00e9fd8a860e89c2a1c5d4c71d5
Reviewed-on: http://gerrit.cloudera.org:8080/13275
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Impala in CDH 5.2.0 fails to compile with hadoop 2.7
> ----------------------------------------------------
>
> Key: IMPALA-2029
> URL: https://issues.apache.org/jira/browse/IMPALA-2029
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 2.2
> Environment: Red Hat 6.4 and gcc 4.3.4
> Reporter: Varun Saxena
> Assignee: Todd Lipcon
> Priority: Major
>
> Compilation fails with below error message :
> ../../build/release/exec/libExec.a(hbase-table-scanner.cc.o): In function
> `impala::HBaseTableScanner::Init()':
> /usr1/code/Impala/code/current/impala/be/src/exec/hbase-table-scanner.cc:113:
> undefined reference to `getJNIEnv'
> ../../build/release/exprs/libExprs.a(hive-udf-call.cc.o):/usr1/code/Impala/code/current/impala/be/src/exprs/hive-udf-call.cc:227:
> more undefined references to `getJNIEnv' follow
> collect2: ld returned 1 exit status
> make[3]: *** [be/build/release/service/impalad] Error 1
> make[2]: *** [be/src/service/CMakeFiles/impalad.dir/all] Error 2
> make[1]: *** [be/src/service/CMakeFiles/impalad.dir/rule] Error 2
> make: *** [impalad] Error 2
> Compiler Impala Failed, exit
> This issue is coming because libhdfs in hadoop 2.7.0 has made visibility of
> getJNIEnv() as hidden.
> But shouldn't Impala create its own JNIEnv ?
> These HBase related files seem to have no direct connection with libhdfs
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
