[
https://issues.apache.org/jira/browse/IMPALA-10371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17290600#comment-17290600
]
ASF subversion and git services commented on IMPALA-10371:
----------------------------------------------------------
Commit 8059af8cee3563b42ffe04179add49c15c8067a8 in impala's branch
refs/heads/master from Daniel Becker
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=8059af8 ]
IMPALA-10371: test_java_udfs crash impalad if result spooling is enabled
IMPALA-7658 introduced proper codegen for HiveUdfCall. Because of a bug
in LLVM (see https://bugs.llvm.org/show_bug.cgi?id=21431), codegen code
could not use JniUtil::GetJNIEnv directly as it involves thread-local
variables, which LLVM JIT does not (yet) support. The original solution
to get around this problem was to cache the JNIEnv pointer in the
FunctionContext but it turned out that this leads to a crash if result
spooling is enabled because multiple threads can end up using the same
JNIEnv object.
This commit fixes the problem by using a different solution: instead of
caching the JNIEnv pointer we use a wrapper function
(HiveUdfCall:GetJniEnvNotInlined) that prevents JniUtil::GetJNIEnv from
being inlined in codegen code, thereby ensuring that the handling of the
thread-local variable is compiled by GCC.
Testing:
Manually verified that TestUdfExecution::test_java_udfs passes with
SPOOL_QUERY_RESULTS enabled.
Change-Id: Ie3aadb8ccc0f1f9b7b87a5744c22a0555b325ee6
Reviewed-on: http://gerrit.cloudera.org:8080/17106
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> test_java_udfs crash impalad if result spooling is enabled
> ----------------------------------------------------------
>
> Key: IMPALA-10371
> URL: https://issues.apache.org/jira/browse/IMPALA-10371
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 3.4.0
> Reporter: Riza Suminto
> Assignee: Daniel Becker
> Priority: Blocker
> Attachments: 46a19881-resolved.txt, hs_err_pid12878.log
>
>
> The following test query from TestUdfExecution::test_java_udfs crash impalad
> when result spooling is enabled.
> {code:java}
> select throws_exception() from functional.alltypestiny{code}
> The following is a truncated JVM crash log related to the crash
> {code:java}
> --------------- T H R E A D ---------------Current thread
> (0x000000000fb4c000): JavaThread "Thread-700" [_thread_in_native, id=30853,
> stack(0x00007f79715ff000,0x00007f7971dff000)]Stack:
> [0x00007f79715ff000,0x00007f7971dff000], sp=0x00007f7971dfa280, free
> space=8172k
> Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native
> code)
> V [libjvm.so+0xb6b032]
> V [libjvm.so+0x4f14bd]
> V [libjvm.so+0x80fa8f]
> V [libjvm.so+0x7e0991]
> V [libjvm.so+0x69fa10]
> j
> org.apache.impala.TestUdfException.evaluate()Lorg/apache/hadoop/io/BooleanWritable;+9
> v ~StubRoutines::call_stub
> V [libjvm.so+0x6af9ba]
> V [libjvm.so+0xa1def8]
> V [libjvm.so+0xa1f8d5]
> V [libjvm.so+0x7610f8] JVM_InvokeMethod+0x128
> J 2286
> sun.reflect.NativeMethodAccessorImpl.invoke0(Ljava/lang/reflect/Method;Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
> (0 bytes) @ 0x00007f7acb553ced [0x00007f7acb553c00+0xed]
> J 6921 C2
> sun.reflect.NativeMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object;
> (104 bytes) @ 0x00007f7acbd1de38 [0x00007f7acbd1ddc0+0x78]
> J 3645 C2 org.apache.impala.hive.executor.UdfExecutor.evaluate()V (396 bytes)
> @ 0x00007f7acaf6e894 [0x00007f7acaf6e640+0x254]
> v ~StubRoutines::call_stub
> V [libjvm.so+0x6af9ba]
> V [libjvm.so+0x72c046]
> V [libjvm.so+0x730523]
> C 0x00007f7ab4c5d0d0
> C [impalad+0x26a2648]
> impala::ScalarExprEvaluator::GetValue(impala::ScalarExpr const&,
> impala::TupleRow const*)+0x7a
> C [impalad+0x26a25cb]
> impala::ScalarExprEvaluator::GetValue(impala::TupleRow const*)+0x2b
> C [impalad+0x21f4f78]
> impala::AsciiQueryResultSet::AddRows(std::vector<impala::ScalarExprEvaluator*,
> std::allocator<impala::ScalarExprEvaluator*> > const&, impala::RowBatch*,
> int, int)+0x4c2
> C [impalad+0x25c5862]
> impala::BufferedPlanRootSink::GetNext(impala::RuntimeState*,
> impala::QueryResultSet*, int, bool*, long)+0x70c
> C [impalad+0x296cf17] impala::Coordinator::GetNext(impala::QueryResultSet*,
> int, bool*, long)+0x557
> C [impalad+0x219f5fe] impala::ClientRequestState::FetchRowsInternal(int,
> impala::QueryResultSet*, long)+0x6b2
> C [impalad+0x219d98e] impala::ClientRequestState::FetchRows(int,
> impala::QueryResultSet*, long)+0x46
> C [impalad+0x21c1d29]
> impala::ImpalaServer::FetchInternal(impala::TUniqueId, bool, int,
> beeswax::Results*)+0x717
> C [impalad+0x21bbde9] impala::ImpalaServer::fetch(beeswax::Results&,
> beeswax::QueryHandle const&, bool, int)+0x577
> {code}
> If result spooling is enabled, BufferedPlanRootSink will be used and
> ScalarExprEvaluation will be called in BufferedPlanRootSink::GetNext, leading
> to this crash.
> Without result spooling, BlockingPlanRootSink will be used and
> ScalarExprEvaluation is called in BlockingPlanRootSink::Send. No crash happen
> when result spooling is disabled.
> Attached is the full JVM crash log and resolved minidump.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
