[
https://issues.apache.org/jira/browse/IMPALA-13770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17930176#comment-17930176
]
ASF subversion and git services commented on IMPALA-13770:
----------------------------------------------------------
Commit bab5308cde35861ad95d2947ab1cf8ad5991aeed in impala's branch
refs/heads/branch-4.5.0 from Daniel Becker
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=bab5308cd ]
IMPALA-13770: Updating Iceberg tables with UDFs crashes Impala
When using a native UDF in the target value of an UPDATE statement or in
a filter predicate or target value of a MERGE statement, Impala crashes
with the following DCHECK:
be/src/exprs/expr.cc:47 47 DCHECK(cache_entry_ == nullptr);
This DCHECK is in the destructor of Expr, and it fires because Close()
has not been called for the expression. In the UPDATE case this is
caused by MultiTableSinkConfig: it creates child DataSinkConfig objects
but does not call Close() on them, and consequently these child sink
configs do not call Close() on their output expressions.
In the MERGE case it is because various expressions are not closed in
IcebergMergeCase and IcebergMergeNode.
This patch fixes the issue by overriding Close() in MultiTableSinkConfig,
calling Close() on the child sinks as well as closing the expressions in
IcebergMergeCase and IcebergMergeNode.
Testing:
- Added EE regression tests for the UPDATE and MERGE cases in
iceberg-update-basic.test and iceberg-merge.test
Change-Id: Id86638c8d6d86062c68cc9d708ec9c7b0a4e95eb
Reviewed-on: http://gerrit.cloudera.org:8080/22508
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Updating Iceberg tables with UDFs crashes Impala
> ------------------------------------------------
>
> Key: IMPALA-13770
> URL: https://issues.apache.org/jira/browse/IMPALA-13770
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Reporter: Noemi Pap-Takacs
> Assignee: Daniel Becker
> Priority: Major
> Labels: impala-iceberg
>
> When updating an Iceberg table setting the column to a value defined by a
> UDF, Impala crashes during execution on the Backend with the following stack
> trace:
> {noformat}
> #7 0x0000000005c89709 in google::LogMessageFatal::~LogMessageFatal() ()
> #8 0x00000000041caa55 in impala::Expr::~Expr (this=0x148bfc00,
> __in_chrg=<optimized out>) at /home/user/Impala/be/src/exprs/expr.cc:47
> #9 0x00000000041cf4c6 in impala::ScalarExpr::~ScalarExpr (this=0x148bfc00,
> __in_chrg=<optimized out>) at /home/user/Impala/be/src/exprs/scalar-expr.h:140
> #10 0x00000000041e1bd0 in impala::ScalarFnCall::~ScalarFnCall
> (this=0x148bfc00, __in_chrg=<optimized out>) at
> /home/user/Impala/be/src/exprs/scalar-fn-call.h:68
> #11 0x00000000041e1bec in impala::ScalarFnCall::~ScalarFnCall
> (this=0x148bfc00, __in_chrg=<optimized out>) at
> /home/user/Impala/be/src/exprs/scalar-fn-call.h:68
> #12 0x00000000041e241e in
> impala::ObjectPool::Add<impala::ScalarFnCall>(impala::ScalarFnCall*)::{lambda(void*)#1}::operator()(void*)
> const (this=0x0, obj=0x148bfc00) at
> /home/user/Impala/be/src/common/object-pool.h:41
> #13 0x00000000041e243e in
> impala::ObjectPool::Add<impala::ScalarFnCall>(impala::ScalarFnCall*)::{lambda(void*)#1}::_FUN(void*)
> () at /home/user/Impala/be/src/common/object-pool.h:41
> #14 0x00000000023fd213 in impala::ObjectPool::Clear (this=0x153289a0) at
> /home/user/Impala/be/src/common/object-pool.h:47
> #15 0x00000000023fd17c in impala::ObjectPool::~ObjectPool (this=0x153289a0,
> __in_chrg=<optimized out>) at /home/user/Impala/be/src/common/object-pool.h:34
> #16 0x0000000002b31f80 in impala::FragmentState::~FragmentState
> (this=0x153289a0, __in_chrg=<optimized out>) at
> /home/user/Impala/be/src/runtime/fragment-state.cc:192
> #17 0x0000000002b34000 in
> impala::ObjectPool::Add<impala::FragmentState>(impala::FragmentState*)::{lambda(void*)#1}::operator()(void*)
> const (this=0x0, obj=0x153289a0) at
> /home/user/Impala/be/src/common/object-pool.h:41
> #18 0x0000000002b34031 in
> impala::ObjectPool::Add<impala::FragmentState>(impala::FragmentState*)::{lambda(void*)#1}::_FUN(void*)
> () at /home/user/Impala/be/src/common/object-pool.h:41
> #19 0x00000000023fd213 in impala::ObjectPool::Clear (this=0x13c0aab8) at
> /home/user/Impala/be/src/common/object-pool.h:47
> #20 0x00000000023fd17c in impala::ObjectPool::~ObjectPool (this=0x13c0aab8,
> __in_chrg=<optimized out>) at /home/user/Impala/be/src/common/object-pool.h:34
> #21 0x0000000002a6d18d in impala::QueryState::~QueryState (this=0x13c0a000,
> __in_chrg=<optimized out>) at
> /home/user/Impala/be/src/runtime/query-state.cc:151
> #22 0x0000000002a621e4 in impala::QueryExecMgr::ReleaseQueryState
> (this=0x139e3680, qs=0x0) at
> /home/user/Impala/be/src/runtime/query-exec-mgr.cc:220
> #23 0x0000000004571a0b in impala::Coordinator::~Coordinator (this=0x16307700,
> __in_chrg=<optimized out>) at
> /home/user/Impala/be/src/runtime/coordinator.cc:147
> #24 0x0000000002fc0f63 in boost::checked_delete<impala::Coordinator>
> (x=0x16307700) at
> /opt/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/core/checked_delete.hpp:36
> #25 0x0000000002fbe791 in boost::scoped_ptr<impala::Coordinator>::~scoped_ptr
> (this=0x15e6d588, __in_chrg=<optimized out>) at
> /opt/Impala-Toolchain/toolchain-packages-gcc10.4.0/boost-1.74.0-p1/include/boost/smart_ptr/scoped_ptr.hpp:88
> #26 0x0000000002f9f1b6 in impala::ClientRequestState::~ClientRequestState
> (this=0x15e6cc00, __in_chrg=<optimized out>) at
> /home/user/Impala/be/src/service/client-request-state.cc:195{noformat}
>
> More precisely, it runs into a DCHECK at
> {noformat}
> #8 0x00000000041caa55 in impala::Expr::~Expr (this=0x148bfc00,
> __in_chrg=<optimized out>) at /home/user/Impala/be/src/exprs/expr.cc:47 47
> DCHECK(cache_entry_ == nullptr);
> {noformat}
> To reproduce the issue, simply use any native UDF or use the one defined in
> the test-warehouse:
> {noformat}
> create function if not exists default.identity(int) returns int location
> '/test-warehouse/libTestUdfs.so' symbol='Identity';{noformat}
> Interestingly, it still returns the result set as if the update was
> successful and returns the prompt:
> {noformat}
> [localhost:21050] default> UPDATE ice_t SET int_col = identity(int_col);
> Query: UPDATE ice_t SET int_col = identity(int_col)
> Query submitted at: [...]
> Query state can be monitored at: [...]
> Modified 1 row(s) in 0.82s
> [localhost:21050] default> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
