[
https://issues.apache.org/jira/browse/IMPALA-13891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17942145#comment-17942145
]
ASF subversion and git services commented on IMPALA-13891:
----------------------------------------------------------
Commit 70391d7f0dac459b15b0f168a476ec1aa92c2586 in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=70391d7f0 ]
IMPALA-13891: Unregister MemTrackers from parents in TupleFileReader/Writer
Running failure/test_failpoints.py with tuple caching causes a crash.
The underlying cause is that the MemTrackers for TupleFileReader and
TupleFileWriter get deleted without being unregistered from their
parents, causing a SIGSEGV. Ordinarily, MemTrackers live from Prepare()
to the final teardown of the query. This means that they are always
alive past the point when something would be reading from them.
TupleFileReader/TupleFileWriter can be deleted before the query is
over, so this changes the code to also deregister the MemTracker
from its parent.
Testing:
- Verified that failure/test_failpoints.py stops crashing when running
with tuple caching
Change-Id: I14e5484de35f78da74c1246d319553496793de81
Reviewed-on: http://gerrit.cloudera.org:8080/22665
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> failure/test_failpoints.py crashes when running with tuple caching
> ------------------------------------------------------------------
>
> Key: IMPALA-13891
> URL: https://issues.apache.org/jira/browse/IMPALA-13891
> Project: IMPALA
> Issue Type: Bug
> Components: Backend
> Affects Versions: Impala 5.0.0
> Reporter: Joe McDonnell
> Assignee: Joe McDonnell
> Priority: Critical
>
> When running failure/test_failpoints.py with tuple caching enabled, Impala
> crashes with this stack:
> {noformat}
> Crash reason: SIGSEGV /SEGV_MAPERR
> Crash address: 0xb8
> 0 impalad!impala::MemTracker::LogUsage(int,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, long*) [atomic_base.h : 436 + 0x0]
> 1 impalad!impala::MemTracker::LogUsage(int,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, std::__cxx11::list<impala::MemTracker*,
> std::allocator<impala::MemTracker*> > const&, long*) [mem-tracker.cc : 370 +
> 0x13]
> 2 impalad!impala::MemTracker::LogUsage(int,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, long*) [mem-tracker.cc : 345 + 0x23]
> 3 impalad!impala::MemTracker::LogUsage(int,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, std::__cxx11::list<impala::MemTracker*,
> std::allocator<impala::MemTracker*> > const&, long*) [mem-tracker.cc : 370 +
> 0x13]
> 4 impalad!impala::MemTracker::LogUsage(int,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, long*) [mem-tracker.cc : 345 + 0x23]
> 5 impalad!impala::MemTracker::LogUsage(int,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, std::__cxx11::list<impala::MemTracker*,
> std::allocator<impala::MemTracker*> > const&, long*) [mem-tracker.cc : 370 +
> 0x13]
> 6 impalad!impala::MemTracker::LogUsage(int,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, long*) [mem-tracker.cc : 345 + 0x23]
> 7 impalad!impala::MemTracker::MemLimitExceeded(impala::MemTracker*,
> impala::RuntimeState*, std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&, long) [mem-tracker.cc
> : 518 + 0x21]
> 8
> impalad!impala::ExecNode::ExecDebugActionImpl(impala::TExecNodePhase::type,
> impala::RuntimeState*) [exec-node.cc : 454 + 0x14]
> 9 impalad!impala::PartitionedHashJoinNode::GetNext(impala::RuntimeState*,
> impala::RowBatch*, bool*) [exec-node.h : 459 + 0x12]
> 10 impalad!impala::FragmentInstanceState::ExecInternal()
> [fragment-instance-state.cc : 446 + 0x12]
> 11 impalad!impala::FragmentInstanceState::Exec() [fragment-instance-state.cc
> : 104 + 0xc]
> 12 impalad!impala::QueryState::ExecFInstance(impala::FragmentInstanceState*)
> [query-state.cc : 1013 + 0xc]
> 13 impalad!invoke [query-state.cc : 918 + 0x8]
> 14 impalad!impala::Thread::SuperviseThread(std::__cxx11::basic_string<char,
> std::char_traits<char>, std::allocator<char> > const&,
> std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>
> > const&, boost::function<void ()> const&, impala::ThreadDebugInfo const*,
> impala::Promise<long, (impala::PromiseMode)0>*) [function_template.hpp : 763
> + 0x3]
> ...{noformat}
> After some debugging, it looks like the MemTrackers on the TupleFileWriter
> and TupleFileReader get deleted without getting deregistered from their
> parent MemTracker.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
