xinyiZzz opened a new pull request, #11485:
URL: https://github.com/apache/doris/pull/11485
# Proposed changes
Issue Number: close #11480
## Problem summary
Optimize the returned information when query exceeds memory limit.
mysql client return error:
````
Memory limit exceeded: exec node:unknown, `set exec_mem_limit=xxx` to change
limit, failed mem consume:<consume_size=977.55 KB, mem_limit=222,
mem_used=1088408,
tracker_label=RuntimeState:instance:2147f2fe254a49b1-ad986d48faa4c9d9, in
backend=172.24.47.117 free memory left=98.58 GB. details mem usage see be.INFO.
````
be.INFO log, divided into three parts:
1. Exceed memory limit info;
2. mem tracker limiter and child snapshots
3. Stack trace
```
W0803 22:10:10.533586 3537580 mem_tracker_limiter.cpp:239] exec
node:unknown, `set exec_mem_limit=xxx` to change limit, failed mem
consume:<consume_size=979.46 KB, mem_limit=222, mem_used=2874039,
tracker_label=RuntimeState:instance:2147f2fe254a49b1-ad986d48faa4ca12, in
backend=172.24.47.117 free memory left=98.44 GB. details mem usage see
be.INFO
MemTrackerLimiter
Label=RuntimeState:instance:2147f2fe254a49b1-ad986d48faa4ca12, Limit=222.00 B,
Used=2.74 MB, Peak=1.78 MB, Exceeded=true
MemTracker Label=RuntimeFilterMgr, Parent
Label=RuntimeState:instance:2147f2fe254a49b1-ad986d48faa4ca12, Used=0, Peak=0
MemTracker Label=BufferedBlockMgr2, Parent
Label=RuntimeState:instance:2147f2fe254a49b1-ad986d48faa4ca12, Used=0, Peak=0
MemTracker Label=ExecNode:VAGGREGATION_NODE (id=14), Parent
Label=RuntimeState:instance:2147f2fe254a49b1-ad986d48faa4ca12, Used=0, Peak=0
MemTracker Label=ExecNode:VOLAP_SCAN_NODE (id=13), Parent
Label=RuntimeState:instance:2147f2fe254a49b1-ad986d48faa4ca12, Used=0, Peak=0
MemTracker Label=OlapScanners, Parent
Label=RuntimeState:instance:2147f2fe254a49b1-ad986d48faa4ca12, Used=0, Peak=0
MemTracker Label=AggregationNode:Data, Parent
Label=RuntimeState:instance:2147f2fe254a49b1-ad986d48faa4ca12, Used=515.98 KB,
Peak=259.98 KB
MemTracker Label=VDataStreamSender:2147f2fe254a49b1-ad986d48faa4ca12,
Parent Label=RuntimeState:instance:2147f2fe254a49b1-ad986d48faa4ca12, Used=0,
Peak=0
0#
doris::MemTrackerLimiter::mem_limit_exceeded(std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const&, long) at
/mnt/disk1/liyifan/doris/core/be/src/runtime/memory/mem_tracker_limiter.cpp:238
1# doris::ThreadMemTrackerMgr::exceeded(long) at
/mnt/disk1/liyifan/doris/core/be/src/runtime/memory/thread_mem_tracker_mgr.cpp:65
2# void doris::ThreadMemTrackerMgr::flush_untracked_mem<true>() at
/mnt/disk1/liyifan/doris/core/be/src/runtime/memory/thread_mem_tracker_mgr.h:199
3# doris::ThreadMemTrackerMgr::consume(long) at
/mnt/disk1/liyifan/doris/core/be/src/runtime/memory/thread_mem_tracker_mgr.h:170
4# new_hook(void const*, unsigned long) at
/mnt/disk1/liyifan/doris/core/be/src/runtime/memory/tcmalloc_hook.h:48
5# MallocHook::InvokeNewHookSlow(void const*, unsigned long) at
src/malloc_hook.cc:498
6# tcmalloc::allocate_full_cpp_throw_oom(unsigned long) at
src/tcmalloc.cc:1815
7# snappy::internal::WorkingMemory::WorkingMemory(unsigned long) in
/mnt/disk1/liyifan/doris/core/output_run/be/lib/doris_be
8# snappy::Compress(snappy::Source*, snappy::Sink*) in
/mnt/disk1/liyifan/doris/core/output_run/be/lib/doris_be
9# snappy::RawCompress(char const*, unsigned long, char*, unsigned long*)
in /mnt/disk1/liyifan/doris/core/output_run/be/lib/doris_be
10# doris::vectorized::Block::serialize(doris::PBlock*, unsigned long*,
unsigned long*, bool) const at
/mnt/disk1/liyifan/doris/core/be/src/vec/core/block.cpp:722
11#
doris::vectorized::VDataStreamSender::serialize_block(doris::vectorized::Block*,
doris::PBlock*, int) at
/mnt/disk1/liyifan/doris/core/be/src/vec/sink/vdata_stream_sender.cpp:592
12# doris::vectorized::VDataStreamSender::Channel::send_current_block(bool)
at /mnt/disk1/liyifan/doris/core/be/src/vec/sink/vdata_stream_sender.cpp:87
13#
doris::vectorized::VDataStreamSender::Channel::add_rows(doris::vectorized::Block*,
std::vector<int, std::allocator<int> > const&) at
/mnt/disk1/liyifan/doris/core/be/src/vec/sink/vdata_stream_sender.cpp:218
14# doris::Status
doris::vectorized::VDataStreamSender::channel_add_rows<std::vector<doris::vectorized::VDataStreamSender::Channel*,
std::allocator<doris::vectorized::VDataStreamSender::Channel*> >,
std::vector<unsigned long, std::allocator<unsigned long> > >(std::vecto
r<doris::vectorized::VDataStreamSender::Channel*,
std::allocator<doris::vectorized::VDataStreamSender::Channel*> >&, int,
std::vector<unsigned long, std::allocator<unsigned long> > const&, int,
doris::vectorized::Block*) at /mnt/disk1/liyifan/doris/core/be/src/vec/sink/
vdata_stream_sender.h:316
15# doris::vectorized::VDataStreamSender::send(doris::RuntimeState*,
doris::vectorized::Block*) at
/mnt/disk1/liyifan/doris/core/be/src/vec/sink/vdata_stream_sender.cpp:516
16# doris::PlanFragmentExecutor::open_vectorized_internal() at
/mnt/disk1/liyifan/doris/core/be/src/runtime/plan_fragment_executor.cpp:299
17# doris::PlanFragmentExecutor::open() at
/mnt/disk1/liyifan/doris/core/be/src/runtime/plan_fragment_executor.cpp:239
18# doris::FragmentExecState::execute() at
/mnt/disk1/liyifan/doris/core/be/src/runtime/fragment_mgr.cpp:246
19#
doris::FragmentMgr::_exec_actual(std::shared_ptr<doris::FragmentExecState>,
std::function<void (doris::PlanFragmentExecutor*)>) at
/mnt/disk1/liyifan/doris/core/be/src/runtime/fragment_mgr.cpp:500
20# doris::FragmentMgr::exec_plan_fragment(doris::TExecPlanFragmentParams
const&, std::function<void
(doris::PlanFragmentExecutor*)>)::{lambda()#1}::operator()() const at
/mnt/disk1/liyifan/doris/core/be/src/runtime/fragment_mgr.cpp:681
21# void std::__invoke_impl<void,
doris::FragmentMgr::exec_plan_fragment(doris::TExecPlanFragmentParams const&,
std::function<void
(doris::PlanFragmentExecutor*)>)::{lambda()#1}&>(std::__invoke_other,
doris::FragmentMgr::exec_plan_fragment(doris::TExecPlanFragmentParams
const&, std::function<void (doris::PlanFragmentExecutor*)>)::{lambda()#1}&)
at /mnt/disk1/liyifan/doris/ldb_toolchain/include/c++/11/bits/invoke.h:61
22# std::enable_if<is_invocable_r_v<void,
doris::FragmentMgr::exec_plan_fragment(doris::TExecPlanFragmentParams const&,
std::function<void (doris::PlanFragmentExecutor*)>)::{lambda()#1}&>,
void>::type std::__invoke_r<void,
doris::FragmentMgr::exec_plan_fragment(doris::T
ExecPlanFragmentParams const&, std::function<void
(doris::PlanFragmentExecutor*)>)::{lambda()#1}&>(doris::FragmentMgr::exec_plan_fragment(doris::TExecPlanFragmentParams
const&, std::function<void (doris::PlanFragmentExecutor*)>)::{lambda()#1}&) at
/mnt/disk1/liyifan/dor
is/ldb_toolchain/include/c++/11/bits/invoke.h:117
23# std::_Function_handler<void (),
doris::FragmentMgr::exec_plan_fragment(doris::TExecPlanFragmentParams const&,
std::function<void
(doris::PlanFragmentExecutor*)>)::{lambda()#1}>::_M_invoke(std::_Any_data
const&) at /mnt/disk1/liyifan/doris/ldb_toolchain/include/c++/1
1/bits/std_function.h:292
24# std::function<void ()>::operator()() const at
/mnt/disk1/liyifan/doris/ldb_toolchain/include/c++/11/bits/std_function.h:560
25# doris::FunctionRunnable::run() at
/mnt/disk1/liyifan/doris/core/be/src/util/threadpool.cpp:45
26# doris::ThreadPool::dispatch_thread() at
/mnt/disk1/liyifan/doris/core/be/src/util/threadpool.cpp:548
27# void std::__invoke_impl<void, void (doris::ThreadPool::*&)(),
doris::ThreadPool*&>(std::__invoke_memfun_deref, void
(doris::ThreadPool::*&)(), doris::ThreadPool*&) at
/mnt/disk1/liyifan/doris/ldb_toolchain/include/c++/11/bits/invoke.h:74
28# std::__invoke_result<void (doris::ThreadPool::*&)(),
doris::ThreadPool*&>::type std::__invoke<void (doris::ThreadPool::*&)(),
doris::ThreadPool*&>(void (doris::ThreadPool::*&)(), doris::ThreadPool*&) at
/mnt/disk1/liyifan/doris/ldb_toolchain/include/c++/11/bits/invo
ke.h:97
29# void std::_Bind<void
(doris::ThreadPool::*(doris::ThreadPool*))()>::__call<void, ,
0ul>(std::tuple<>&&, std::_Index_tuple<0ul>) at
/mnt/disk1/liyifan/doris/ldb_toolchain/include/c++/11/functional:422
30# void std::_Bind<void
(doris::ThreadPool::*(doris::ThreadPool*))()>::operator()<, void>() at
/mnt/disk1/liyifan/doris/ldb_toolchain/include/c++/11/functional:505
31# void std::__invoke_impl<void, std::_Bind<void
(doris::ThreadPool::*(doris::ThreadPool*))()>&>(std::__invoke_other,
std::_Bind<void (doris::ThreadPool::*(doris::ThreadPool*))()>&) at
/mnt/disk1/liyifan/doris/ldb_toolchain/include/c++/11/bits/invoke.h:61
32# std::enable_if<is_invocable_r_v<void, std::_Bind<void
(doris::ThreadPool::*(doris::ThreadPool*))()>&>, void>::type
std::__invoke_r<void, std::_Bind<void
(doris::ThreadPool::*(doris::ThreadPool*))()>&>(std::_Bind<void
(doris::ThreadPool::*(doris::ThreadPool*))()>&) a
t /mnt/disk1/liyifan/doris/ldb_toolchain/include/c++/11/bits/invoke.h:117
33# std::_Function_handler<void (), std::_Bind<void
(doris::ThreadPool::*(doris::ThreadPool*))()> >::_M_invoke(std::_Any_data
const&) at
/mnt/disk1/liyifan/doris/ldb_toolchain/include/c++/11/bits/std_function.h:292
34# std::function<void ()>::operator()() const at
/mnt/disk1/liyifan/doris/ldb_toolchain/include/c++/11/bits/std_function.h:560
35# doris::Thread::supervise_thread(void*) at
/mnt/disk1/liyifan/doris/core/be/src/util/thread.cpp:409
36# start_thread in /lib64/libpthread.so.0
37# clone in /lib64/libc.so.6
```
## Checklist(Required)
1. Does it affect the original behavior:
- [ ] Yes
- [ ] No
- [ ] I don't know
2. Has unit tests been added:
- [ ] Yes
- [ ] No
- [ ] No Need
3. Has document been added or modified:
- [ ] Yes
- [ ] No
- [ ] No Need
4. Does it need to update dependencies:
- [ ] Yes
- [ ] No
5. Are there any changes that cannot be rolled back:
- [ ] Yes (If Yes, please explain WHY)
- [ ] No
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[[email protected]](mailto:[email protected]) by explaining why you
chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
