xinyiZzz opened a new pull request, #16537: URL: https://github.com/apache/doris/pull/16537
# Proposed changes Issue Number: close #xxx ## Problem summary Compilation by release, core in `je_mallctl`, similar issue: https://github.com/jemalloc/jemalloc/issues/1234, comment `doris::MemInfo::refresh_allocator_mem()`; will also core ``` I0207 16:53:37.499392 2461885 load_channel.cpp:42] load channel removed. mem peak usage=0, info=label: LoadChannel#senderIp=10.16.10.7#loadID=ba4c39e32a2cd0f6-1da4e4bb5935e3b4; consumption: 0; peak_consumption: 0; , load_id=ba4c39e32a2cd0f6-1da4e4bb5935e3b4, is high priority=0, sender_ip=10.16.10.7 *** Query id: 0-0 *** *** Aborted at 1675760017 (unix time) try "date -d @1675760017" if you are using GNU date *** *** Current BE git commitID: 27216dc7e *** *** SIGSEGV unkown detail explain (@0x0) received by PID 2460953 (TID 2461055 OR 0x7fae3d72c700) from PID 0; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /doris/be/src/common/signal_handler.h:428 1# os::Linux::chained_handler(int, siginfo*, void*) in /.local/jdk1.8.0_131/jre/lib/amd64/server/libjvm.so 2# JVM_handle_linux_signal in /.local/jdk1.8.0_131/jre/lib/amd64/server/libjvm.so 3# signalHandler(int, siginfo*, void*) in /.local/jdk1.8.0_131/jre/lib/amd64/server/libjvm.so 4# 0x00007FAE77CD8C20 in /lib64/libpthread.so.0 5# je_arena_stats_merge at ../src/arena.c:206 6# ctl_arena_stats_amerge at ../src/ctl.c:808 7# ctl_refresh at ../src/ctl.c:1065 8# epoch_ctl at ../src/ctl.c:1626 9# je_ctl_byname at ../src/ctl.c:1313 10# doris::MemInfo::refresh_allocator_mem() at /doris/be/src/util/mem_info.cpp:76 11# doris::Daemon::memory_maintenance_thread() at /doris/be/src/common/daemon.cpp:184 12# doris::Thread::supervise_thread(void*) at /doris/be/src/util/thread.cpp:454 13# start_thread in /lib64/libpthread.so.0 14# __GI___clone in /lib64/libc.so.6 ``` Compiled by Asan, the core is in `LocalFileReader::read_at`, actually `result.data` stack memory overflow. ``` 0 0x0000561b927ca8e4 in __asan::Allocator::Allocate(unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType, bool) () #1 0x0000561b927c6889 in __asan::asan_memalign(unsigned long, unsigned long, __sanitizer::BufferedStackTrace*, __asan::AllocType) () #2 0x0000561b928536c0 in operator new(unsigned long) () #3 0x0000561b928debd8 in __gnu_cxx::new_allocator<boost::stacktrace::frame>::allocate (this=0x7feb5eb405b0, __n=37) at /doris/ldb_toolchain/include/c++/11/ext/new_allocator.h:121 #4 0x0000561b928dc6e6 in std::allocator_traits<std::allocator<boost::stacktrace::frame> >::allocate (__a=..., __n=37) at /doris/ldb_toolchain/include/c++/11/bits/alloc_traits.h:460 #5 0x0000561b928d9306 in std::_Vector_base<boost::stacktrace::frame, std::allocator<boost::stacktrace::frame> >::_M_allocate (this=0x7feb5eb405b0, __n=37) at /doris/ldb_toolchain/include/c++/11/bits/stl_vector.h:346 #6 0x0000561b928d5283 in std::vector<boost::stacktrace::frame, std::allocator<boost::stacktrace::frame> >::reserve (this=0x7feb5eb405b0, __n=37) at /doris/ldb_toolchain/include/c++/11/bits/vector.tcc:78 #7 0x0000561b928d1345 in boost::stacktrace::basic_stacktrace<std::allocator<boost::stacktrace::frame> >::fill (this=0x7feb5eb405b0, begin=0x7feb5eb40060, size=37) at /doris/core/thirdparty/installed/include/boost/stacktrace/stacktrace.hpp:50 #8 0x0000561b928cb38e in boost::stacktrace::basic_stacktrace<std::allocator<boost::stacktrace::frame> >::init (this=0x7feb5eb405b0, frames_to_skip=0, max_depth=18446744073709551615) at /doris/core/thirdparty/installed/include/boost/stacktrace/stacktrace.hpp:77 #9 0x0000561b9289fc63 in boost::stacktrace::basic_stacktrace<std::allocator<boost::stacktrace::frame> >::basic_stacktrace (this=<optimized out>) at /doris/core/thirdparty/installed/include/boost/stacktrace/stacktrace.hpp:126 #10 doris::signal::(anonymous namespace)::FailureSignalHandler (signal_number=11, signal_info=0x7feb5eb409f0, ucontext=0x7feb5eb408c0) at /doris/core/be/src/common/signal_handler.h:428 #11 0x00007fecb33311a2 in os::Linux::chained_handler(int, siginfo*, void*) () from /doris/jdk1.8.0_131/jre/lib/amd64/server/libjvm.so #12 0x00007fecb3337826 in JVM_handle_linux_signal () from /doris/jdk1.8.0_131/jre/lib/amd64/server/libjvm.so #13 0x00007fecb332de13 in signalHandler(int, siginfo*, void*) () from /doris/jdk1.8.0_131/jre/lib/amd64/server/libjvm.so #14 <signal handler called> #15 0x0000561b9286095d in __asan::GetCurrentTidOrInvalid() () #16 0x0000561b9285a319 in __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) () #17 0x0000561b927dda51 in __interceptor_pread.part.0 () #18 0x0000561b929c9043 in doris::io::LocalFileReader::read_at (this=0x60b00069e730, offset=1075220955, result=..., bytes_read=0x7feb5f4029a0) at /doris/core/be/src/io/fs/local_file_reader.cpp:64 #19 0x0000561ba3dbefc6 in doris::vectorized::parse_thrift_footer (file=..., file_metadata=...) at /doris/core/be/src/vec/exec/format/parquet/parquet_thrift_util.h:64 #20 0x0000561ba3dc558a in doris::vectorized::ParquetReader::_open_file (this=0x619000042980) at /doris/core/be/src/vec/exec/format/parquet/vparquet_reader.cpp:169 #21 0x0000561ba3dc59a3 in doris::vectorized::ParquetReader::open (this=0x619000042980) at /doris/core/be/src/vec/exec/format/parquet/vparquet_reader.cpp:182 #22 0x0000561ba45bcf68 in doris::vectorized::VFileScanner::_get_next_reader (this=0x61a000063080) at /doris/core/be/src/vec/exec/scan/vfile_scanner.cpp:493 #23 0x0000561ba45b3ede in doris::vectorized::VFileScanner::_get_block_impl (this=0x61a000063080, state=0x61c00005b080, block=0x60d0012743f0, eof=0x7feb5f403fb0) at /doris/core/be/src/vec/exec/scan/vfile_scanner.cpp:138 #24 0x0000561ba427ec7e in doris::vectorized::VScanner::get_block (this=0x61a000063080, state=0x61c00005b080, block=0x60d0012743f0, eof=0x7feb5f403fb0) at /doris/core/be/src/vec/exec/scan/vscanner.cpp:58 #25 0x0000561ba424d42b in doris::vectorized::ScannerScheduler::_scanner_scan (this=0x6060001001c0, scheduler=0x6060001001c0, ctx=0x6160009a0280, scanner=0x61a000063080) at /doris/core/be/src/vec/exec/scan/scanner_scheduler.cpp:244 #26 0x0000561ba424ba16 in operator() (__closure=0x60300057c040) at /doris/core/be/src/vec/exec/scan/scanner_scheduler.cpp:167 #27 0x0000561ba424fe82 in std::__invoke_impl<void, doris::vectorized::ScannerScheduler::_schedule_scanners(doris::vectorized::ScannerContext*)::<lambda()>&>(std::__invoke_other, struct {...} &) (__f=...) at /doris/ldb_toolchain/include/c++/11/bits/invoke.h:61 #28 0x0000561ba424fcfa in std::__invoke_r<void, doris::vectorized::ScannerScheduler::_schedule_scanners(doris::vectorized::ScannerContext*)::<lambda()>&>(struct {...} &) (__fn=...) at /doris/ldb_toolchain/include/c++/11/bits/invoke.h:111 #29 0x0000561ba424f78e in std::_Function_handler<void(), doris::vectorized::ScannerScheduler::_schedule_scanners(doris::vectorized::ScannerContext*)::<lambda()> >::_M_invoke(const std::_Any_data &) (__functor=...) at /doris/ldb_toolchain/include/c++/11/bits/std_function.h:291 #30 0x0000561b94368fba in std::function<void ()>::operator()() const (this=0x7feb5f4043f8) at /doris/ldb_toolchain/include/c++/11/bits/std_function.h:560 #31 0x0000561b94366667 in doris::PriorityThreadPool::work_thread (this=0x6140000c0e40, thread_id=0) at /doris/core/be/src/util/priority_thread_pool.hpp:138 #32 0x0000561b94378d39 in std::__invoke_impl<void, void (doris::PriorityThreadPool::* const&)(int), doris::PriorityThreadPool*&, int&> (__f= @0x604000572c98: (void (doris::PriorityThreadPool::*)(doris::PriorityThreadPool * const, int)) 0x561b943664b2 <doris::PriorityThreadPool::work_thread(int)>, __t=@0x604000572cb0: 0x6140000c0e40) at /doris/ldb_toolchain/include/c++/11/bits/invoke.h:74 #33 0x0000561b94378b79 in std::__invoke<void (doris::PriorityThreadPool::* const&)(int), doris::PriorityThreadPool*&, int&> (__fn= @0x604000572c98: (void (doris::PriorityThreadPool::*)(doris::PriorityThreadPool * const, int)) 0x561b943664b2 <doris::PriorityThreadPool::work_thread(int)>) at /doris/ldb_toolchain/include/c++/11/bits/invoke.h:96 #34 0x0000561b94378b15 in std::_Mem_fn_base<void (doris::PriorityThreadPool::*)(int), true>::operator()<doris::PriorityThreadPool*&, int&> (this=0x604000572c98) at /doris/ldb_toolchain/include/c++/11/functional:131 #35 0x0000561b94378a8e in std::__invoke_impl<void, std::_Mem_fn<void (doris::PriorityThreadPool::*)(int)>&, doris::PriorityThreadPool*&, int&> (__f=...) at /doris/ldb_toolchain/include/c++/11/bits/invoke.h:61 #36 0x0000561b943789e5 in std::__invoke_r<void, std::_Mem_fn<void (doris::PriorityThreadPool::*)(int)>&, doris::PriorityThreadPool*&, int&> (__fn=...) at /doris/ldb_toolchain/include/c++/11/bits/invoke.h:111 #37 0x0000561b9437889d in std::_Bind_result<void, std::_Mem_fn<void (doris::PriorityThreadPool::*)(int)> (doris::PriorityThreadPool*, int)>::__call<void, , 0ul, 1ul>(std::tuple<>&&, std::_Index_tuple<0ul, 1ul>) (this=0x604000572c98, __args=...) at /doris/ldb_toolchain/include/c++/11/functional:570 #38 0x0000561b94378713 in std::_Bind_result<void, std::_Mem_fn<void (doris::PriorityThreadPool::*)(int)> (doris::PriorityThreadPool*, int)>::operator()<>() (this=0x604000572c98) at /doris/ldb_toolchain/include/c++/11/functional:629 #39 0x0000561b94378650 in std::__invoke_impl<void, std::_Bind_result<void, std::_Mem_fn<void (doris::PriorityThreadPool::*)(int)> (doris::PriorityThreadPool*, int)>>(std::__invoke_other, std::_Bind_result<void, std::_Mem_fn<void (doris::PriorityThreadPool::*)(int)> (doris::PriorityThreadPool*, int)>&&) (__f=...) at /doris/ldb_toolchain/include/c++/11/bits/invoke.h:61 #40 0x0000561b943785f7 in std::__invoke<std::_Bind_result<void, std::_Mem_fn<void (doris::PriorityThreadPool::*)(int)> (doris::PriorityThreadPool*, int)>>(std::_Bind_result<void, std::_Mem_fn<void (doris::PriorityThreadPool::*)(int)> (doris::PriorityThreadPool*, int)>&&) (__fn=...) at /doris/ldb_toolchain/include/c++/11/bits/invoke.h:96 ``` ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [[email protected]](mailto:[email protected]) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
