ryanzryu opened a new pull request, #29365:
URL: https://github.com/apache/doris/pull/29365
## Proposed changes
export table whit jsonb field will be core，get stack like this：
*** Query id: 0-0 ***
*** Aborted at 1702436737 (unix time) try "date -d @1702436737" if you are
using GNU date ***
*** Current BE git commitID: 0b1372c ***
*** SIGSEGV address not mapped to object (@0x0) received by PID 449814 (TID
0x7fee0bb14700) from PID 0; stack trace: ***
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int,
siginfo_t*, void*) at /data/doris-1.x/be/src/common/signal_handler.h:420
1# os::Linux::chained_handler(int, siginfo*, void*) in
/usr/local/jdk/jre/lib/amd64/server/libjvm.so
2# JVM_handle_linux_signal in /usr/local/jdk/jre/lib/amd64/server/libjvm.so
3# signalHandler(int, siginfo*, void*) in
/usr/local/jdk/jre/lib/amd64/server/libjvm.so
4# 0x00007FEE4AB51400 in /lib64/libc.so.6
5# memcpy at
/data/doris-1.x/be/src/glibc-compatibility/memcpy/memcpy_x86_64.cpp:219
6# doris::FieldTypeTraits<(doris::FieldType)13>::direct_copy(void*, void
const*) at /data/doris-1.x/be/src/olap/types.h:1439
7# void doris::direct_copy_row<doris::RowCursor,
doris::RowCursor>(doris::RowCursor*, doris::RowCursor const&) at
/data/doris-1.x/be/src/olap/row.h:120
8# doris::TupleReader::_unique_key_next_row(doris::RowCursor*,
doris::MemPool*, doris::ObjectPool*, bool*) at
/data/doris-1.x/be/src/olap/tuple_reader.cpp:190
9# doris::OlapScanner::get_batch(doris::RuntimeState*, doris::RowBatch*,
bool*) at /data/doris-1.x/be/src/exec/olap_scanner.cpp:372
10# doris::OlapScanNode::scanner_thread(doris::OlapScanner*) at
/data/doris-1.x/be/src/exec/olap_scan_node.cpp:1751
11# doris::PriorityWorkStealingThreadPool::work_thread(int) at
/data/doris-1.x/be/src/util/priority_work_stealing_thread_pool.hpp:136
12# execute_native_thread_routine at
../../../../../libstdc++-v3/src/c++11/thread.cc:84
13# start_thread in /lib64/libpthread.so.0
14# __clone in /lib64/libc.so.6
and I gdb this core
(gdb) f 8
#8 0x000055a595362f65 in doris::RowBlock::init (this=0x7f0bdc238c20,
block_info=...) at /data/doris-1.x/be/src/olap/row_block.cpp:61
61 /data/doris-1.x/be/src/olap/row_block.cpp: No such file or directory.
(gdb) p
_schema._M_ptr._cols._M_impl._M_finish-_schema._M_ptr._cols._M_impl._M_start
$37 = 53 (表的列数)
(gdb) p _field_offset_in_memory._M_impl._M_finish -
_field_offset_in_memory._M_impl._M_start
$38 = 53 (记录遍历每个列需要申请的内存数，一直递增,这里一共进行了5次跳变，每次 2G，最后变成了10G)
(gdb) p *(_field_offset_in_memory._M_impl._M_start)@53
$39 = {0, 2, 6, 15, 18, 27, 44, 46, 65582, 131118, 131120, 196656, 196665,
196674, 196691, 196708, 196725, 196742, 196744, 262280, 2147745928, 4295229576,
4295229578, 4295229595, 4295229612, 4295229617, 4295229634, 4295229643,
6442713291, 6442713293, 8590196941, 8590196958, 8590196975, 8590196984,
8590196986, 8590197003, 8590262539, 8590328075, 10737811723, 10737877259,
10737942795, 10738008331, 10738008333, 10738073869, 10738073874, 10738073879,
10738073884, 10738073886, 10738073903, 10738073912, 10738073929,
18446744073709551615, 18446744073709551615} ------这里中间跳变了5次2G大小的长度
(gdb) p _mem_buf_bytes
$40 = 10995787712512 (_compute_layout 函数的计算结果，有下面的
_info.row_num*_mem_row_bytes 得到，结果是10T，这里直接导致了 _mem_buf = new
char[_mem_buf_bytes] core掉 )
(gdb) p _info.row_num
$41 = 1024
(gdb) p _mem_row_bytes
$42 = 10738073938
so this pr support jsonb as a variable length for avoid this core
<!--Describe your changes.-->
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[[email protected]](mailto:[email protected]) by explaining why you
chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
