empiredan opened a new issue, #1848:
URL: https://github.com/apache/incubator-pegasus/issues/1848
Coredump information are as below:
```
#0 0x00007f6733ec02c7 in raise () from /lib64/libc.so.6
#1 0x00007f6733ec19b8 in abort () from /lib64/libc.so.6
#2 0x00007f6735778b66 in tcmalloc::Log(tcmalloc::LogMode, char const*, int,
tcmalloc::LogItem, tcmalloc::LogItem, tcmalloc::LogItem, tcmalloc::LogItem) ()
from /lib64/libtcmalloc_and_profiler.so.4
#3 0x00007f673576b444 in (anonymous namespace)::InvalidFree(void*) () from
/lib64/libtcmalloc_and_profiler.so.4
#4 0x00007f6736689c9e in rocksdb::ColumnFamilyData::~ColumnFamilyData() ()
from /home/sa_cluster/sp/pegasus/lib/librocksdb.so.8
#5 0x00007f6736689f35 in rocksdb::ColumnFamilyData::UnrefAndTryDelete() ()
from /home/sa_cluster/sp/pegasus/lib/librocksdb.so.8
#6 0x00007f6736689fa8 in rocksdb::SuperVersion::Cleanup() () from
/home/sa_cluster/sp/pegasus/lib/librocksdb.so.8
#7 0x00007f6736689f12 in rocksdb::ColumnFamilyData::UnrefAndTryDelete() ()
from /home/sa_cluster/sp/pegasus/lib/librocksdb.so.8
#8 0x00007f673668b8c5 in rocksdb::ColumnFamilySet::~ColumnFamilySet() ()
from /home/sa_cluster/sp/pegasus/lib/librocksdb.so.8
#9 0x00007f6736866e3b in rocksdb::VersionSet::~VersionSet() () from
/home/sa_cluster/sp/pegasus/lib/librocksdb.so.8
#10 0x00007f6736867341 in rocksdb::VersionSet::~VersionSet() () from
/home/sa_cluster/sp/pegasus/lib/librocksdb.so.8
#11 0x00007f673670315e in rocksdb::DBImpl::CloseHelper() () from
/home/sa_cluster/sp/pegasus/lib/librocksdb.so.8
#12 0x00007f67367038d1 in rocksdb::DBImpl::CloseImpl() () from
/home/sa_cluster/sp/pegasus/lib/librocksdb.so.8
#13 0x00007f673672174b in rocksdb::DBImpl::~DBImpl() () from
/home/sa_cluster/sp/pegasus/lib/librocksdb.so.8
#14 0x00007f6736788731 in rocksdb::DBImplReadOnly::~DBImplReadOnly() [clone
.localalias.348] () from /home/sa_cluster/sp/pegasus/lib/librocksdb.so.8
#15 0x000000000058a9bd in
pegasus::server::pegasus_server_impl::<lambda(bool)>::operator()(bool) const
(__closure=__closure@entry=0x7f66e08b4cc0,
remove_checkpoint=remove_checkpoint@entry=false) at
/data/sa_cluster/src/empiredan-pegasus-new/src/server/pegasus_server_impl.cpp:2147
#16 0x000000000058d3b9 in
pegasus::server::pegasus_server_impl::copy_checkpoint_to_dir_unsafe
(this=this@entry=0x2e71600, checkpoint_dir=<optimized out>,
checkpoint_decree=checkpoint_decree@entry=0x7f66e08b5a90,
flush_memtable=flush_memtable@entry=false)
at
/data/sa_cluster/src/empiredan-pegasus-new/src/server/pegasus_server_impl.cpp:2172
#17 0x00000000005a7555 in
pegasus::server::pegasus_server_impl::async_checkpoint (this=0x2e71600,
flush_memtable=<optimized out>) at
/data/sa_cluster/src/empiredan-pegasus-new/src/server/pegasus_server_impl.cpp:2039
#18 0x00007f6737fdeae3 in
dsn::replication::replica::background_async_checkpoint (this=0x2f47200,
is_emergency=<optimized out>) at
/data/sa_cluster/src/empiredan-pegasus-new/src/replica/replica_chkpt.cpp:273
#19 0x00007f673820e401 in dsn::task::exec_internal (this=0x56ca690) at
/data/sa_cluster/src/empiredan-pegasus-new/src/runtime/task/task.cpp:173
#20 0x00007f6738226f07 in dsn::task_worker::loop (this=0x26f3810) at
/data/sa_cluster/src/empiredan-pegasus-new/src/runtime/task/task_worker.cpp:245
#21 0x00007f6738227a70 in dsn::task_worker::run_internal (this=0x26f3810) at
/data/sa_cluster/src/empiredan-pegasus-new/src/runtime/task/task_worker.cpp:225
#22 0x00007f6736dab23f in execute_native_thread_routine () from
/home/sa_cluster/sp/pegasus/lib/librocksdb.so.8
#23 0x00007f673553ddd5 in start_thread () from /lib64/libpthread.so.0
#24 0x00007f6733f8802d in clone () from /lib64/libc.so.6
```
Replica server exits abnormally with coredump immediately after this
logging:
```
I2024-01-08 18:41:18.943 (1704710478943709712 43700)
replica.rep_long5.0404001700000097:
pegasus_server_impl.cpp:2131:copy_checkpoint_to_dir_unsafe():
[[email protected]:8171] copy checkpoint to
dir(/data/sa_cluster/pegasus_cluster/replica/reps/1.0.pegasus/data/checkpoint.tmp.1704710478941804)
succeed
```
Running commands:
```
./run.sh build --test -t release -v -j $(nproc)
```
GCC version:
```
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/opt/rh/devtoolset-7/root/usr/libexec/gcc/x86_64-redhat-linux/7/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap
--enable-languages=c,c++,fortran,lto --prefix=/opt/rh/devtoolset-7/root/usr
--mandir=/opt/rh/devtoolset-7/root/usr/share/man
--infodir=/opt/rh/devtoolset-7/root/usr/share/info
--with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared
--enable-threads=posix --enable-checking=release --enable-multilib
--with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions
--enable-gnu-unique-object --enable-linker-build-id
--with-gcc-major-version-only --enable-plugin --with-linker-hash-style=gnu
--enable-initfini-array --with-default-libstdcxx-abi=gcc4-compatible
--with-isl=/builddir/build/BUILD/gcc-7.3.1-20180303/obj-x86_64-redhat-linux/isl-install
--enable-libmpx --enable-gnu-indirect-function --with-tune=generic
--with-arch_32=i686 --build=x86_64-redhat-linux
Thread model: posix
gcc version 7.3.1 20180303 (Red Hat 7.3.1-5) (GCC)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
