ninsmiracle opened a new issue, #2006: URL: https://github.com/apache/incubator-pegasus/issues/2006
## Bug Report 1. What did you do? Doing bulkload (download sst file stage) with any action which need to restart ONE node,may cause ALL nodes coredump.  2. What did you see ? There are three kind of coredump in different nodes Type one: ``` (gdb) #0 0x00007fc9093669ef in signalHandler(int, siginfo*, void*) () from /opt/soft/openjdk1.8.0/jre/lib/amd64/server/libjvm.so #1 <signal handler called> #2 0x00007fc9072ea793 in dsn::utils::filesystem::get_normalized_path ( path=..., npath=...) at /home/work/temp/pegasus/src/rdsn/src/utils/filesystem.cpp:116 #3 0x00007fc9072ebb4d in dsn::utils::filesystem::path_combine (path1=..., path2=...) at /home/work/temp/pegasus/src/rdsn/src/utils/filesystem.cpp:618 #4 0x00007fc9085696e7 in dsn::replication::replica_bulk_loader::download_sst_file (this=0x5a947a40, remote_dir=..., local_dir=..., file_index=<optimized out>, fs=0x275f74d0) at /home/work/temp/pegasus/src/rdsn/src/replica/bulk_load/replica_bulk_loader.cpp:460 #5 0x00007fc9086b7811 in dsn::task::exec_internal ( this=this@entry=0xa95362d0) at /home/work/temp/pegasus/src/rdsn/src/runtime/task/task.cpp:176 #6 0x00007fc9086ccec2 in dsn::task_worker::loop (this=0x24e31e0) at /home/work/temp/pegasus/src/rdsn/src/runtime/task/task_worker.cpp:224 #7 0x00007fc9086cd040 in dsn::task_worker::run_internal (this=0x24e31e0) at /home/work/temp/pegasus/src/rdsn/src/runtime/task/task_worker.cpp:204 #8 0x00007fc90734ba1f in execute_native_thread_routine () from /home/work/app/pegasus/alsgsrv-monetization-master/replica/package/bin/libdsn_utils.so #9 0x00007fc905156dc5 in start_thread () from /lib64/libpthread.so.0 #10 0x00007fc90365573d in clone () from /lib64/libc.so.6 (gdb) quit ``` Type two: ``` #0 0x00007f8d106371d7 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-13.el7.x86_64 elfutils-libelf-0.166-2.el7.x86_64 elfutils-libs-0.166-2.el7.x86_64 glibc-2.17-157.el7_3.1.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.14.1-27.el7_3.x86_64 libattr-2.4.46-12.el7.x86_64 libcap-2.22-8.el7.x86_64 libcom_err-1.42.9-9.el7.x86_64 libgcc-4.8.5-28.el7_5.1.x86_64 libselinux-2.5-6.el7.x86_64 pcre-8.32-15.el7_2.1.x86_64 systemd-libs-219-30.el7_3.8.x86_64 xz-libs-5.2.2-1.el7.x86_64 zlib-1.2.7-17.el7.x86_64 (gdb) #0 0x00007f8d106371d7 in raise () from /lib64/libc.so.6 #1 0x00007f8d106388c8 in abort () from /lib64/libc.so.6 #2 0x00007f8d12435dbb in tcmalloc::Log (mode=mode@entry=tcmalloc::kCrash, filename=filename@entry=0x7f8d1244c14e "src/tcmalloc.cc", line=line@entry=332, a=..., b=..., c=..., d=...) at src/internal_logging.cc:118 #3 0x00007f8d124281d9 in (anonymous namespace)::InvalidFree ( ptr=<optimized out>) at src/tcmalloc.cc:332 #4 0x00007f8d16adaef3 in _dl_update_slotinfo () from /lib64/ld-linux-x86-64.so.2 #5 0x00007f8d16aca136 in update_get_addr () from /lib64/ld-linux-x86-64.so.2 #6 0x00007f8d1438e71c in dsn::utils::filesystem::get_normalized_path ( path=..., npath=...) at /home/work/temp/pegasus/src/rdsn/src/utils/filesystem.cpp:116 #7 0x00007f8d1438fb4d in dsn::utils::filesystem::path_combine (path1=..., path2=...) at /home/work/temp/pegasus/src/rdsn/src/utils/filesystem.cpp:618 #8 0x00007f8d1560d6e7 in dsn::replication::replica_bulk_loader::download_sst_file (this=0x35ba0c0, remote_dir=..., local_dir=..., file_index=<optimized out>, fs=0x8090b1500) at /home/work/temp/pegasus/src/rdsn/src/replica/bulk_load/replica_bulk_loader.cpp:460 #9 0x00007f8d1575b811 in dsn::task::exec_internal ( this=this@entry=0x8d3964f00) at /home/work/temp/pegasus/src/rdsn/src/runtime/task/task.cpp:176 #10 0x00007f8d15770ec2 in dsn::task_worker::loop (this=0x1f32fd0) at /home/work/temp/pegasus/src/rdsn/src/runtime/task/task_worker.cpp:224 #11 0x00007f8d15771040 in dsn::task_worker::run_internal (this=0x1f32fd0) at /home/work/temp/pegasus/src/rdsn/src/runtime/task/task_worker.cpp:204 #12 0x00007f8d143efa1f in execute_native_thread_routine () from /home/work/app/pegasus/alsgsrv-monetization-master/replica/package/bin/libdsn_utils.so #13 0x00007f8d121fadc5 in start_thread () from /lib64/libpthread.so.0 #14 0x00007f8d106f973d in clone () from /lib64/libc.so.6 (gdb) quit ``` Type three: ``` Program terminated with signal 6, Aborted. #0 0x00007fad388fa1d7 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install bzip2-libs-1.0.6-13.el7.x86_64 elfutils-libelf-0.166-2.el7.x86_64 elfutils-libs-0.166-2.el7.x86_64 glibc-2.17-157.el7_3.1.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.14.1-27.el7_3.x86_64 libattr-2.4.46-12.el7.x86_64 libcap-2.22-8.el7.x86_64 libcom_err-1.42.9-9.el7.x86_64 libgcc-4.8.5-28.el7_5.1.x86_64 libselinux-2.5-6.el7.x86_64 pcre-8.32-15.el7_2.1.x86_64 systemd-libs-219-30.el7_3.8.x86_64 xz-libs-5.2.2-1.el7.x86_64 zlib-1.2.7-17.el7.x86_64 (gdb) #0 0x00007fad388fa1d7 in raise () from /lib64/libc.so.6 #1 0x00007fad388fb8c8 in abort () from /lib64/libc.so.6 #2 0x00007fad3925ca8d in __gnu_cxx::__verbose_terminate_handler() () from /home/work/app/pegasus/alsgsrv-monetization-master/replica/package/bin/libstdc++.so.6 #3 0x00007fad3925abe6 in ?? () from /home/work/app/pegasus/alsgsrv-monetization-master/replica/package/bin/libstdc++.so.6 #4 0x00007fad3925ac13 in std::terminate() () from /home/work/app/pegasus/alsgsrv-monetization-master/replica/package/bin/libstdc++.so.6 #5 0x00007fad3c6b2a4e in execute_native_thread_routine () from /home/work/app/pegasus/alsgsrv-monetization-master/replica/package/bin/libdsn_utils.so #6 0x00007fad3a4bddc5 in start_thread () from /lib64/libpthread.so.0 #7 0x00007fad389bc73d in clone () from /lib64/libc.so.6 (gdb) quit ``` 3. What version of Pegasus are you using? v2.4 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
