msumit commented on issue #7935: URL: https://github.com/apache/airflow/issues/7935#issuecomment-690875882
All system vitals like the disk, cpu, and mem are absolutely fine whenever the stuck happens for us. Whenever the process stuck, it doesn't respond to any other kill signals except 9 & 11. I did a strace on the stuck process, it shows the following `futex(0x14d9390, FUTEX_WAIT_PRIVATE, 0, NULL` Then I killed the process with `kill -11` and loaded the core in gdb, and below is the stack trace > (gdb) bt #0 0x00007fe49b18b49b in raise () from /lib64/libpthread.so.0 #1 <signal handler called> #2 0x00007fe49b189adb in do_futex_wait.constprop.1 () from /lib64/libpthread.so.0 #3 0x00007fe49b189b6f in __new_sem_wait_slow.constprop.0 () from /lib64/libpthread.so.0 #4 0x00007fe49b189c0b in sem_wait@@GLIBC_2.2.5 () from /lib64/libpthread.so.0 #5 0x0000000000430bc5 in PyThread_acquire_lock_timed () #6 0x0000000000521a4c in acquire_timed () #7 0x0000000000521af6 in rlock_acquire () #8 0x00000000004826cd in _PyCFunction_FastCallDict () #9 0x00000000004f4143 in call_function () #10 0x00000000004f7971 in _PyEval_EvalFrameDefault () #11 0x00000000004f33c0 in _PyFunction_FastCall () #12 0x00000000004f40d6 in call_function () #13 0x00000000004f7971 in _PyEval_EvalFrameDefault () #14 0x00000000004f33c0 in _PyFunction_FastCall () #15 0x00000000004f40d6 in call_function () #16 0x00000000004f7971 in _PyEval_EvalFrameDefault () #17 0x00000000004f33c0 in _PyFunction_FastCall () #18 0x00000000004f40d6 in call_function () #19 0x00000000004f7971 in _PyEval_EvalFrameDefault () #20 0x00000000004f33c0 in _PyFunction_FastCall () #21 0x00000000004f40d6 in call_function () ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
