viirya opened a new issue, #7041: URL: https://github.com/apache/datafusion-comet/issues/7041
### Describe the bug The panic hook installed by `errors::init()` (`native/jni-bridge/src/errors.rs`) stores the backtrace of every native panic in one process-wide slot, `PANIC_BACKTRACE: Arc<Mutex<Option<String>>>`. When a panic reaches a JNI entry point, `NativeFailure::from` takes whatever the slot holds and formats it into the message of the `CometNativeException`. Spark runs many tasks on one executor, so two threads can panic at about the same time. If thread B's panic hook runs between thread A's hook and thread A taking the slot, A's exception carries B's backtrace, and B's exception gets no backtrace at all. The message of each exception is still its own, so the stack trace points at code that did not fail. ### Steps to reproduce It is timing-dependent, so it shows up as an occasional wrong stack trace rather than on demand. In a Rust test it can be forced: two threads each panic inside `catch_native`, and a value that thread A drops while it unwinds (after its panic hook, before `catch_native` takes the slot) waits until thread B's panic hook has run. A's `NativeFailure` then holds B's backtrace, or none if B has already taken the slot. ### Expected behavior Each `CometNativeException` from a panic carries the backtrace of the panic that caused it. The panic hook and the `catch_unwind` at the JNI boundary run on the same thread, so the backtrace can be kept per thread instead of in a shared slot. ### Additional context Found while working on #6434. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
