GitHub user seanmuth edited a comment on the discussion: Proposal: supervisor-level liveness enforcement for tasks with blocking native calls (was: JVM process isolation)
Fair correction, and you're right, I overreached there. Most well-behaved C extensions do release the GIL around blocking calls, that's the standard pattern for exactly the reason you're describing, so pymongo and paramiko/cryptography aren't in the same boat as JPype by default. JPype specifically is the outlier here, I checked their native source directly, there's no GIL-release pattern anywhere in it, and their own tracker has a GIL-deadlock issue where the maintainer describes their threading model as "hopelessly opaque." Your second point is the one that actually fixes my framing, though. The real justification for a supervisor-level check isn't "the GIL problem generalizes to everything," it's that alive-but-not-progressing is a broader failure class than any single mechanism, a cross-thread deadlock, an infinite loop, a GIL-cooperative I/O call whose remote end went silent, all of these are just as invisible to a supervisor that only checks process liveness as JPype's case is, they just get there differently. JPype's GIL behavior is the most severe instance, since it also rules out a same-process mitigation, but it's not the general argument for why this belongs at the supervisor layer. Concretely, we see this with some regularity in customer deployments as SSL/TLS handshake hangs, a well-behaved, GIL-releasing library, but the handshake itself just never completes if the remote end goes quiet mid-negotiation. There are timeout safeguards for most of that today, but they're per-call and per-library, so coverage is uneven, and a supervisor-level backstop would catch what slips through regardless of which library or mechanism is involved. That's a better foundation for the proposal than what I wrote, thanks for the catch. 🤖 Drafted with Claude Code (Claude Sonnet 5) GitHub link: https://github.com/apache/airflow/discussions/70055#discussioncomment-17706366 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
