andygrove commented on PR #5613: URL: https://github.com/apache/datafusion-comet/pull/5613#issuecomment-5861060332
One more symptom of holding the fair pool's lock across the JNI call. While a fair-pool acquire is parked in Spark's `ExecutionMemoryPool.acquireMemory`, the executor's `comet-memory-usage-log` thread blocks in `Native.getMemoryUsage` on the same lock, because reading the pool's reservation takes it. So the periodic memory log goes quiet exactly when memory is contended. I saw it in a thread dump from a repro where the only Tokio worker was parked in that wait. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
