FelixYBW commented on PR #13038: URL: https://github.com/apache/gluten/pull/13038#issuecomment-5789664053
I'm clear now. @rui-mo can you move your function of releaseQueuedAndWaitForRunning to https://github.com/apache/gluten/pull/11882 's HookedExecutor? Let's release the queue before waiting in ~VeloxRuntime instead of ~GlutenDirectBufferedInput. There are 2 states a function holding the memory pool in the IO thread queue when task exits: 1. Waiting, it will be released by this PR, then free the memory pool 2. Running, #11882 will wait for `spark.gluten.sql.columnar.backend.velox.asyncTimeoutOnTaskStopping (default 30s)` then report error. Since @boneanxs tested pr12919 and it works. So the root cause is that the function isn't scheduled in 30s, but not running more than 30. If the task queue is clear, we needn't wait until the IO thread exit. So both PR needed. @boneanxs can you test this PR again and see if it can solve your issue? We don't have the env to reproduce the issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
