caneGuy commented on issue #23826: [SPARK-26914][SQL] Fix scheduler pool may be unpredictable when we only want to use default pool and do not set spark.scheduler.pool for the session URL: https://github.com/apache/spark/pull/23826#issuecomment-476588012 It is because the local property has been updated by other query execution. @srowen For example: 1. user a submit query1 -> pool A query1 finished in pool A 2. user b submit query2 -> pool B query2 finished in pool B 3. user c submit query3 -> pool A but it get a thread inherited from which user B has used. Then it will get a wrong pool property.And submit query to pool B. https://jira.apache.org/jira/browse/SPARK-26914 here i post an example. And annotation here: `* These properties are inherited by child threads spawned from this thread. This * may have unexpected consequences when working with thread pools. The standard java * implementation of thread pools have worker threads spawn other worker threads. * As a result, local properties may propagate unpredictably.`
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
