squito commented on issue #24898: [SPARK-22340][PYTHON] Add a mode to pin Python thread into JVM's URL: https://github.com/apache/spark/pull/24898#issuecomment-529161542 Thanks for pointing me at this @HyukjinKwon , I agree its the same as the other issue. I do think pinning seems like the simplest approach, but I think we should spend some time to understand what's happening in py4j in that mode (eg., why isn't that just the default approach recommended by py4j). yes, we could probably also fix this the last way you're suggesting, with changes to py4j which let you pass through some thread-locals, but (a) I'd rather not be stuck on a py4j release and (b) seems like a questionable feature for py4j anyway, the api might be ugly.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
