HyukjinKwon removed a comment on issue #24946: [SPARK-27234][SS][PYTHON] Use InheritableThreadLocal for current epoch in EpochTracker (to support Python UDFs) URL: https://github.com/apache/spark/pull/24946#issuecomment-514012160 Thanks for review and approval, @zsxwing. BTW, I understand the general concern (I assume you have this concern like others and me) - the existing way (using thread local to keep such status itself) isn't quite reliable. Similar discussion is going on in https://github.com/apache/spark/pull/24958. I believe this code path still has the same problem I described there https://github.com/apache/spark/pull/24958#issuecomment-511364075 (with and without this fix - it's orthogonal). Maybe we should completely switch the approach we're doing here but I believe can be orthogonally done. Hope this alleviates your concern.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
