holdenk commented on issue #24898: [SPARK-22340][PYTHON] Add a mode to pin Python thread into JVM's URL: https://github.com/apache/spark/pull/24898#issuecomment-511901052 I have some general questions about the design/goals: 1) Do we think there might be any problems with blocking around how we do streaming in Python? 2) When you say "experimental" do you mean experimental in PySpark or experimental in Py4J as well? 3) Is this only to support the Job Group or is there other benefits that you think we would get from this? (Because if it's just fixing the job group ID we might be able to find a simpler way to track that information in the Python thread and pass it through each time)?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
