BryanCutler commented on issue #24070: [SPARK-23961][PYTHON] Fix error when toLocalIterator goes out of scope URL: https://github.com/apache/spark/pull/24070#issuecomment-478149067 Thanks @ueshin , I updated the code. I also made a change for the JVM to send a response to indicate if there are more partitions to read instead of relying on closing the socket and catching an error, which could have masked real errors. This change did not have any significant difference to the timings above. Please take another look when you can.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
