srowen commented on pull request #33644: URL: https://github.com/apache/spark/pull/33644#issuecomment-894839816
I end up neutral on this - that the driver may not have enough memory doesn't seem quite like justification here, but possibly the argument about better aggregation on the executor (really?). I am concerned there is no quantified argument about overhead yet here. I wouldn't veto it if others feel strongly but might ask for a third (fourth?) opinion to be cautious -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
