mridulm commented on PR #40629: URL: https://github.com/apache/spark/pull/40629#issuecomment-1498430883
Why not special case TorrentBroadcast when there is a single chunk and inline it ? For the cases mentioned, this should come within 4mb - will prevent adding a new codepath, while also making all cases of broadcast faster, not just those using this specific api (which runs into issues if the broadcast value is not small) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
