JoshRosen commented on PR #37413: URL: https://github.com/apache/spark/pull/37413#issuecomment-1206762340
@mridulm, it doesn't break local mode because there's a carve-out to preserve the existing behavior in that case: in both places where the `if(serializedOnly` check changes behavior, there's a check for `isLocalMaster` to avoid behavior changes: We'll still store the original object in the driver block manager at write time in local mode: https://github.com/apache/spark/blob/75ab18ee0e382b8117bf65fc9ef05190d4fdf01a/core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala#L133-L136 There's a similar carve-out in `readBroadcastBlock` (although I don't think we'd ever actually hit that branch in local mode given that we would have already stored the re-assembled broadcast block in `writeBlocks`): https://github.com/apache/spark/blob/75ab18ee0e382b8117bf65fc9ef05190d4fdf01a/core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala#L277-L284 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
