WeichenXu123 commented on PR #40695: URL: https://github.com/apache/spark/pull/40695#issuecomment-1501384634
> What is local mode Let me clarify it, TorchDistributer has "local mode" configure, if true, it just run torch program in client side, if False, it launches a spark job to run the torch program. So the TorchDistributer side "local mode" has nothing to do with spark master local mode. > why would you not support it on the client? I assume you mean to run torch program in spark connect client machine, we can support this, but I think it is less meaningful, because client machine should usually run lightweight workloads, but torch programs are heavy workloads and they often requires GPU, which client machine are hard to satisfy the condition. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
