[
https://issues.apache.org/jira/browse/SPARK-17313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451797#comment-15451797
]
Sean Owen commented on SPARK-17313:
-----------------------------------
You don't have to run the driver on a laptop or other development machine. You
can run it on a cluster machine even in client mode, BTW, the same machine
you'll be running it on in any YARN container.
Supporting the shell in yarn-cluster mode is not the same as using Livy, but it
accomplishes something like a similar purpose. Why not use Livy? if that
addresses this use case, then I'd close this.
> Support spark-shell on cluster mode
> -----------------------------------
>
> Key: SPARK-17313
> URL: https://issues.apache.org/jira/browse/SPARK-17313
> Project: Spark
> Issue Type: New Feature
> Reporter: Mahmoud Elgamal
>
> The main issue with the current spark shell is that the driver is running on
> the user machine. If the driver resource requirement is beyond user machine
> capacity, then spark shell will be useless. If we are to add the cluster
> mode(Yarn or Mesos ) for spark shell via some sort of proxy where user
> machine only hosts a rest client to the running driver at the cluster, the
> shell will be more powerful
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
