[
https://issues.apache.org/jira/browse/SPARK-17313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451428#comment-15451428
]
Sean Owen commented on SPARK-17313:
-----------------------------------
I don't think this is what the OP means. This is about 'yarn-client' vs
'yarn-cluster' mode in Spark. The shell has to run in yarn-client mode because
you're presenting an interactive shell to a user on machine X and so the driver
needs to run on machine X. Otherwise it's possible in yarn-cluster mode to run
the Spark job from machine X but have it start the driver within the cluster.
You can't use Zeppelin with 'yarn-cluster' mode right now either.
> Support spark-shell on cluster mode
> -----------------------------------
>
> Key: SPARK-17313
> URL: https://issues.apache.org/jira/browse/SPARK-17313
> Project: Spark
> Issue Type: New Feature
> Reporter: Mahmoud Elgamal
>
> The main issue with the current spark shell is that the driver is running on
> the user machine. If the driver resource requirement is beyond user machine
> capacity, then spark shell will be useless. If we are to add the cluster
> mode(Yarn or Mesos ) for spark shell via some sort of proxy where user
> machine only hosts a rest client to the running driver at the cluster, the
> shell will be more powerful
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
