[ https://issues.apache.org/jira/browse/SPARK-18405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15696978#comment-15696978 ]
Jeff Zhang edited comment on SPARK-18405 at 11/26/16 1:01 AM: -------------------------------------------------------------- I think he mean to launch multiple spark thrift server in yarn-cluster mode so each we can evenly distribute workload across them(e.g. one spark thrift server per department). And it is much easier to request large container for AM in yarn-cluster while the the memory of the host in yarn-client mode may be limited and without capacity control under yarn. was (Author: zjffdu): I think he mean to launch multiple spark thrift server in yarn-cluster mode so each we can evenly distribute workload across them. And it is much easier to request large container for AM in yarn-cluster while the the memory of the host in yarn-client mode may be limited and without capacity control under yarn. > Add yarn-cluster mode support to Spark Thrift Server > ---------------------------------------------------- > > Key: SPARK-18405 > URL: https://issues.apache.org/jira/browse/SPARK-18405 > Project: Spark > Issue Type: Improvement > Components: Spark Core > Affects Versions: 1.6.0, 1.6.2, 2.0.0, 2.0.1 > Reporter: Prabhu Kasinathan > Labels: Spark, ThriftServer2 > > Currently, spark thrift server can run only on yarn-client mode. > Can we add Yarn-Cluster mode support to spark thrift server? > This will help us to launch multiple spark thrift server with different spark > configurations and it really help in large distributed clusters where there > is requirement to run complex sqls through STS. With client mode, there is a > chance to overload local host with too much driver memory. > Please let me know your thoughts. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org