[ https://issues.apache.org/jira/browse/SPARK-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14279869#comment-14279869 ]
Patrick Wendell edited comment on SPARK-5176 at 1/16/15 6:28 AM: ----------------------------------------------------------------- Yes, we should add a check here similar to the existing ones for the thriftserver class: https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L143 [~tpanning] are you interested in contributing this? If not, someone else will pick it up. was (Author: pwendell): Yes, we should add a check here similar to the existing ones for the thriftserver class: https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala#L143 > Thrift server fails with confusing error message when deploy-mode is cluster > ---------------------------------------------------------------------------- > > Key: SPARK-5176 > URL: https://issues.apache.org/jira/browse/SPARK-5176 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.1.0, 1.2.0 > Reporter: Tom Panning > Labels: starter > > With Spark 1.2.0, when I try to run > {noformat} > $SPARK_HOME/sbin/start-thriftserver.sh --deploy-mode cluster --master > spark://xd-spark.xdata.data-tactics-corp.com:7077 > {noformat} > The log output is > {noformat} > Spark assembly has been built with Hive, including Datanucleus jars on > classpath > Spark Command: /usr/java/latest/bin/java -cp > ::/home/tpanning/Projects/spark/spark-1.2.0-bin-hadoop2.4/sbin/../conf:/home/tpanning/Projects/spark/spark-1.2.0-bin-hadoop2.4/lib/spark-assembly-1.2.0-hadoop2.4.0.jar:/home/tpanning/Projects/spark/spark-1.2.0-bin-hadoop2.4/lib/datanucleus-core-3.2.10.jar:/home/tpanning/Projects/spark/spark-1.2.0-bin-hadoop2.4/lib/datanucleus-rdbms-3.2.9.jar:/home/tpanning/Projects/spark/spark-1.2.0-bin-hadoop2.4/lib/datanucleus-api-jdo-3.2.6.jar > -XX:MaxPermSize=128m -Xms512m -Xmx512m org.apache.spark.deploy.SparkSubmit > --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 > --deploy-mode cluster --master > spark://xd-spark.xdata.data-tactics-corp.com:7077 spark-internal > ======================================== > Jar url 'spark-internal' is not in valid format. > Must be a jar file path in URL format (e.g. hdfs://host:port/XX.jar, > file:///XX.jar) > Usage: DriverClient [options] launch <active-master> <jar-url> <main-class> > [driver options] > Usage: DriverClient kill <active-master> <driver-id> > Options: > -c CORES, --cores CORES Number of cores to request (default: 1) > -m MEMORY, --memory MEMORY Megabytes of memory to request (default: > 512) > -s, --supervise Whether to restart the driver on failure > -v, --verbose Print more debugging output > > Using Spark's default log4j profile: > org/apache/spark/log4j-defaults.properties > {noformat} > I do not get this error if deploy-mode is set to client. The --deploy-mode > option is described by the --help output, so I expected it to work. I > checked, and this behavior seems to be present in Spark 1.1.0 as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org