Hi Helene,
Thanks for the report. In Spark 1.1, we use a special exit code to
indicate |SparkSubmit| fails because of class not found. But
unfortunately I chose a not so special exit code — 1… So whenever the
process exit with 1 as exit code, the |-Phive| error message is shown. A
PR that changes 1 to 101 has been merged to master, hopefully to reduce
potential exit code conflicts.
Cheng
On 9/25/14 5:44 PM, Hélène Delanoeye wrote:
Hi
We've just experienced an issue with the new Spark-1.1.0 and the
start-thriftserver.sh script.
We tried to launch start-thriftserver.sh with "--master yarn" option
and got the following error message :
/Failed to load Hive Thrift server main class
org.apache.spark.sql.hive.thriftserver.HiveThriftServer2.
You need to build Spark with -Phive. /
In fact Spark was built with -Phive option, but the real problem was
this one :
/Application appattempt_1411058337040_0118_000001 submitted by user
xxxxx to unknown queue: default/
So the solution was to specified the queue, and it works :
/opt/spark/sbin/start-thriftserver.sh --master yarn --queue spark-batch
Hope this could help, as the error message is not really clear (and
rather wrong).
Helene
--
Kelkoo
*Hélène Delanoeye *Software engineer / Search team
*E*helene.delano...@kelkoo.com <mailto:helene.delano...@kelkoo.com>
*Y!Messenger* kelkoohelened
*T* (+33) 4 56 09 07 57
*A* 6, rue des Méridiens 38130 Echirolles FRANCE
------------------------------------------------------------------------
Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 8, rue du Sentier 75002 Paris
425 093 069 RCS Paris
Ce message et les pièces jointes sont confidentiels et établis à
l'attention exclusive de leurs destinataires. Si vous n'êtes pas le
destinataire de ce message, merci de le détruire et d'en avertir
l'expéditeur.
