It may have to do with the akka heartbeat interval per SPARK-3923 - https://issues.apache.org/jira/plugins/servlet/mobile#issue/SPARK-3923 ?
On Tue, Feb 24, 2015 at 16:40 Xi Shen <davidshe...@gmail.com> wrote: > Hi Sean, > > I launched the spark-shell on the same machine as I started YARN service. > I don't think port will be an issue. > > I am new to spark. I checked the HDFS web UI and the YARN web UI. But I > don't know how to check the AM. Can you help? > > > Thanks, > David > > > On Tue, Feb 24, 2015 at 8:37 PM Sean Owen <so...@cloudera.com> wrote: > >> I don't think the build is at issue. The error suggests your App Master >> can't be contacted. Is there a network port issue? did the AM fail? >> >> On Tue, Feb 24, 2015 at 9:15 AM, Xi Shen <davidshe...@gmail.com> wrote: >> >>> Hi Arush, >>> >>> I got the pre-build from https://spark.apache.org/downloads.html. When >>> I start spark-shell, it prompts: >>> >>> Spark assembly has been built with Hive, including Datanucleus jars >>> on classpath >>> >>> So we don't have pre-build with YARN support? If so, how the >>> spark-submit work? I checked the YARN log, and job is really submitted and >>> ran successfully. >>> >>> >>> Thanks, >>> David >>> >>> >>> >>> >>> >>> On Tue Feb 24 2015 at 6:35:38 PM Arush Kharbanda < >>> ar...@sigmoidanalytics.com> wrote: >>> >>>> Hi >>>> >>>> Are you sure that you built Spark for Yarn.If standalone works, not >>>> sure if its build for Yarn. >>>> >>>> Thanks >>>> Arush >>>> On Tue, Feb 24, 2015 at 12:06 PM, Xi Shen <davidshe...@gmail.com> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> I followed this guide, >>>>> http://spark.apache.org/docs/1.2.1/running-on-yarn.html, and tried to >>>>> start spark-shell with yarn-client >>>>> >>>>> ./bin/spark-shell --master yarn-client >>>>> >>>>> >>>>> But I got >>>>> >>>>> WARN ReliableDeliverySupervisor: Association with remote system >>>>> [akka.tcp://sparkYarnAM@10.0.2.15:38171] has failed, address is now gated >>>>> for [5000] ms. Reason is: [Disassociated]. >>>>> >>>>> In the spark-shell, and other exceptions in they yarn log. Please see >>>>> http://stackoverflow.com/questions/28671171/spark-shell-cannot-connect-to-yarn >>>>> for more detail. >>>>> >>>>> >>>>> However, submitting to the this cluster works. Also, spark-shell as >>>>> standalone works. >>>>> >>>>> >>>>> My system: >>>>> >>>>> - ubuntu amd64 >>>>> - spark 1.2.1 >>>>> - yarn from hadoop 2.6 stable >>>>> >>>>> >>>>> Thanks, >>>>> >>>>> [image: --] >>>>> Xi Shen >>>>> [image: http://]about.me/davidshen >>>>> <http://about.me/davidshen?promo=email_sig> >>>>> <http://about.me/davidshen> >>>>> >>>>> >>>> -- >>>> >>>> [image: Sigmoid Analytics] >>>> <http://htmlsig.com/www.sigmoidanalytics.com> >>>> >>>> *Arush Kharbanda* || Technical Teamlead >>>> >>>> ar...@sigmoidanalytics.com || www.sigmoidanalytics.com >>>> >>> >>
