Are you using the master branch of Spark, or one of the releases (e.g. 0.3)? The releases work with the pre-protobuf version of Mesos but the master branch of Spark actually requires the master branch of Mesos now. That could cause the kind of error you're seeing.
Matei On Aug 22, 2011, at 3:06 PM, Joe Xie wrote: > Sorry this should have been sent to user mail list. > > On 8/22/11 2:19 PM, "Joe Xie" <[email protected]> wrote: > >> Hi there: >> >> I has build and installed mesos (pre-protobuf branch for spark) and spark, >> and deployed them to a cluster. However, I have following error when I run >> spark task on the mesos URL of my master, the spark task runs fine in >> local mode. Any suggestion or help? Thanks a lot! >> Joe >> >> My os: CentOS 5.2 >> Python: 2.6 >> Gcc: 4.2.1 >> Swig:1.3.40 >> ./run spark.examples.SparkPi mesos://[email protected]:5050 1 >> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered actor on port >> 50501 >> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered actor on >> port 50501 >> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job... >> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID 0 with cache >> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering RDD 0 with 1 >> partitions >> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for current cache >> locations >> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage: Stage 0 >> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final stage: >> List() >> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents: List() >> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage 0, which has >> no missing parents >> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1 tasks >> Exception in thread "Spark scheduler" java.lang.UnsatisfiedLinkError: >> org.apache.mesos.MesosSchedulerDriver.initialize()V >> at org.apache.mesos.MesosSchedulerDriver.initialize(Native Method) >> at >> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:42) >> at >> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:46) >> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94) >> >> Sent to wrong address previously >> >> >>> >>> >>> On 8/21/11 4:30 PM, "[email protected]" >>> <[email protected]> wrote: >>> >>>> Hi! This is the ezmlm program. I'm managing the >>>> [email protected] mailing list. >>>> >>>> I'm working for my owner, who can be reached >>>> at [email protected]. >>>> >>>> Acknowledgment: I have added the address >>>> >>>> [email protected] >>>> >>>> to the mesos-dev mailing list. >>>> >>>> Welcome to [email protected]! >>>> >>>> Please save this message so that you know the address you are >>>> subscribed under, in case you later want to unsubscribe or change your >>>> subscription address. >>>> >>>> >>>> --- Administrative commands for the mesos-dev list --- >>>> >>>> I can handle administrative requests automatically. Please >>>> do not send them to the list address! Instead, send >>>> your message to the correct command address: >>>> >>>> To subscribe to the list, send a message to: >>>> <[email protected]> >>>> >>>> To remove your address from the list, send a message to: >>>> <[email protected]> >>>> >>>> Send mail to the following for info and FAQ for this list: >>>> <[email protected]> >>>> <[email protected]> >>>> >>>> Similar addresses exist for the digest list: >>>> <[email protected]> >>>> <[email protected]> >>>> >>>> To get messages 123 through 145 (a maximum of 100 per request), mail: >>>> <[email protected]> >>>> >>>> To get an index with subject and author for messages 123-456 , mail: >>>> <[email protected]> >>>> >>>> They are always returned as sets of 100, max 2000 per request, >>>> so you'll actually get 100-499. >>>> >>>> To receive all messages with the same subject as message 12345, >>>> send a short message to: >>>> <[email protected]> >>>> >>>> The messages should contain one line or word of text to avoid being >>>> treated as sp@m, but I will ignore their content. >>>> Only the ADDRESS you send to is important. >>>> >>>> You can start a subscription for an alternate address, >>>> for example "[email protected]", just add a hyphen and your >>>> address (with '=' instead of '@') after the command word: >>>> <[email protected]> >>>> >>>> To stop subscription for this address, mail: >>>> <[email protected]> >>>> >>>> In both cases, I'll send a confirmation message to that address. When >>>> you receive it, simply reply to it to complete your subscription. >>>> >>>> If despite following these instructions, you do not get the >>>> desired results, please contact my owner at >>>> [email protected]. Please be patient, my owner is a >>>> lot slower than I am ;-) >>>> >>>> --- Enclosed is a copy of the request I received. >>>> >>>> Return-Path: <[email protected]> >>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011 23:30:07 -0000 >>>> Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) >>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:07 >>>> +0000 >>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0 >>>> tests=SPF_PASS >>>> X-Spam-Check-By: apache.org >>>> Received-SPF: pass (nike.apache.org: domain of [email protected] designates >>>> 67.231.145.42 as permitted sender) >>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com) >>>> (67.231.145.42) >>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:00 >>>> +0000 >>>> Received: from pps.filterd (m0004348 [127.0.0.1]) >>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id p7LNTI6g015638 >>>> for >>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.a >>>> p >>>> a >>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700 >>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net >>>> [66.220.144.38]) >>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2 >>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT) >>>> for >>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.a >>>> p >>>> a >>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700 >>>> Received: from SC-MBX01-5.TheFacebook.com ([fe80::f05e:c7f1:c7dd:e640]) >>>> by >>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id >>>> 14.01.0289.001; >>>> Sun, >>>> 21 Aug 2011 16:29:14 -0700 >>>> From: Joe Xie <[email protected]> >>>> To: >>>> "mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator.a >>>> p >>>> a >>>> che.org" >>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator. >>>> a >>>> p >>>> ache.org> >>>> Subject: Re: confirm subscribe to [email protected] >>>> Thread-Topic: confirm subscribe to [email protected] >>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA >>>> Date: Sun, 21 Aug 2011 23:29:13 +0000 >>>> Message-ID: <ca76e2a4.4259%[email protected]> >>>> In-Reply-To: <[email protected]> >>>> Accept-Language: en-US >>>> Content-Language: en-US >>>> X-MS-Has-Attach: >>>> X-MS-TNEF-Correlator: >>>> x-originating-ip: [192.168.18.252] >>>> Content-Type: text/plain; charset="us-ascii" >>>> Content-ID: <[email protected]> >>>> Content-Transfer-Encoding: quoted-printable >>>> MIME-Version: 1.0 >>>> X-Virus-Checked: Checked by ClamAV on apache.org >>>> >>>> Thanks! >>>> Joe >>>> On 8/21/11 3:43 PM, "[email protected]" >>>> <[email protected]> wrote: >>>> >>>>> Hi! This is the ezmlm program. I'm managing the >>>>> [email protected] mailing list. >>>>> >>>>> I'm working for my owner, who can be reached >>>>> at [email protected]. >>>>> >>>>> To confirm that you would like >>>>> >>>>> [email protected] >>>>> >>>>> added to the mesos-dev mailing list, please send >>>>> a short reply to this address: >>>>> >>>>> =20 >>>>> mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@incubator >>>>> . >>>>> a >>>>> p= >>>> ac >>>>> he.org >>>>> >>>>> Usually, this happens when you just hit the "reply" button. >>>>> If this does not work, simply copy the address and paste it into >>>>> the "To:" field of a new message. >>>>> >>>>> or click here: >>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@i >>>>> n >>>>> c >>>>> u= >>>> ba >>>>> tor.apache.org >>>>> >>>>> This confirmation serves two purposes. First, it verifies that I am >>>>> able >>>>> to get mail through to you. Second, it protects you in case someone >>>>> forges a subscription request in your name. >>>>> >>>>> Please note that ALL Apache dev- and user- mailing lists are publicly >>>>> archived. Do familiarize yourself with Apache's public archive policy >>>>> at >>>>> >>>>> http://www.apache.org/foundation/public-archives.html >>>>> >>>>> prior to subscribing and posting messages to >>>>> [email protected]. >>>>> If you're not sure whether or not the policy applies to this mailing >>>>> list, >>>>> assume it does unless the list name contains the word "private" in it. >>>>> >>>>> Some mail programs are broken and cannot handle long addresses. If you >>>>> cannot reply to this request, instead send a message to >>>>> <[email protected]> and put the >>>>> entire address listed above into the "Subject:" line. >>>>> >>>>> >>>>> --- Administrative commands for the mesos-dev list --- >>>>> >>>>> I can handle administrative requests automatically. Please >>>>> do not send them to the list address! Instead, send >>>>> your message to the correct command address: >>>>> >>>>> To subscribe to the list, send a message to: >>>>> <[email protected]> >>>>> >>>>> To remove your address from the list, send a message to: >>>>> <[email protected]> >>>>> >>>>> Send mail to the following for info and FAQ for this list: >>>>> <[email protected]> >>>>> <[email protected]> >>>>> >>>>> Similar addresses exist for the digest list: >>>>> <[email protected]> >>>>> <[email protected]> >>>>> >>>>> To get messages 123 through 145 (a maximum of 100 per request), mail: >>>>> <[email protected]> >>>>> >>>>> To get an index with subject and author for messages 123-456 , mail: >>>>> <[email protected]> >>>>> >>>>> They are always returned as sets of 100, max 2000 per request, >>>>> so you'll actually get 100-499. >>>>> >>>>> To receive all messages with the same subject as message 12345, >>>>> send a short message to: >>>>> <[email protected]> >>>>> >>>>> The messages should contain one line or word of text to avoid being >>>>> treated as sp@m, but I will ignore their content. >>>>> Only the ADDRESS you send to is important. >>>>> >>>>> You can start a subscription for an alternate address, >>>>> for example "[email protected]", just add a hyphen and your >>>>> address (with '=3D' instead of '@') after the command word: >>>>> <[email protected]> >>>>> >>>>> To stop subscription for this address, mail: >>>>> <[email protected]> >>>>> >>>>> In both cases, I'll send a confirmation message to that address. When >>>>> you receive it, simply reply to it to complete your subscription. >>>>> >>>>> If despite following these instructions, you do not get the >>>>> desired results, please contact my owner at >>>>> [email protected]. Please be patient, my owner is a >>>>> lot slower than I am ;-) >>>>> >>>>> --- Enclosed is a copy of the request I received. >>>>> >>>>> Return-Path: <[email protected]> >>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011 22:43:26 -0000 >>>>> Received: from athena.apache.org (HELO athena.apache.org) >>>>> (140.211.11.136) >>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:26 >>>>> +0000 >>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0 >>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS >>>>> X-Spam-Check-By: apache.org >>>>> Received-SPF: pass (athena.apache.org: domain of [email protected] >>>>> designates 67.231.145.42 as permitted sender) >>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com) >>>>> (67.231.145.42) >>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:19 >>>>> +0000 >>>>> Received: from pps.filterd (m0004347 [127.0.0.1]) >>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id p7LMcbSq017336 >>>>> for <[email protected]>; Sun, 21 Aug 2011 >>>>> 15:42:58 -0700 >>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net >>>>> [66.220.144.38]) >>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1 >>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128 verify=3DNOT) >>>>> for <[email protected]>; Sun, 21 Aug 2011 >>>>> 15:42:58 -0700 >>>>> Received: from SC-MBX01-5.TheFacebook.com ([fe80::f05e:c7f1:c7dd:e640]) >>>>> by >>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id >>>>> 14.01.0289.001; >>>>> Sun, >>>>> 21 Aug 2011 15:42:53 -0700 >>>>> From: Joe Xie <[email protected]> >>>>> To: "[email protected]" >>>>> <[email protected]> >>>>> Subject: can i join this >>>>> Thread-Topic: can i join this >>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D >>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000 >>>>> Message-ID: <ca76d7f7.4256%[email protected]> >>>>> Accept-Language: en-US >>>>> Content-Language: en-US >>>>> X-MS-Has-Attach:=20 >>>>> X-MS-TNEF-Correlator: >>>>> x-originating-ip: [192.168.18.252] >>>>> Content-Type: multipart/alternative; >>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_" >>>>> MIME-Version: 1.0 >>>>> >>>>> --_000_CA76D7F74256zhiyongfbcom_ >>>>> Content-Type: text/plain; charset=3D"us-ascii" >>>>> Content-Transfer-Encoding: quoted-printable >>>>> >>>>> Best, >>>>> Joe >>>>> >>>>> --_000_CA76D7F74256zhiyongfbcom_ >>>>> Content-Type: text/html; charset=3D"us-ascii" >>>>> Content-ID: <[email protected]> >>>>> Content-Transfer-Encoding: quoted-printable >>>>> >>>>> <html> >>>>> <head> >>>>> <meta http-equiv=3D3D"Content-Type" content=3D3D"text/html; >>>>> charset=3D3Dus-ascii"=3D >>>>>> >>>>> </head> >>>>> <body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode: space; >>>>> -webkit-lin=3D >>>>> e-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; >>>>> font-fami=3D >>>>> ly: Calibri, sans-serif; "> >>>>> <div>Best,</div> >>>>> <div>Joe</div> >>>>> </body> >>>>> </html> >>>>> >>>>> --_000_CA76D7F74256zhiyongfbcom_-- >>>> >>> >> >
