Hi Matei, Thanks for your feedback. I bet I'm using the master brach of Spark and pre-protobuf version of Mesos. Do you mean I should go for Spark (master) + protobuf (master). Let's me know the most stable combination of braches from Spark and pre-protobuf, I can try them out.
Best, Joe On 8/22/11 3:09 PM, "Matei Zaharia" <[email protected]> wrote: >Are you using the master branch of Spark, or one of the releases (e.g. >0.3)? The releases work with the pre-protobuf version of Mesos but the >master branch of Spark actually requires the master branch of Mesos now. >That could cause the kind of error you're seeing. > >Matei > >On Aug 22, 2011, at 3:06 PM, Joe Xie wrote: > >> Sorry this should have been sent to user mail list. >> >> On 8/22/11 2:19 PM, "Joe Xie" <[email protected]> wrote: >> >>> Hi there: >>> >>> I has build and installed mesos (pre-protobuf branch for spark) and >>>spark, >>> and deployed them to a cluster. However, I have following error when I >>>run >>> spark task on the mesos URL of my master, the spark task runs fine in >>> local mode. Any suggestion or help? Thanks a lot! >>> Joe >>> >>> My os: CentOS 5.2 >>> Python: 2.6 >>> Gcc: 4.2.1 >>> Swig:1.3.40 >>> ./run spark.examples.SparkPi mesos://[email protected]:5050 1 >>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered actor on >>>port >>> 50501 >>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered actor on >>> port 50501 >>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job... >>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID 0 with >>>cache >>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering RDD 0 with >>>1 >>> partitions >>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for current cache >>> locations >>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage: Stage 0 >>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final stage: >>> List() >>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents: List() >>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage 0, which >>>has >>> no missing parents >>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1 tasks >>> Exception in thread "Spark scheduler" java.lang.UnsatisfiedLinkError: >>> org.apache.mesos.MesosSchedulerDriver.initialize()V >>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native Method) >>> at >>> >>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:4 >>>2) >>> at >>> >>>org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:4 >>>6) >>> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94) >>> >>> Sent to wrong address previously >>> >>> >>>> >>>> >>>> On 8/21/11 4:30 PM, "[email protected]" >>>> <[email protected]> wrote: >>>> >>>>> Hi! This is the ezmlm program. I'm managing the >>>>> [email protected] mailing list. >>>>> >>>>> I'm working for my owner, who can be reached >>>>> at [email protected]. >>>>> >>>>> Acknowledgment: I have added the address >>>>> >>>>> [email protected] >>>>> >>>>> to the mesos-dev mailing list. >>>>> >>>>> Welcome to [email protected]! >>>>> >>>>> Please save this message so that you know the address you are >>>>> subscribed under, in case you later want to unsubscribe or change >>>>>your >>>>> subscription address. >>>>> >>>>> >>>>> --- Administrative commands for the mesos-dev list --- >>>>> >>>>> I can handle administrative requests automatically. Please >>>>> do not send them to the list address! Instead, send >>>>> your message to the correct command address: >>>>> >>>>> To subscribe to the list, send a message to: >>>>> <[email protected]> >>>>> >>>>> To remove your address from the list, send a message to: >>>>> <[email protected]> >>>>> >>>>> Send mail to the following for info and FAQ for this list: >>>>> <[email protected]> >>>>> <[email protected]> >>>>> >>>>> Similar addresses exist for the digest list: >>>>> <[email protected]> >>>>> <[email protected]> >>>>> >>>>> To get messages 123 through 145 (a maximum of 100 per request), mail: >>>>> <[email protected]> >>>>> >>>>> To get an index with subject and author for messages 123-456 , mail: >>>>> <[email protected]> >>>>> >>>>> They are always returned as sets of 100, max 2000 per request, >>>>> so you'll actually get 100-499. >>>>> >>>>> To receive all messages with the same subject as message 12345, >>>>> send a short message to: >>>>> <[email protected]> >>>>> >>>>> The messages should contain one line or word of text to avoid being >>>>> treated as sp@m, but I will ignore their content. >>>>> Only the ADDRESS you send to is important. >>>>> >>>>> You can start a subscription for an alternate address, >>>>> for example "[email protected]", just add a hyphen and your >>>>> address (with '=' instead of '@') after the command word: >>>>> <[email protected]> >>>>> >>>>> To stop subscription for this address, mail: >>>>> <[email protected]> >>>>> >>>>> In both cases, I'll send a confirmation message to that address. When >>>>> you receive it, simply reply to it to complete your subscription. >>>>> >>>>> If despite following these instructions, you do not get the >>>>> desired results, please contact my owner at >>>>> [email protected]. Please be patient, my owner is >>>>>a >>>>> lot slower than I am ;-) >>>>> >>>>> --- Enclosed is a copy of the request I received. >>>>> >>>>> Return-Path: <[email protected]> >>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011 23:30:07 -0000 >>>>> Received: from nike.apache.org (HELO nike.apache.org) >>>>>(192.87.106.230) >>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:07 >>>>> +0000 >>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0 >>>>> tests=SPF_PASS >>>>> X-Spam-Check-By: apache.org >>>>> Received-SPF: pass (nike.apache.org: domain of [email protected] >>>>>designates >>>>> 67.231.145.42 as permitted sender) >>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com) >>>>> (67.231.145.42) >>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:00 >>>>> +0000 >>>>> Received: from pps.filterd (m0004348 [127.0.0.1]) >>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id p7LNTI6g015638 >>>>> for >>>>> >>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator >>>>>.a >>>>> p >>>>> a >>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700 >>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net >>>>> [66.220.144.38]) >>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2 >>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT) >>>>> for >>>>> >>>>><mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator >>>>>.a >>>>> p >>>>> a >>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700 >>>>> Received: from SC-MBX01-5.TheFacebook.com >>>>>([fe80::f05e:c7f1:c7dd:e640]) >>>>> by >>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id >>>>> 14.01.0289.001; >>>>> Sun, >>>>> 21 Aug 2011 16:29:14 -0700 >>>>> From: Joe Xie <[email protected]> >>>>> To: >>>>> >>>>>"mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator >>>>>.a >>>>> p >>>>> a >>>>> che.org" >>>>> >>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubato >>>>>r. >>>>> a >>>>> p >>>>> ache.org> >>>>> Subject: Re: confirm subscribe to [email protected] >>>>> Thread-Topic: confirm subscribe to [email protected] >>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA >>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000 >>>>> Message-ID: <ca76e2a4.4259%[email protected]> >>>>> In-Reply-To: <[email protected]> >>>>> Accept-Language: en-US >>>>> Content-Language: en-US >>>>> X-MS-Has-Attach: >>>>> X-MS-TNEF-Correlator: >>>>> x-originating-ip: [192.168.18.252] >>>>> Content-Type: text/plain; charset="us-ascii" >>>>> Content-ID: <[email protected]> >>>>> Content-Transfer-Encoding: quoted-printable >>>>> MIME-Version: 1.0 >>>>> X-Virus-Checked: Checked by ClamAV on apache.org >>>>> >>>>> Thanks! >>>>> Joe >>>>> On 8/21/11 3:43 PM, "[email protected]" >>>>> <[email protected]> wrote: >>>>> >>>>>> Hi! This is the ezmlm program. I'm managing the >>>>>> [email protected] mailing list. >>>>>> >>>>>> I'm working for my owner, who can be reached >>>>>> at [email protected]. >>>>>> >>>>>> To confirm that you would like >>>>>> >>>>>> [email protected] >>>>>> >>>>>> added to the mesos-dev mailing list, please send >>>>>> a short reply to this address: >>>>>> >>>>>> =20 >>>>>> >>>>>>mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@incubat >>>>>>or >>>>>> . >>>>>> a >>>>>> p= >>>>> ac >>>>>> he.org >>>>>> >>>>>> Usually, this happens when you just hit the "reply" button. >>>>>> If this does not work, simply copy the address and paste it into >>>>>> the "To:" field of a new message. >>>>>> >>>>>> or click here: >>>>>> >>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com >>>>>>@i >>>>>> n >>>>>> c >>>>>> u= >>>>> ba >>>>>> tor.apache.org >>>>>> >>>>>> This confirmation serves two purposes. First, it verifies that I am >>>>>> able >>>>>> to get mail through to you. Second, it protects you in case someone >>>>>> forges a subscription request in your name. >>>>>> >>>>>> Please note that ALL Apache dev- and user- mailing lists are >>>>>>publicly >>>>>> archived. Do familiarize yourself with Apache's public archive >>>>>>policy >>>>>> at >>>>>> >>>>>> http://www.apache.org/foundation/public-archives.html >>>>>> >>>>>> prior to subscribing and posting messages to >>>>>> [email protected]. >>>>>> If you're not sure whether or not the policy applies to this mailing >>>>>> list, >>>>>> assume it does unless the list name contains the word "private" in >>>>>>it. >>>>>> >>>>>> Some mail programs are broken and cannot handle long addresses. If >>>>>>you >>>>>> cannot reply to this request, instead send a message to >>>>>> <[email protected]> and put the >>>>>> entire address listed above into the "Subject:" line. >>>>>> >>>>>> >>>>>> --- Administrative commands for the mesos-dev list --- >>>>>> >>>>>> I can handle administrative requests automatically. Please >>>>>> do not send them to the list address! Instead, send >>>>>> your message to the correct command address: >>>>>> >>>>>> To subscribe to the list, send a message to: >>>>>> <[email protected]> >>>>>> >>>>>> To remove your address from the list, send a message to: >>>>>> <[email protected]> >>>>>> >>>>>> Send mail to the following for info and FAQ for this list: >>>>>> <[email protected]> >>>>>> <[email protected]> >>>>>> >>>>>> Similar addresses exist for the digest list: >>>>>> <[email protected]> >>>>>> <[email protected]> >>>>>> >>>>>> To get messages 123 through 145 (a maximum of 100 per request), >>>>>>mail: >>>>>> <[email protected]> >>>>>> >>>>>> To get an index with subject and author for messages 123-456 , mail: >>>>>> <[email protected]> >>>>>> >>>>>> They are always returned as sets of 100, max 2000 per request, >>>>>> so you'll actually get 100-499. >>>>>> >>>>>> To receive all messages with the same subject as message 12345, >>>>>> send a short message to: >>>>>> <[email protected]> >>>>>> >>>>>> The messages should contain one line or word of text to avoid being >>>>>> treated as sp@m, but I will ignore their content. >>>>>> Only the ADDRESS you send to is important. >>>>>> >>>>>> You can start a subscription for an alternate address, >>>>>> for example "[email protected]", just add a hyphen and your >>>>>> address (with '=3D' instead of '@') after the command word: >>>>>> <[email protected]> >>>>>> >>>>>> To stop subscription for this address, mail: >>>>>> <[email protected]> >>>>>> >>>>>> In both cases, I'll send a confirmation message to that address. >>>>>>When >>>>>> you receive it, simply reply to it to complete your subscription. >>>>>> >>>>>> If despite following these instructions, you do not get the >>>>>> desired results, please contact my owner at >>>>>> [email protected]. Please be patient, my owner >>>>>>is a >>>>>> lot slower than I am ;-) >>>>>> >>>>>> --- Enclosed is a copy of the request I received. >>>>>> >>>>>> Return-Path: <[email protected]> >>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011 22:43:26 >>>>>>-0000 >>>>>> Received: from athena.apache.org (HELO athena.apache.org) >>>>>> (140.211.11.136) >>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:26 >>>>>> +0000 >>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0 >>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS >>>>>> X-Spam-Check-By: apache.org >>>>>> Received-SPF: pass (athena.apache.org: domain of [email protected] >>>>>> designates 67.231.145.42 as permitted sender) >>>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com) >>>>>> (67.231.145.42) >>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:19 >>>>>> +0000 >>>>>> Received: from pps.filterd (m0004347 [127.0.0.1]) >>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id p7LMcbSq017336 >>>>>> for <[email protected]>; Sun, 21 Aug 2011 >>>>>> 15:42:58 -0700 >>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net >>>>>> [66.220.144.38]) >>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1 >>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128 verify=3DNOT) >>>>>> for <[email protected]>; Sun, 21 Aug 2011 >>>>>> 15:42:58 -0700 >>>>>> Received: from SC-MBX01-5.TheFacebook.com >>>>>>([fe80::f05e:c7f1:c7dd:e640]) >>>>>> by >>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id >>>>>> 14.01.0289.001; >>>>>> Sun, >>>>>> 21 Aug 2011 15:42:53 -0700 >>>>>> From: Joe Xie <[email protected]> >>>>>> To: "[email protected]" >>>>>> <[email protected]> >>>>>> Subject: can i join this >>>>>> Thread-Topic: can i join this >>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D >>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000 >>>>>> Message-ID: <ca76d7f7.4256%[email protected]> >>>>>> Accept-Language: en-US >>>>>> Content-Language: en-US >>>>>> X-MS-Has-Attach:=20 >>>>>> X-MS-TNEF-Correlator: >>>>>> x-originating-ip: [192.168.18.252] >>>>>> Content-Type: multipart/alternative; >>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_" >>>>>> MIME-Version: 1.0 >>>>>> >>>>>> --_000_CA76D7F74256zhiyongfbcom_ >>>>>> Content-Type: text/plain; charset=3D"us-ascii" >>>>>> Content-Transfer-Encoding: quoted-printable >>>>>> >>>>>> Best, >>>>>> Joe >>>>>> >>>>>> --_000_CA76D7F74256zhiyongfbcom_ >>>>>> Content-Type: text/html; charset=3D"us-ascii" >>>>>> Content-ID: <[email protected]> >>>>>> Content-Transfer-Encoding: quoted-printable >>>>>> >>>>>> <html> >>>>>> <head> >>>>>> <meta http-equiv=3D3D"Content-Type" content=3D3D"text/html; >>>>>> charset=3D3Dus-ascii"=3D >>>>>>> >>>>>> </head> >>>>>> <body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode: space; >>>>>> -webkit-lin=3D >>>>>> e-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; >>>>>> font-fami=3D >>>>>> ly: Calibri, sans-serif; "> >>>>>> <div>Best,</div> >>>>>> <div>Joe</div> >>>>>> </body> >>>>>> </html> >>>>>> >>>>>> --_000_CA76D7F74256zhiyongfbcom_-- >>>>> >>>> >>> >> >
