If you want the most stable versions right now, I recommend Spark 0.3 (download it from the website or the 0.3-scala-2.9 tag in Git) and pre-protobuf Mesos. However, the current master Spark + master Mesos should also work OK; it's just that further commits may break things temporarily.
Matei On Aug 22, 2011, at 3:17 PM, Joe Xie wrote: > Hi Matei, > Thanks for your feedback. I bet I'm using the master brach of Spark and > pre-protobuf version of Mesos. Do you mean I should go for Spark (master) > + protobuf (master). Let's me know the most stable combination of braches > from Spark and pre-protobuf, I can try them out. > > Best, > Joe > > > On 8/22/11 3:09 PM, "Matei Zaharia" <[email protected]> wrote: > >> Are you using the master branch of Spark, or one of the releases (e.g. >> 0.3)? The releases work with the pre-protobuf version of Mesos but the >> master branch of Spark actually requires the master branch of Mesos now. >> That could cause the kind of error you're seeing. >> >> Matei >> >> On Aug 22, 2011, at 3:06 PM, Joe Xie wrote: >> >>> Sorry this should have been sent to user mail list. >>> >>> On 8/22/11 2:19 PM, "Joe Xie" <[email protected]> wrote: >>> >>>> Hi there: >>>> >>>> I has build and installed mesos (pre-protobuf branch for spark) and >>>> spark, >>>> and deployed them to a cluster. However, I have following error when I >>>> run >>>> spark task on the mesos URL of my master, the spark task runs fine in >>>> local mode. Any suggestion or help? Thanks a lot! >>>> Joe >>>> >>>> My os: CentOS 5.2 >>>> Python: 2.6 >>>> Gcc: 4.2.1 >>>> Swig:1.3.40 >>>> ./run spark.examples.SparkPi mesos://[email protected]:5050 1 >>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registered actor on >>>> port >>>> 50501 >>>> 11/08/22 13:31:11 INFO spark.MapOutputTrackerActor: Registered actor on >>>> port 50501 >>>> 11/08/22 13:31:11 INFO spark.SparkContext: Starting job... >>>> 11/08/22 13:31:11 INFO spark.CacheTracker: Registering RDD ID 0 with >>>> cache >>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Registering RDD 0 with >>>> 1 >>>> partitions >>>> 11/08/22 13:31:11 INFO spark.CacheTrackerActor: Asked for current cache >>>> locations >>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Final stage: Stage 0 >>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Parents of final stage: >>>> List() >>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Missing parents: List() >>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Submitting Stage 0, which >>>> has >>>> no missing parents >>>> 11/08/22 13:31:12 INFO spark.MesosScheduler: Got a job with 1 tasks >>>> Exception in thread "Spark scheduler" java.lang.UnsatisfiedLinkError: >>>> org.apache.mesos.MesosSchedulerDriver.initialize()V >>>> at org.apache.mesos.MesosSchedulerDriver.initialize(Native Method) >>>> at >>>> >>>> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:4 >>>> 2) >>>> at >>>> >>>> org.apache.mesos.MesosSchedulerDriver.<init>(MesosSchedulerDriver.java:4 >>>> 6) >>>> at spark.MesosScheduler$$anon$1.run(MesosScheduler.scala:94) >>>> >>>> Sent to wrong address previously >>>> >>>> >>>>> >>>>> >>>>> On 8/21/11 4:30 PM, "[email protected]" >>>>> <[email protected]> wrote: >>>>> >>>>>> Hi! This is the ezmlm program. I'm managing the >>>>>> [email protected] mailing list. >>>>>> >>>>>> I'm working for my owner, who can be reached >>>>>> at [email protected]. >>>>>> >>>>>> Acknowledgment: I have added the address >>>>>> >>>>>> [email protected] >>>>>> >>>>>> to the mesos-dev mailing list. >>>>>> >>>>>> Welcome to [email protected]! >>>>>> >>>>>> Please save this message so that you know the address you are >>>>>> subscribed under, in case you later want to unsubscribe or change >>>>>> your >>>>>> subscription address. >>>>>> >>>>>> >>>>>> --- Administrative commands for the mesos-dev list --- >>>>>> >>>>>> I can handle administrative requests automatically. Please >>>>>> do not send them to the list address! Instead, send >>>>>> your message to the correct command address: >>>>>> >>>>>> To subscribe to the list, send a message to: >>>>>> <[email protected]> >>>>>> >>>>>> To remove your address from the list, send a message to: >>>>>> <[email protected]> >>>>>> >>>>>> Send mail to the following for info and FAQ for this list: >>>>>> <[email protected]> >>>>>> <[email protected]> >>>>>> >>>>>> Similar addresses exist for the digest list: >>>>>> <[email protected]> >>>>>> <[email protected]> >>>>>> >>>>>> To get messages 123 through 145 (a maximum of 100 per request), mail: >>>>>> <[email protected]> >>>>>> >>>>>> To get an index with subject and author for messages 123-456 , mail: >>>>>> <[email protected]> >>>>>> >>>>>> They are always returned as sets of 100, max 2000 per request, >>>>>> so you'll actually get 100-499. >>>>>> >>>>>> To receive all messages with the same subject as message 12345, >>>>>> send a short message to: >>>>>> <[email protected]> >>>>>> >>>>>> The messages should contain one line or word of text to avoid being >>>>>> treated as sp@m, but I will ignore their content. >>>>>> Only the ADDRESS you send to is important. >>>>>> >>>>>> You can start a subscription for an alternate address, >>>>>> for example "[email protected]", just add a hyphen and your >>>>>> address (with '=' instead of '@') after the command word: >>>>>> <[email protected]> >>>>>> >>>>>> To stop subscription for this address, mail: >>>>>> <[email protected]> >>>>>> >>>>>> In both cases, I'll send a confirmation message to that address. When >>>>>> you receive it, simply reply to it to complete your subscription. >>>>>> >>>>>> If despite following these instructions, you do not get the >>>>>> desired results, please contact my owner at >>>>>> [email protected]. Please be patient, my owner is >>>>>> a >>>>>> lot slower than I am ;-) >>>>>> >>>>>> --- Enclosed is a copy of the request I received. >>>>>> >>>>>> Return-Path: <[email protected]> >>>>>> Received: (qmail 39962 invoked by uid 99); 21 Aug 2011 23:30:07 -0000 >>>>>> Received: from nike.apache.org (HELO nike.apache.org) >>>>>> (192.87.106.230) >>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:07 >>>>>> +0000 >>>>>> X-ASF-Spam-Status: No, hits=-0.0 required=5.0 >>>>>> tests=SPF_PASS >>>>>> X-Spam-Check-By: apache.org >>>>>> Received-SPF: pass (nike.apache.org: domain of [email protected] >>>>>> designates >>>>>> 67.231.145.42 as permitted sender) >>>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com) >>>>>> (67.231.145.42) >>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 23:30:00 >>>>>> +0000 >>>>>> Received: from pps.filterd (m0004348 [127.0.0.1]) >>>>>> by m0004348.ppops.net (8.14.4/8.14.4) with SMTP id p7LNTI6g015638 >>>>>> for >>>>>> >>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator >>>>>> .a >>>>>> p >>>>>> a >>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700 >>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net >>>>>> [66.220.144.38]) >>>>>> by m0004348.ppops.net with ESMTP id yb5cerc88-2 >>>>>> (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NOT) >>>>>> for >>>>>> >>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator >>>>>> .a >>>>>> p >>>>>> a >>>>>> che.org>; Sun, 21 Aug 2011 16:29:18 -0700 >>>>>> Received: from SC-MBX01-5.TheFacebook.com >>>>>> ([fe80::f05e:c7f1:c7dd:e640]) >>>>>> by >>>>>> sc-hub04.TheFacebook.com ([192.168.18.212]) with mapi id >>>>>> 14.01.0289.001; >>>>>> Sun, >>>>>> 21 Aug 2011 16:29:14 -0700 >>>>>> From: Joe Xie <[email protected]> >>>>>> To: >>>>>> >>>>>> "mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubator >>>>>> .a >>>>>> p >>>>>> a >>>>>> che.org" >>>>>> >>>>>> <mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=fb.com@incubato >>>>>> r. >>>>>> a >>>>>> p >>>>>> ache.org> >>>>>> Subject: Re: confirm subscribe to [email protected] >>>>>> Thread-Topic: confirm subscribe to [email protected] >>>>>> Thread-Index: AQHMYFO/ABVA64TXDkGEVkIetlsT6JUn85UA >>>>>> Date: Sun, 21 Aug 2011 23:29:13 +0000 >>>>>> Message-ID: <ca76e2a4.4259%[email protected]> >>>>>> In-Reply-To: <[email protected]> >>>>>> Accept-Language: en-US >>>>>> Content-Language: en-US >>>>>> X-MS-Has-Attach: >>>>>> X-MS-TNEF-Correlator: >>>>>> x-originating-ip: [192.168.18.252] >>>>>> Content-Type: text/plain; charset="us-ascii" >>>>>> Content-ID: <[email protected]> >>>>>> Content-Transfer-Encoding: quoted-printable >>>>>> MIME-Version: 1.0 >>>>>> X-Virus-Checked: Checked by ClamAV on apache.org >>>>>> >>>>>> Thanks! >>>>>> Joe >>>>>> On 8/21/11 3:43 PM, "[email protected]" >>>>>> <[email protected]> wrote: >>>>>> >>>>>>> Hi! This is the ezmlm program. I'm managing the >>>>>>> [email protected] mailing list. >>>>>>> >>>>>>> I'm working for my owner, who can be reached >>>>>>> at [email protected]. >>>>>>> >>>>>>> To confirm that you would like >>>>>>> >>>>>>> [email protected] >>>>>>> >>>>>>> added to the mesos-dev mailing list, please send >>>>>>> a short reply to this address: >>>>>>> >>>>>>> =20 >>>>>>> >>>>>>> mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com@incubat >>>>>>> or >>>>>>> . >>>>>>> a >>>>>>> p= >>>>>> ac >>>>>>> he.org >>>>>>> >>>>>>> Usually, this happens when you just hit the "reply" button. >>>>>>> If this does not work, simply copy the address and paste it into >>>>>>> the "To:" field of a new message. >>>>>>> >>>>>>> or click here: >>>>>>> >>>>>>> >>>>>>> mailto:mesos-dev-sc.1313966607.anahdnaofbfbnfgijcem-zhiyong=3Dfb.com >>>>>>> @i >>>>>>> n >>>>>>> c >>>>>>> u= >>>>>> ba >>>>>>> tor.apache.org >>>>>>> >>>>>>> This confirmation serves two purposes. First, it verifies that I am >>>>>>> able >>>>>>> to get mail through to you. Second, it protects you in case someone >>>>>>> forges a subscription request in your name. >>>>>>> >>>>>>> Please note that ALL Apache dev- and user- mailing lists are >>>>>>> publicly >>>>>>> archived. Do familiarize yourself with Apache's public archive >>>>>>> policy >>>>>>> at >>>>>>> >>>>>>> http://www.apache.org/foundation/public-archives.html >>>>>>> >>>>>>> prior to subscribing and posting messages to >>>>>>> [email protected]. >>>>>>> If you're not sure whether or not the policy applies to this mailing >>>>>>> list, >>>>>>> assume it does unless the list name contains the word "private" in >>>>>>> it. >>>>>>> >>>>>>> Some mail programs are broken and cannot handle long addresses. If >>>>>>> you >>>>>>> cannot reply to this request, instead send a message to >>>>>>> <[email protected]> and put the >>>>>>> entire address listed above into the "Subject:" line. >>>>>>> >>>>>>> >>>>>>> --- Administrative commands for the mesos-dev list --- >>>>>>> >>>>>>> I can handle administrative requests automatically. Please >>>>>>> do not send them to the list address! Instead, send >>>>>>> your message to the correct command address: >>>>>>> >>>>>>> To subscribe to the list, send a message to: >>>>>>> <[email protected]> >>>>>>> >>>>>>> To remove your address from the list, send a message to: >>>>>>> <[email protected]> >>>>>>> >>>>>>> Send mail to the following for info and FAQ for this list: >>>>>>> <[email protected]> >>>>>>> <[email protected]> >>>>>>> >>>>>>> Similar addresses exist for the digest list: >>>>>>> <[email protected]> >>>>>>> <[email protected]> >>>>>>> >>>>>>> To get messages 123 through 145 (a maximum of 100 per request), >>>>>>> mail: >>>>>>> <[email protected]> >>>>>>> >>>>>>> To get an index with subject and author for messages 123-456 , mail: >>>>>>> <[email protected]> >>>>>>> >>>>>>> They are always returned as sets of 100, max 2000 per request, >>>>>>> so you'll actually get 100-499. >>>>>>> >>>>>>> To receive all messages with the same subject as message 12345, >>>>>>> send a short message to: >>>>>>> <[email protected]> >>>>>>> >>>>>>> The messages should contain one line or word of text to avoid being >>>>>>> treated as sp@m, but I will ignore their content. >>>>>>> Only the ADDRESS you send to is important. >>>>>>> >>>>>>> You can start a subscription for an alternate address, >>>>>>> for example "[email protected]", just add a hyphen and your >>>>>>> address (with '=3D' instead of '@') after the command word: >>>>>>> <[email protected]> >>>>>>> >>>>>>> To stop subscription for this address, mail: >>>>>>> <[email protected]> >>>>>>> >>>>>>> In both cases, I'll send a confirmation message to that address. >>>>>>> When >>>>>>> you receive it, simply reply to it to complete your subscription. >>>>>>> >>>>>>> If despite following these instructions, you do not get the >>>>>>> desired results, please contact my owner at >>>>>>> [email protected]. Please be patient, my owner >>>>>>> is a >>>>>>> lot slower than I am ;-) >>>>>>> >>>>>>> --- Enclosed is a copy of the request I received. >>>>>>> >>>>>>> Return-Path: <[email protected]> >>>>>>> Received: (qmail 19986 invoked by uid 99); 21 Aug 2011 22:43:26 >>>>>>> -0000 >>>>>>> Received: from athena.apache.org (HELO athena.apache.org) >>>>>>> (140.211.11.136) >>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:26 >>>>>>> +0000 >>>>>>> X-ASF-Spam-Status: No, hits=3D0.2 required=3D10.0 >>>>>>> tests=3DASF_LIST_OPS,HTML_MESSAGE,SPF_PASS >>>>>>> X-Spam-Check-By: apache.org >>>>>>> Received-SPF: pass (athena.apache.org: domain of [email protected] >>>>>>> designates 67.231.145.42 as permitted sender) >>>>>>> Received: from [67.231.145.42] (HELO mx0a-00082601.pphosted.com) >>>>>>> (67.231.145.42) >>>>>>> by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 21 Aug 2011 22:43:19 >>>>>>> +0000 >>>>>>> Received: from pps.filterd (m0004347 [127.0.0.1]) >>>>>>> by m0004347.ppops.net (8.14.4/8.14.4) with SMTP id >>>>>>> p7LMcbSq017336 >>>>>>> for <[email protected]>; Sun, 21 Aug 2011 >>>>>>> 15:42:58 -0700 >>>>>>> Received: from mail.thefacebook.com (corpout1.snc1.tfbnw.net >>>>>>> [66.220.144.38]) >>>>>>> by m0004347.ppops.net with ESMTP id yb0jhrkt9-1 >>>>>>> (version=3DTLSv1/SSLv3 cipher=3DAES128-SHA bits=3D128 >>>>>>> verify=3DNOT) >>>>>>> for <[email protected]>; Sun, 21 Aug 2011 >>>>>>> 15:42:58 -0700 >>>>>>> Received: from SC-MBX01-5.TheFacebook.com >>>>>>> ([fe80::f05e:c7f1:c7dd:e640]) >>>>>>> by >>>>>>> sc-hub03.TheFacebook.com ([192.168.18.198]) with mapi id >>>>>>> 14.01.0289.001; >>>>>>> Sun, >>>>>>> 21 Aug 2011 15:42:53 -0700 >>>>>>> From: Joe Xie <[email protected]> >>>>>>> To: "[email protected]" >>>>>>> <[email protected]> >>>>>>> Subject: can i join this >>>>>>> Thread-Topic: can i join this >>>>>>> Thread-Index: AQHMYFOp36obbNljKE2n3mexQg9sQQ=3D=3D >>>>>>> Date: Sun, 21 Aug 2011 22:42:53 +0000 >>>>>>> Message-ID: <ca76d7f7.4256%[email protected]> >>>>>>> Accept-Language: en-US >>>>>>> Content-Language: en-US >>>>>>> X-MS-Has-Attach:=20 >>>>>>> X-MS-TNEF-Correlator: >>>>>>> x-originating-ip: [192.168.18.252] >>>>>>> Content-Type: multipart/alternative; >>>>>>> boundary=3D"_000_CA76D7F74256zhiyongfbcom_" >>>>>>> MIME-Version: 1.0 >>>>>>> >>>>>>> --_000_CA76D7F74256zhiyongfbcom_ >>>>>>> Content-Type: text/plain; charset=3D"us-ascii" >>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>> >>>>>>> Best, >>>>>>> Joe >>>>>>> >>>>>>> --_000_CA76D7F74256zhiyongfbcom_ >>>>>>> Content-Type: text/html; charset=3D"us-ascii" >>>>>>> Content-ID: <[email protected]> >>>>>>> Content-Transfer-Encoding: quoted-printable >>>>>>> >>>>>>> <html> >>>>>>> <head> >>>>>>> <meta http-equiv=3D3D"Content-Type" content=3D3D"text/html; >>>>>>> charset=3D3Dus-ascii"=3D >>>>>>>> >>>>>>> </head> >>>>>>> <body style=3D3D"word-wrap: break-word; -webkit-nbsp-mode: space; >>>>>>> -webkit-lin=3D >>>>>>> e-break: after-white-space; color: rgb(0, 0, 0); font-size: 14px; >>>>>>> font-fami=3D >>>>>>> ly: Calibri, sans-serif; "> >>>>>>> <div>Best,</div> >>>>>>> <div>Joe</div> >>>>>>> </body> >>>>>>> </html> >>>>>>> >>>>>>> --_000_CA76D7F74256zhiyongfbcom_-- >>>>>> >>>>> >>>> >>> >> >
