Help us review #MesosCon proposals
A total of 24 proposals were submitted for #MesosConhttp://events.linuxfoundation.org/events/mesoscon, and as I previously indicated the program committee is opening them up for community review/feedback to better-inform our decisions about what should be included in the program. We've created the following form where community members can evaluate abstracts/titles: https://docs.google.com/forms/d/1hMgMNm-d5bJpFsOT_A_SJlsOG5Lk0HUzH54Ec3PzGSA/viewform. The form also includes an opportunity to indicate which sessions you didn't see proposed but would like to attend. Thanks in advance for your participation. We have a tight deadline for finalizing the program, so the initial evaluative results will be discussed during the program committee's meeting this Thursday. The form will close on 5/26, one week from today. Dave
Re: callback port
Can you set some other variable via the same mechanism and see if you can see it in your app? On Mon, May 19, 2014 at 10:54 AM, Scott Clasen sc...@heroku.com wrote: hmm, I am setting it via the Environment passed in the REST call to create the app, and if I look at the marathon UI I do see the var set. On Mon, May 19, 2014 at 10:19 AM, Vinod Kone vinodk...@gmail.com wrote: Probably. How are you setting the LIBPROCESS_PORT in Marathon? It has to be set via CommandInfo.Environment() of the task/executor for this to take effect. On Fri, May 16, 2014 at 9:41 AM, Scott Clasen sc...@heroku.com wrote: Aha, thanks! I am still having an issue. I am executing the process via marathon, with LIBPROCESS_PORT set in the env. However when I log the value of LIBPROCESS_PORT in my program, I get 0. Found a thread in this ML called Re: Review Request: Unset LIBPROCESS_PORT before applying executor/command environment variables (in the event they modify it). Is this what is happening to me? On Thu, May 15, 2014 at 7:16 PM, Benjamin Mahler benjamin.mah...@gmail.com wrote: You can set LIBPROCESS_PORT in the environment. On Wed, May 14, 2014 at 1:58 PM, Scott Clasen sc...@heroku.com wrote: I raised this question on the Spark ML but it may be more a Mesos question. I would like to be able to configure the port used to communicate between the Mesos master and Spark tasks running across mesos slaves. It appears that spark's usage of the mesos library must use the default port = 0. At least that what I think happens in UPID. https://github.com/apache/mesos/blob/master/3rdparty/libprocess/src/pid.cpp Here is an example... In this case if the port 56311 is not opened up via iptables and security groups, the detecting new master step will hang indefinitely, and failures will be logged on the master. group.cpp:310] Group process ((2)@1.2.3.4:56311) connected to ZooKeeper group.cpp:752] Syncing group operations: queue size (joins, cancels, datas) = (0, 0, 0) group.cpp:367] Trying to create path '/mesos' in ZooKeeper detector.cpp:134] Detected a new leader: (id='2') group.cpp:629] Trying to get '/mesos/info_02' in ZooKeeper detector.cpp:351] A new leading master (UPID=master@10.118.41.27:5050) is detected sched.cpp:230] Detecting new master Is there a way to tell the mesos native lib to use a specific port rather than a random port from the spark side of things? thanks SC
Re: callback port
Oh for sure, setting quite a few that I am seeing in the env. On Mon, May 19, 2014 at 12:05 PM, Vinod Kone vinodk...@gmail.com wrote: Can you set some other variable via the same mechanism and see if you can see it in your app? On Mon, May 19, 2014 at 10:54 AM, Scott Clasen sc...@heroku.com wrote: hmm, I am setting it via the Environment passed in the REST call to create the app, and if I look at the marathon UI I do see the var set. On Mon, May 19, 2014 at 10:19 AM, Vinod Kone vinodk...@gmail.com wrote: Probably. How are you setting the LIBPROCESS_PORT in Marathon? It has to be set via CommandInfo.Environment() of the task/executor for this to take effect. On Fri, May 16, 2014 at 9:41 AM, Scott Clasen sc...@heroku.com wrote: Aha, thanks! I am still having an issue. I am executing the process via marathon, with LIBPROCESS_PORT set in the env. However when I log the value of LIBPROCESS_PORT in my program, I get 0. Found a thread in this ML called Re: Review Request: Unset LIBPROCESS_PORT before applying executor/command environment variables (in the event they modify it). Is this what is happening to me? On Thu, May 15, 2014 at 7:16 PM, Benjamin Mahler benjamin.mah...@gmail.com wrote: You can set LIBPROCESS_PORT in the environment. On Wed, May 14, 2014 at 1:58 PM, Scott Clasen sc...@heroku.comwrote: I raised this question on the Spark ML but it may be more a Mesos question. I would like to be able to configure the port used to communicate between the Mesos master and Spark tasks running across mesos slaves. It appears that spark's usage of the mesos library must use the default port = 0. At least that what I think happens in UPID. https://github.com/apache/mesos/blob/master/3rdparty/libprocess/src/pid.cpp Here is an example... In this case if the port 56311 is not opened up via iptables and security groups, the detecting new master step will hang indefinitely, and failures will be logged on the master. group.cpp:310] Group process ((2)@1.2.3.4:56311) connected to ZooKeeper group.cpp:752] Syncing group operations: queue size (joins, cancels, datas) = (0, 0, 0) group.cpp:367] Trying to create path '/mesos' in ZooKeeper detector.cpp:134] Detected a new leader: (id='2') group.cpp:629] Trying to get '/mesos/info_02' in ZooKeeper detector.cpp:351] A new leading master (UPID=master@10.118.41.27:5050) is detected sched.cpp:230] Detecting new master Is there a way to tell the mesos native lib to use a specific port rather than a random port from the spark side of things? thanks SC
Re: [VOTE] Release Apache Mesos 0.18.2 (rc1)
Hi all, Thanks everyone for chiming in! The vote has passed with 3 binding and 5 non-binding votes. I will announce the release today. Niklas On Sat, May 17, 2014 at 11:07 PM, Chengwei Yang chengwei.yang...@gmail.comwrote: +1 'make distcheck' passed on RHEL 6.4, gcc 4.4.7 -- Thanks, Chengwei On Wed, May 14, 2014 at 12:06:44PM -0700, Niklas Nielsen wrote: Hi all, Please vote on releasing the following candidate as Apache Mesos 0.18.2. 0.18.2 includes the following: [MESOS-1313] - The executor bit is now essentially ignored with the 0.18.1 fetcher implementation The CHANGELOG for the release is available at: https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb =0.18.2-rc1 The candidate for Mesos 0.18.2 release is available at: https://dist.apache.org/repos/dist/dev/mesos/0.18.2-rc1/mesos-0.18.2.tar.gz The tag to be voted on is 0.18.2-rc1: https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=commit;h=0.18.2-rc1 The MD5 checksum of the tarball can be found at: https://dist.apache.org/repos/dist/dev/mesos/0.18.2-rc1/mesos-0.18.2.tar.gz.md5 The signature of the tarball can be found at: https://dist.apache.org/repos/dist/dev/mesos/0.18.2-rc1/mesos-0.18.2.tar.gz.asc The PGP key used to sign the release is here: https://dist.apache.org/repos/dist/release/mesos/KEYS The JAR is up in Maven in a staging repository here: https://repository.apache.org/content/repositories/orgapachemesos-1020 Please vote on releasing this package as Apache Mesos 0.18.2! The vote is open until Sat May 17 12:04:39 PDT 2014 and passes if a majority of at least 3 +1 PMC votes are cast. [ ] +1 Release this package as Apache Mesos 0.18.2 [ ] -1 Do not release this package because ... Thanks, Niklas -- Niklas
[RESULT][VOTE] Release Apache Mesos 0.18.2 (rc1)
Hi all, The vote for Mesos 0.18.2 (rc1) has passed with the following votes. +1 (Binding) -- Jie Yu Vinod Kone Till Toenshoff +1 (Non-binding) -- Iven Hsu Adam Bordelon Tom Arnfeld Tomas Barton Chengwei Yang There were no 0 or -1 votes. Please find the release at: https://dist.apache.org/repos/dist/release/mesos/0.18.2 It is recommended to use a mirror to download the release: http://www.apache.org/dyn/closer.cgi The CHANGELOG for the release is available at: https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.18.2 The mesos-0.18.2.jar has been released to: https://repository.apache.org The website (http://mesos.apache.org) will be updated shortly to reflect this release. Thanks, Niklas
Re: [RESULT][VOTE] Release Apache Mesos 0.18.2 (rc1)
Thanks Niklas - Updated Fedora channels in preparation: http://koji.fedoraproject.org/koji/buildinfo?buildID=516853 Cheers, Tim - Original Message - From: Niklas Nielsen n...@qni.dk To: user@mesos.apache.org, dev d...@mesos.apache.org Sent: Monday, May 19, 2014 3:57:19 PM Subject: [RESULT][VOTE] Release Apache Mesos 0.18.2 (rc1) Hi all, The vote for Mesos 0.18.2 (rc1) has passed with the following votes. +1 (Binding) -- Jie Yu Vinod Kone Till Toenshoff +1 (Non-binding) -- Iven Hsu Adam Bordelon Tom Arnfeld Tomas Barton Chengwei Yang There were no 0 or -1 votes. Please find the release at: https://dist.apache.org/repos/dist/release/mesos/0.18.2 It is recommended to use a mirror to download the release: http://www.apache.org/dyn/closer.cgi The CHANGELOG for the release is available at: https://git-wip-us.apache.org/repos/asf?p=mesos.git;a=blob_plain;f=CHANGELOG;hb=0.18.2 The mesos-0.18.2.jar has been released to: https://repository.apache.org The website (http://mesos.apache.org) will be updated shortly to reflect this release. Thanks, Niklas -- Cheers, Tim Freedom, Features, Friends, First - Fedora https://fedoraproject.org/wiki/SIGs/bigdata
