Probably. How are you setting the LIBPROCESS_PORT in Marathon? It has to be set via CommandInfo.Environment() of the task/executor for this to take effect.
On Fri, May 16, 2014 at 9:41 AM, Scott Clasen <[email protected]> wrote: > Aha, thanks! I am still having an issue. I am executing the process via > marathon, with LIBPROCESS_PORT set in the env. However when I log the > value of LIBPROCESS_PORT in my program, I get 0. > > Found a thread in this ML called > Re: Review Request: "Unset" LIBPROCESS_PORT before applying > executor/command environment variables (in the event they modify it). Is > this what is happening to me? > > > > On Thu, May 15, 2014 at 7:16 PM, Benjamin Mahler < > [email protected]> wrote: > >> You can set LIBPROCESS_PORT in the environment. >> >> >> On Wed, May 14, 2014 at 1:58 PM, Scott Clasen <[email protected]> wrote: >> >>> I raised this question on the Spark ML but it may be more a Mesos >>> question. >>> >>> I would like to be able to configure the port used to communicate >>> between the Mesos master and Spark tasks running across mesos slaves. It >>> appears that spark's usage of the mesos library must use the default port = >>> 0. >>> >>> At least that what I think happens in UPID. >>> https://github.com/apache/mesos/blob/master/3rdparty/libprocess/src/pid.cpp >>> >>> Here is an example... >>> >>> In this case if the port 56311 is not opened up via iptables and >>> security groups, the detecting new master step will hang indefinitely, and >>> failures will be logged on the master. >>> >>> group.cpp:310] Group process ((2)@1.2.3.4:56311) connected to ZooKeeper >>> group.cpp:752] Syncing group operations: queue size (joins, cancels, >>> datas) = (0, 0, 0) >>> group.cpp:367] Trying to create path '/mesos' in ZooKeeper >>> detector.cpp:134] Detected a new leader: (id='2') >>> group.cpp:629] Trying to get '/mesos/info_0000000002' in ZooKeeper >>> detector.cpp:351] A new leading master ([email protected]:5050) >>> is detected >>> sched.cpp:230] Detecting new master >>> >>> >>> Is there a way to tell the mesos native lib to use a specific port >>> rather than a random port from the spark side of things? >>> >>> >>> thanks >>> SC >>> >> >> >
