Aha, thanks! I am still having an issue. I am executing the process via marathon, with LIBPROCESS_PORT set in the env. However when I log the value of LIBPROCESS_PORT in my program, I get 0.
Found a thread in this ML called Re: Review Request: "Unset" LIBPROCESS_PORT before applying executor/command environment variables (in the event they modify it).Is this what is happening to me? On Thu, May 15, 2014 at 7:16 PM, Benjamin Mahler <[email protected]>wrote: > You can set LIBPROCESS_PORT in the environment. > > > On Wed, May 14, 2014 at 1:58 PM, Scott Clasen <[email protected]> wrote: > >> I raised this question on the Spark ML but it may be more a Mesos >> question. >> >> I would like to be able to configure the port used to communicate between >> the Mesos master and Spark tasks running across mesos slaves. It appears >> that spark's usage of the mesos library must use the default port = 0. >> >> At least that what I think happens in UPID. >> https://github.com/apache/mesos/blob/master/3rdparty/libprocess/src/pid.cpp >> >> Here is an example... >> >> In this case if the port 56311 is not opened up via iptables and security >> groups, the detecting new master step will hang indefinitely, and failures >> will be logged on the master. >> >> group.cpp:310] Group process ((2)@1.2.3.4:56311) connected to ZooKeeper >> group.cpp:752] Syncing group operations: queue size (joins, cancels, >> datas) = (0, 0, 0) >> group.cpp:367] Trying to create path '/mesos' in ZooKeeper >> detector.cpp:134] Detected a new leader: (id='2') >> group.cpp:629] Trying to get '/mesos/info_0000000002' in ZooKeeper >> detector.cpp:351] A new leading master ([email protected]:5050) >> is detected >> sched.cpp:230] Detecting new master >> >> >> Is there a way to tell the mesos native lib to use a specific port rather >> than a random port from the spark side of things? >> >> >> thanks >> SC >> > >
