[
https://issues.apache.org/jira/browse/MESOS-183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13266525#comment-13266525
]
[email protected] commented on MESOS-183:
-----------------------------------------------------
bq. On 2012-04-24 21:45:19, Benjamin Hindman wrote:
bq. > frameworks/mpi/nmpiexec.py, line 270
bq. > <https://reviews.apache.org/r/4768/diff/3/?file=103693#file103693line270>
bq. >
bq. > Did someone actually ask for this feature?
No, but I thought it wouldn't hurt...
bq. On 2012-04-24 21:45:19, Benjamin Hindman wrote:
bq. > frameworks/mpi/nmpiexec.py, line 253
bq. > <https://reviews.apache.org/r/4768/diff/3/?file=103693#file103693line253>
bq. >
bq. > I'd like to make this whole thing simpler. In particular, I don't
see any need for an executor here (i.e., the startmpd.py script). We actually
have a 1-1 mapping from tasks to mpd's, so let's just have our TaskInfo's have
a CommandInfo which just launches the mpd. Something like this for that
CommandInfo's value:
bq. >
bq. > ...command.value = "${MPICH2PATH}mpd -n --host=${MPD_MASTER_IP}
--port=${MPD_MASTER_PORT}"
bq. >
bq. > (Note your code, as does what I have above, assumes that MPICH2PATH
includes the trailing '/'. Also, you might need to change the string based on
if you're passing --ifhn. Finally, note I used the long options --host and
--port. This is for readability for other people that might not know mpd very
well. Likewise, if there is a long option for -n it would be great to use that
instead.)
bq. >
bq. > Of course, this will require setting the command's environment
variables appropriately. But, this should let us completely eliminate the
startmpd.py script!!!!! Yeah! Less things to maintain!
Ok. Switched to no-executor. I directly specified the variables during
string/command construction, so there isn't a need to set environment
variables. Got rid of startmpd.py/startmpd.h =)
bq. On 2012-04-24 21:45:19, Benjamin Hindman wrote:
bq. > frameworks/mpi/nmpiexec.py, line 199
bq. > <https://reviews.apache.org/r/4768/diff/3/?file=103693#file103693line199>
bq. >
bq. > s/slot/mpd
bq. >
bq. > Same as above, is there something we can/should set when we launch
the mpd?
There doesn't seem to be anything to set for memory when launching the mpd...
bq. On 2012-04-24 21:45:19, Benjamin Hindman wrote:
bq. > frameworks/mpi/nmpiexec.py, line 28
bq. > <https://reviews.apache.org/r/4768/diff/3/?file=103693#file103693line28>
bq. >
bq. > I know this wasn't the style of the codebase you've inherited, but
I'd like spaces around all operators please. For example, this line should read:
bq. >
bq. > print "Got " + str(TOTAL_TASKS) + " mpd slots, running mpiexec"
bq. >
bq. > It looks like you've already done this with some of the code you've
added (which is awesome!), but please clean up all the code. Thanks!
Done. I converted most string operations to use %, roughly following the Google
style guide for Python.
bq. On 2012-04-24 21:45:19, Benjamin Hindman wrote:
bq. > frameworks/mpi/README.txt, line 76
bq. > <https://reviews.apache.org/r/4768/diff/3/?file=103692#file103692line76>
bq. >
bq. > Does nmpiexec work out of the box without changes (not included in
this review).
bq. >
bq. > Also, let's change the names from "nmpiexec*" to "mpiexec-mesos*"! ;)
bq. >
bq. > Finally, just pass host:port, no need for the 'master@' prefix.
Yup, nmpiexec works.
bq. On 2012-04-24 21:45:19, Benjamin Hindman wrote:
bq. > frameworks/mpi/nmpiexec.py, line 196
bq. > <https://reviews.apache.org/r/4768/diff/3/?file=103693#file103693line196>
bq. >
bq. > s/slot/mpd
bq. >
bq. > Also, do we want to set the '--ncpus' option on the actual mpd that
we launch on a Mesos slave?
Added --npus to slave mpd calls.
bq. On 2012-04-24 21:45:19, Benjamin Hindman wrote:
bq. > frameworks/mpi/nmpiexec.py, lines 91-92
bq. > <https://reviews.apache.org/r/4768/diff/3/?file=103693#file103693line91>
bq. >
bq. > Use driver.declineOffer please.
Done for this and others below. Issue#188 (small bug fix for Python/C++
binding) would have to be committed for driver.declineOffer to work though.
- Harvey
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/4768/#review7179
-----------------------------------------------------------
On 2012-05-02 13:00:12, Harvey Feng wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/4768/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2012-05-02 13:00:12)
bq.
bq.
bq. Review request for mesos, Benjamin Hindman, Charles Reiss, and Jessica.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. Some updates to point out:
bq.
bq. -nmpiexec.py
bq. -> 'mpdallexit' should terminate all slaves' mpds in the ring. I moved
'driver.stop()' to statusUpdate() so that it stops when all tasks have been
finished, which occurs when the executor's launched mpd processes have all
exited.
bq. -startmpd.py
bq. -> Didn't remove cleanup(), and added code in shutdown() that manually
kills mpd processes. They might be useful during abnormal (cleanup) and normal
(shutdown) framework/executor termination...I think. cleanup() still terminates
all mpd's in the slave, but shutdown doesn't.
bq. -> killtask() stops the mpd associated with the given tid.
bq. -> Task states update nicely now. They correspond to the state of a
task's associated mpd process.
bq. -Readme
bq. -> Included additional info on how to setup and run MPICH2 1.2 and
nmpiexec on OS X and Ubuntu/Linux
bq.
bq.
bq. This addresses bug MESOS-183.
bq. https://issues.apache.org/jira/browse/MESOS-183
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. frameworks/mpi/nmpiexec.py a5db9c0
bq. frameworks/mpi/README.txt cdb4553
bq. frameworks/mpi/startmpd.py 8eeba5e
bq. frameworks/mpi/startmpd.sh 44faa05
bq.
bq. Diff: https://reviews.apache.org/r/4768/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq.
bq. Thanks,
bq.
bq. Harvey
bq.
bq.
> Included MPI Framework Fails to Start
> -------------------------------------
>
> Key: MESOS-183
> URL: https://issues.apache.org/jira/browse/MESOS-183
> Project: Mesos
> Issue Type: Bug
> Components: documentation, framework
> Environment: Scientific Linux Cluster
> Reporter: Jessica J
> Assignee: Harvey Feng
> Priority: Blocker
> Labels: documentation, mpi, setup
>
> There are really two facets to this issue. The first is that no good
> documentation exists for setting up and using the included MPI framework. The
> second, and more important issue, is that the framework will not run. The
> second issue is possibly related to the first in that I may not be setting it
> up properly.
> To test the MPI framework, by trial and error I determined I needed to run
> python setup.py build and python setup.py install in the
> MESOS-HOME/src/python directory. Now when I try to run nmpiexec -h, I get an
> AttributeError, below:
> Traceback (most recent call last):
> File "./nmpiexec.py", line 2, in <module>
> import mesos
> File
> "/usr/lib64/python2.6/site-packages/mesos-0.9.0-py2.6-linux-x86_64.egg/mesos.py",
> line 22, in <module>
> import _mesos
> File
> "/usr/lib64/python2.6/site-packages/mesos-0.9.0-py2.6-linux-x86_64.egg/mesos_pb2.py",
> line 1286, in <module>
> DESCRIPTOR.message_types_by_name['FrameworkID'] = _FRAMEWORKID
> AttributeError: 'FileDescriptor' object has no attribute
> 'message_types_by_name'
> I've examined setup.py and determined that the version of protobuf it
> includes (2.4.1) does, indeed, contain a FileDescriptor class in
> descriptor.py that sets self.message_types_by_name, so I'm not sure what the
> issue is. Is this a bug? Or is there a step I'm missing? Do I need to also
> build/install protobuf?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
