Hi Prentice,
I am using Matalb with SGE effectively on a number of our clusters. It
does require SSH keys and configuring a simple parallel environment in
grid engine.
You could automatically create a key for each user, if you have shared
home directories, by sticking something like this in a script in the
/etc/profile.d/ directory:
if [ ! -f $HOME/.ssh/id_rsa ] ; then
/usr/bin/ssh-keygen -P "" -f $HOME/.ssh/id_rsa
/bin/cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys
/bin/chmod -R go= $HOME/.ssh
fi
Bonus points if you sed in a limit to your node IPs and hostnames, like
so, in the authorized_key file:
from="head.node.foo.edu,10.0.0.1" ssh-rsa AAAABNhzasC1ycxxxxxxxxxxxxxx
Matlab does come with distcomp, aka MDCE, a manager/worker model, which
you must start by hand on each node. I do not recommend it, because it
does not provide much, if any, support for multiple simultaneous users
or resource management. I'd say skip the MDCE job manager and use SGE.
Cheers.
On 01/20/2012 05:32 PM, Prentice Bisbal wrote:
> On 01/19/2012 05:30 PM, Alex Chekholko wrote:
>> On 01/19/2012 11:19 AM, Prentice Bisbal wrote:
>>> Does anyone on here have experience configuring SGE to work with
>>> MATLAB's Distributed Computing Environment (MDCE)?
>>>
>>> SGE is not natively supported by MATLAB, but can use their 'generic
>>> scheduler' facility. They provide all of the scripts needed, and even
>>> include a
>>> template for the PE. I've followed all the instructions, but when I try
>>> to submit a parallel job using the validation function of MATLAB's
>>> parallel configuration manager, it fails.
>>>
>>> Digging into the problem, it looks like MATLAB uses the first task to
>>> start an mpd daemon, because in the MATLAB logs I see this:
>> ...
>>>
>>> I suspect this is a problem with SGE not forwarding my SSH key, or
>>> something of that nature. I require a passphrase for my SSH key, so I
>>> suspect that might be the problem. Can anyone confirm or deny that?
>>>
>>
>> In some instructions that I read for MDCS ( I think it's "MDCS" not
>> "MDCE" ), it was implied that you need passwordless SSH connectivity
>> to the nodes, because I think underneath it just uses SSH to launch
>> processes on the nodes.
>> I found this link for the installation documentation:
>> http://www.mathworks.com/support/product/DM/installation/ver_current/
>>
>
> Passwordless SSH is definitely needed, as I finally got MDCE working on
> my cluster this afternoon. None of the MATLAB documentation I found said
> anything about that. My only clue was the error message I posted
> earlier. I printed out the MDCS installation guide from the link above,
> but haven't read it yet, so I'm not sure if it's mentioned in there.
>
> MDCS and MDCE are two separate, but related things. I *think* the MATLAB
> Distributed Computing Server is a resource manager, like SGE or Torque,
> but for MATLAB, and MDCE is more of the client side of it. If you read
> through the manual for MDCS at the link above, it uses the acronym
> 'MDCE' throughout, and a lot of the environment variables used my MATLAB
> for distributed computing begin with 'MDCE_'. It's all very confusing...
>
> Prentice
>
>
> _______________________________________________
> users mailing list
> [email protected]
> https://gridengine.org/mailman/listinfo/users
>
--
Gavin W. Burris
Senior Systems Programmer
Information Security and Unix Systems
School of Arts and Sciences
University of Pennsylvania
_______________________________________________
users mailing list
[email protected]
https://gridengine.org/mailman/listinfo/users
