In the message dated: Wed, 12 Sep 2012 12:56:56 -0000,
The pithy ruminations from "MacMullan, Hugh" on
<Re: [gridengine users] How possible to use matlab in mutithreading mode on SGE
?> were:
=> Hi Semi:
=>
=> The real question is: how to NOT use multithreading, eh? :)
That was my first reaction as well.
=>
=> Seriously though, if anyone has figured out a solid way to integrate Matlab
multithreading, I'd
=> love to hear about it as well! It seems to automatically assume it has
access to all cores on a
=> box, so the only integration paths that I can see would involve reserving
whole nodes.
Yeah, Matlab does not play nicely with others.
In our experience, Matlab doesn't seem to grab all the cores on larger
boxes...it appears as if it tries to use something roughly like (N/2)+2
cores, so on our larger machines (12 cores), Matlab jobs seem to top
out at using 8 cores.
Of course, this still represents a major problem when SGE believes these
are single-slot jobs and queues 12 jobs on a single node!
=>
=> For us, if users want to use multiple cores they need to use matlabpool. We
'force' singlethread
=> ing for Matlab and all workers by modifying the matlab and worker scripts in
$MATLAB_ROOT/bin (w
=> e're running 2012a but should work for 2011b and probably others) as follows:
=>
=> # diff matlab.dist matlab
=> 483c483
=> < arglist=""
=> ---
=> > arglist="-singleCompThread"
=>
=> # diff worker.dist worker
=> 20c20
=> < exec "${bindir}/matlab" -dmlworker -nodisplay -r
distcomp_evaluate_filetask $*
=> ---
Hmmmm... It looks like you're using Matlab's Distributed Computing
toolbox. We're not--we simply use SGE to schedule Matlab jobs.
In our environment, we strongly encourge the use of "mcc" to
compile Matlab code into a stand-alone executable (reducing license
requirements). Matlab can create single-threaded executables*, so that's
one way that we prevent Matlab jobs from using an excessive number
of cores.
I would really, really love to see Matlab jobs (interperted and compiled)
accept a generalized command-line option to restrict the number of threads to
a given value, as opposed to the current state of "one or all". This would be
a tremendous help in using Matlab on a cluster.
=> > exec "${bindir}/matlab" -singleCompThread -dmlworker -nodisplay -r
distcomp_evaluate_filetask
=> $*
=>
=> Obviously we also would prefer to be able to integrate the multithreading
correctly with SGE. It
=> would make the "SGE Experience" nicer for our users ... code that works on
their quad-core desk
=> top would then "just work" with more cores on our bigger SGE nodes, and
startup times would be f
=> aster as well.
=>
=> Cheers, Hugh
=>
I'm also very curious about applying Reuti's suggestion in the recent thread
"How to Restrict forking?" to Matlab:
http://gridengine.org/pipermail/users/2012-September/004694.html
Mark
* We do a lot of matrix operations in Matlab, and we've seen instances
where compiled Matlab jobs that are supposed to be single-threaded
seem to use 2 threads, presumably because some of the libraries or
toolboxes are multi-threaded, but that's much less aggressive than
the unrestricted job, which uses many more than 2 CPUs.
=>
=> -----Original Message-----
=> From: [email protected] [mailto:[email protected]] On
Behalf Of Semi
=> Sent: Wednesday, September 12, 2012 7:49 AM
=> To: [email protected]
=> Subject: [gridengine users] How possible to use matlab in mutithreading mode
on SGE?
=>
=>
=> _______________________________________________
=> users mailing list
=> [email protected]
=> https://gridengine.org/mailman/listinfo/users
=>
=> _______________________________________________
=> users mailing list
=> [email protected]
=> https://gridengine.org/mailman/listinfo/users
=>
_______________________________________________
users mailing list
[email protected]
https://gridengine.org/mailman/listinfo/users
