On Tuesday 21 November 2006 20:36, Michael Edwards wrote: > Comments inline: > > On 11/21/06, Saravana Kumar <[EMAIL PROTECTED]> wrote: > > On Monday 20 November 2006 20:01, Michael Edwards wrote: > > Job ID Username Queue Jobname SessID NDS TSK Memory > > Time S Time > > -------------------- -------- -------- ---------- ------ ----- --- ------ > > ----- - ----- > > 66.hulk.gai.net user workq TestJob 6571 3 1 -- > > 10000 R 00:02 > > node2+node1+node3 > > That output only says that you have successfully reserved 3 nodes. > That is, that job has the resources of node 1 2 and 3 available. > Torque does not know if they are being used at all, that is something > you would have to check with ganglia. Open a web browser and go to > http://localhost/ganglia and you will get a lot of performance > information about the cluster. Run the job and watch the cpu graphs > of nodes 1,2 and 3. I strongly suspect you will only see one of them > do anything significant. Yes. You are right. The code is using only one node's cpu. While other nodes are reserved but idle(not used). > > > I don't know why the queue(hp) that i created doesn't execute the job in > > all hosts. > > > > > Do you want one identical copy of the sas script to run on each node? > > > > I don't understand your question. I want the sas job to run in one > > node(to keep file io in one node) but use all the free nodes' cpu cycles > > which i think will improve the performance and decreases the code running > > time. > > Unless SAS is much smarter than I think it is, this is unlikely to > happen with out a lot of work on your part. Clusters (of any sort, > this is not an OSCAR problem) do not divide up work automatically or > smartly as you (and lots of other people, don't feel bad) want them > to. The way programs work is too different for this to be done in any > sort of intelegent way I have heard of. There are some cluster > programs (openMosix, kerrighed) which will try to do this to a very > limited degree, but they still require very specially designed > programs to gain any significant advantage. Kerrighed has an Oscar > project called SIS Oscar which is working on a 5.0 based version, I > believe, if you want to try it. Will try this when i have spare time. > > Anyway, the take home point is that to spread across nodes in the way > you are describing requires specially writen software, which is very > rare. There are basically two ways clusters are used. Custom writen > (usually) code which uses MPI or some other communication protocol to > split up the work among the nodes in a smart (custom) way. This is > called parrallel programing. The other way is to have many > non-parrallel jobs running on the cluster at the same time. You could > run your SAS script on all 3 nodes for example, walking a parameter > space. This is called serial programing, and still usually requires > some clever scripting to deal with the peculiarities of cluster > computing. > > I am neglecting the type of automatic parallelizaton which you are > describing because I see it as a special case of parallel programing > which, as I mentioned, basically doesn't exist. > > Others may disagree with me on this point, so we'll see if they pipe in. > > > My sas jobs have lot of io. Is it better to run the sas job in one node > > and use other node's cpu cycles or can the sasjob itself be split to > > multiple disks and cpus with the script(not clear on this). > > > > The whole point of installing this cluster suite was to minimize the time > > taken by sas jobs. > > I'll poke around and see if SAS has a parallel programing module, I > know matlab does, but they still would require extra coding to use. > > The cluster can still help improve your overall processing time, but > it will probably be in a serial mode rather than a parallel mode. I wanted to implement the cluster to overcome the number of jobs run in parallel and cause iowait issues in my servers.
May be i must look out for solutions to increase the io speed(clustered file systems like coda/ GFS). Regds, SK ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys - and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Oscar-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/oscar-users
