On Thu, 21 Jul 2005, Guy Teverovsky wrote: > I'm trying to find a smart way to centrally manage the Mosix/OpenMosix > cluster nodes and after some google-ing have come up with Oscar which > looks like a good candidate for the task (http://oscar.sf.net > <http://oscar.sf.net/> ). I remember that the name has come up at this > list, so some questions: > > - Does Oscar play well with OpenMosix (or Mosix) kernel?
Did not try, but from the experience I have so far, it does not play well with the regular kernel. Not trying to blame any specific part of the system, since I have not investigated this, but I get a SIGBUS on a running program, when I compile the source code used to generate the executable (over NFS to a netapp raid). No, I do not actually delete the old executable - I use a system of symlinks, keeping the last 5 executables compiled. AFAIK, even if I did delede the original executable file, Linux should not have allowed me to REALLY do it, because the executable file was used, so it should have only appeard as deleted, but really deleted when the execution finished only. > > - My understanding is that Oscar provides userland tools for > running distributed jobs on the cluster nodes without the need for > *Mosix kernel, so this boils up to: what are the advantages of adding > *Mosix kernel into the mix ? (performance ? capabilities ?) > It currently provides openPBS: it helps you send jobs to various nodes, according to groups and queues. You simply start the jobs on different machines, as opposed to mosix which may start them on one machine, and migrates them. The system is fairly simple to use - simpler than condor (closed source, you can get the source if you ask nicely). Suffisticated enough - better than jobd in that sense. More stable than condor - condor's server had a tendency to crash rather often. Another problem we had with condor: condor is meant for CPU-cycle stealing, that is for working on stations with a local user, who has privileges over remote users (condor users). In a situation with a cluster intended for HPC, where you also wish to enable direct login (ssh/rsh/rlogin...) and not just sending of jobs, this means that any job sent via condor gets a low priority. If there is a condor job running, and an rsh job running, the condor job is suspended. In openPBS it is the other way round: the openPBS queue manager ignores the fact that CPUs may be taken by independent jobs. It sneds jobs to any CPU without an openPBS job on it. So people have an incentive to use the openPBS system. Alternatively, they can decide they will be satisfied by half a CPU, because the computation is urgent. They will hurt the prformance of an openPBS job, in order to get the computation begininig at the same time. You can prevent the two problems by lowering the default nice value of jobs outside the batch queuing system. The problem/bug with openPBS: if you send more jobs than free CPUs, and you do it too fast, the jobs without CPUs just disappear from the queue and do not get executed. I am told this does not happen if you wait for 2 seconds between subsequent job posts. Of course, you cannot garrantee that in a system with many users. It also makes the sending very long. Still waiting for a solution. Regarding mosix/openmosix: After several years of considering and convincing regarding this issue, I decided to avoid installing any of them. It is not worth it. I send batch jobs via the batch queueing system, and parallel jobs use pvm or mpi (which can be integrated with openPBS). So far, I have not seen anything that needs more than that, and it is not worth the risk of running something outside the mainstream in the kernel, which means it is far less tested, and interacts closely with the program. > - Long shot: the documentation states that only RHEL 3 Update > 2/3 are supported. Any chance someone witnessed or deployed it on RHEL3 > Update 4 ? I can check this when I am back in Israel (next week). But I doubt we have installed something advanced. > > - A friend of mine mentioned an alternative called Clip > (appears that Team have deployed it), but I seam not to be able to find > anything about it on the web. Any pointer would be helpful. > Clip is closed source, given only to those in commercial contract with Amnon Barak's team (MOSIX) or as a favour from this team. I am told OSCAR management is easier than clip version 1 (or was this 1.2?). I did not get a chance to compare it to CLIP 2.0, which is the new, closely-kept version - did not see any site with it. > > > I am the stage that I have the luxury to build the whole thing from > scratch, so any pointers, tips, alternatives, experiences are welcome. Watch for NFS troubles - what do you intend to use for storage? Build the network according to the kind of HPC you are planning. How do you intend to monitor network usage? > > The intension is to have an HPC cluster for long running tasks that are > heavy on CPU. > > > > TIA, > > Guy > > Orna. -- Orna Agmon http://ladypine.org/ http://haifux.org/~ladypine/ ICQ: 348759096 ================================================================= To unsubscribe, send mail to [EMAIL PROTECTED] with the word "unsubscribe" in the message body, e.g., run the command echo unsubscribe | mail [EMAIL PROTECTED]
