Hi Igor, I've tried running some jobs on a MacPro with Dual quad core processors which are hyperthreaded (the OS recognizes 16 cores, also, I'm still using Trilinos 9.0.3 and FiPy 2.02). I typically see pretty linear scaling when running jobs up to the number of physical cores, but I see no improvement, or even an increase in total job time if I go above that limit (e.g. 8 cores is ~half the time of 4, but 16 is no better or perhaps worse than 8) for a 256x256 grid.
For larger systems (1024x1024) I've seen pretty good performance on our cluster at 64 or even 128 cores (with 8 cores per node and no hyperthreading there). Here, the performance improvement seems to drop off more slowly with an increase in cores compared to the transition from linear to no improvement when attempting to use the hyperthreaded cores on our desktop machines. Based on your experience, it sounds like the aspect ratio of the system can be pretty important which is interesting as my problem will probably require non-square domains. I can let you know when I get around to testing this out, and thanks for sharing! Eddie Schwalbach On Sat, May 22, 2010 at 2:07 PM, Igor <[email protected]> wrote: > > Hi, > > You mention "8 cores": is that 4 dual-core CPUs? Do you use "mpirun > > -np 8 ..." when you run your code? > This is 1 quad-core (each core hyper-threaded) CPU. Yes, I use "mpirun > -np 8". And I see 8 python processes with 100% load under top command. > Actually, if there are 2 cores and I use "mpirun -np 8" there will be > also 8 python processes, but 25% CPU load for each process. > > > Again, thanks Igor (I owe you a beer). > You are welcome :-) > > Cheers, > Igor. > > PS: > What I didn't test is the other mpirun keys, or even mpirun-like > commands. I'm not sure what communication hardware is used with "-np" > key. I know that it is a standard key used for instance to send the > job to the cluster with no special preference to which nodes > individual processes are located. But, I also know, though don't > remember now, that if I'd like to have my job on one node, i.e. not to > use ethernet, etc. for communication between processes but the > motherboard, the keys (or even command) would be different. This I > will ask our administrator next week. > > >
