Also, I've tried to find results about the expected diminishing returns of
many cpus in a regular SMP setup. I assume this occurs because of the
cache coherance traffic on the shared bus. Are there some architectures
on which Linux SMP works that get around this with multiple or special
busses? Otherwise, what kind of SMP-like support is there under Linux for
machines that aren't strictly SMP? For instance the SGI Origin series
isn't an SMP architecture, but the ccNUMA support from the hardware and
a special version of Irix make it SMP-like (at least according to the
promotional literature).
I know that on a high-bandwidth backplane (e.g. Challenge or Sun's
Enterprise servers) the reasonable limit is somewhere between 16-32
CPUs; beyond that the bus contention limits overall machine performance.
Linux cannot be run on the Origin at the current time. There's a lot
of hardware documentation that's not publicly available that would be
required to do the port.
For a sense of scale in the grant proposal, I'll say that my advisor was
initially interested in the Origin series--except that SGI charges way too
much for memory and hard-drives (we need _at_least_ a terrabyte of
storage, maybe two). And the main memory should be no smaller than 32 GB,
though 64 GB is desired.
When I was working at the supercomputer institute here, when we would
buy SGIs, we would by the "frame" (that is, cpu, enclosure, minimal
memory, etc) from SGI, then go ahead & buy the drives and memory from
some place like National Peripheral. IIRC Kingston sells SGI memory,
and virtually any high-perf. SCSI drive will work (although you may
need a drive sled, and those are expensive from SGI as well).
Obviously I'm no multiprocessor guru, but I'm willing to work towards that
end--if someone can point me to information which has some nonzero
probability of being helpful, I'd be grateful. I've spent a lot of time
on manufacturer's web sites with very little to show for it, so I'm
willing to follow any new lead.
I wonder if maybe the Sparc port has been done for the Enterprise servers
with > 4 CPUs? That could be a possibility. I'm not sure if the MIPS
port runs on a Challenge - if it does that would be another possibility.
Having used them all, of course, I can say that you won't be disappointed
if you get an Origin.. ;)
-Bob
