Hi Daniel, > > We'll move our GigE structure to an InfiniBand 4X DDR one ( > prices have dropped quite a bit ). Also we'll build on AMD > Opteron up to 4 or 8 cores. > > In case of 8 cores: > > A 4 socket dual-core solution *must* scale better than > a 2 socket quad-core one, that is talking about memory > bandwith ( nearly double ). > On the other hand, the Hypertransport links on Opteron > 2000/8000 series theorically rated at a 8 GB/s per link, so > that would be as equal as 4X SDR Infiniband... > > A configuration like: > > 2 PCs with 2 socket and 2 dual-core Opterons > linked together with Infiniband 4X DDR ( 8 cores ) > > Should perform as: > > 1 PC with 4 socket ( dual-core ) Opteron based. > > Saving cost on Infiniband hardware. >
As always, depends on the code. I saw cases where it was better to have more servers and less CPUs per servers, and cases that it was the opposite. > When maximizing cores per node, reducing network > connections and network protocol overhead and considering > Opteron memory architecture... > is 8 ( 4 sockets * 2 cores ) an adequate number or a 4 ( 2 > sockets * 2 cores ) is better? > > Also onboard memory InfiniBand HCAs must perform better than > memory-less ones, that is... but how much? any real numbers out there? > No, the mem-free HCAs provide the same and in some cases if better performance than the onboard memory HCAs. Even more, the mem-free HCAs architecture is more advanced and provided extra goodies. There is a white paper on Mellanox web site that cover the mem-free architecture and performance comparison between mem-free and the onboard memory HCAs. If you will not be able to find it, let me know and I will send you a link. Gilad. _______________________________________________ Beowulf mailing list, [email protected] To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
