----- Forwarded message from Mark Hahn <[EMAIL PROTECTED]> ----- From: Mark Hahn <[EMAIL PROTECTED]> Date: Mon, 8 Nov 2004 22:51:18 -0500 (EST) To: Chris Sideroff <[EMAIL PROTECTED]> Cc: [EMAIL PROTECTED] Subject: Re: [Beowulf] torus versus (fat) tree topologies
> I looking for some general comments about the pros/cons of torus (2D/3D) > and (fat) tree network topologies, for HPC of course. latency, of course (bandwidth is easy). > My investigation thus far has led me to believe that one reason a torus > topology might be better is because it eliminates the need for a switch. what do you have against switches? *someone* has to do the switching, so why not dedicated hardware? think of it this way: in an n-dim mesh, you only get 2^n-1 outputs per node, and only half-ish of those will make progress, so each hop gives you only a small amount of "routing computation". compare this to something like myrinet, where switching is done by 16 or 32x crossbar units. not only fewer hops, but think also of the wiring - you don't really want connectors for all those mesh links, do you? not to mention tranceivers for each you save a lot of pain by putting most or all the switching into a dedicated box which can etch all those links into a nice, reliable board with simpler PHY's and higher branching factor at each hop. > On the other hand fat tree interconnects seem to dominate the > larger(est) clusters out there, why? practical factors, I believe: latency and packaging. bisection bandwidth seems to appeal to a lot of people, but I'm not sure why. n-dim tori have good BB which scales with size (and dimension). if you really had code which was mostly near-neighbor, it's pretty easy to see mesh-like interconnect making sense - you'd have little need for much routing intelligence. most codes I see have very non-NN patterns (need nontrivial routing) and often do really care about latency. and don't forget that switch-vs-mesh is not an either-or: you can do interesting things to combine them (FNN, grid-of-switches, etc.) regards, Mark Hahn. _______________________________________________ Beowulf mailing list, [EMAIL PROTECTED] To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf ----- End forwarded message ----- -- Eugen* Leitl <a href="http://leitl.org";>leitl</a> ______________________________________________________________ ICBM: 48.07078, 11.61144 http://www.leitl.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE http://moleculardevices.org http://nanomachines.net ------- To unsubscribe, change your address, or temporarily deactivate your subscription, please go to http://v2.listbox.com/member/[EMAIL PROTECTED]
pgp31t0ya8J0G.pgp
Description: PGP signature
