- The more switches a packet has to go through, the higher the latency, so your response times may deteriorate if you cascade too many switches. Legend says up to 4 is a good number, any further you risk creating a big mess.
- The more switches you add, the higher your bandwidth utilized by broadcasts in the same subnet. http://en.wikipedia.org/wiki/Broadcast_radiation - If you have only one connection between each switch, each switch is going to be limited to that rate (1gbps in this case), possibly creating a bottleneck depending on your application and how exactly it behaves. Consider aggregating uplinks. - Bundling too many Ethernet cables will cause interference (cross-talk), so keep that in mind. I'd purchase F/S/FTP cables and the like. Here I am going off on a tangent: if your friends want to build a "super computer" then there's a way to calculate the most "efficient" number of nodes given your constraints (e.g. linear optimization). This could save you time, money and headaches. An example: maximize the number of TFLOPS while minimizing number of nodes (i.e. number of switch ports). Just a quick thought. On Fri, May 8, 2015 at 1:53 PM, John Levine <[email protected]> wrote: > Some people I know (yes really) are building a system that will have > several thousand little computers in some racks. Each of the > computers runs Linux and has a gigabit ethernet interface. It occurs > to me that it is unlikely that I can buy an ethernet switch with > thousands of ports, and even if I could, would I want a Linux system > to have 10,000 entries or more in its ARP table. > > Most of the traffic will be from one node to another, with > considerably less to the outside. Physical distance shouldn't be a > problem since everything's in the same room, maybe the same rack. > > What's the rule of thumb for number of hosts per switch, cascaded > switches vs. routers, and whatever else one needs to design a dense > network like this? TIA > > R's, > John >
