On 06/06/11 08:22, elton sky wrote:
hello everyone,
As I don't have experience with big scale cluster, I cannot figure out why
the inter-rack communication in a mapreduce job is "significantly" slower
than intra-rack.
I saw cisco catalyst 4900 series switch can reach upto 320Gbps forwarding
capacity. Connected with 48 nodes with 1Gbps ethernet each, it should not be
much contention at the switch, is it?
I don't know enough about these switches; I do hear stories about
buffering and the like, and I also hear that a lot of switches don't
always expect all the ports to light up simultaneously.
Outside hadoop, try setting up some simple bandwidth tests to measure
inter-rack bandwidth: have every node on one rack try and talk to one on
another at full rate.
Set up every node talking to every other node at least once, to make
sure there aren't odd problems between two nodes, which can happen if
one of the NICs is playing up.
Once you are happy that the basic bandwidth between servers is OK, then
it's time to start worrying adding hadoop to the mix
-steve
