hello everyone, As I don't have experience with big scale cluster, I cannot figure out why the inter-rack communication in a mapreduce job is "significantly" slower than intra-rack. I saw cisco catalyst 4900 series switch can reach upto 320Gbps forwarding capacity. Connected with 48 nodes with 1Gbps ethernet each, it should not be much contention at the switch, is it?
Can anyone explain how this "slow"ness happens to me? Thanks Elton
