Bonded will work but you may not see the performance you would expect. If you need >1 GBe, go 10GBe less headache and has even more headroom.
Multiple interfaces won't work. Or I should say didn't work in past releases. If you think about it, clients have to connect to each node. So having two interfaces and trying to manage them makes no sense. Add to this trying to manage this in DNS ... Why make more work for yourself? Going from memory... It looked like you rDNS had to match you hostnames so your internal interfaces had to match hostnames so you had an inverted network. If you draw out your network topology you end up with a ladder. You would be better off (IMHO) to create a subnet where only your edge servers are dual nic'd. But then if your cluster is for development... Now your PCs can't be used as clients... Does this make sense? Sent from a remote device. Please excuse any typos... Mike Segel On May 13, 2011, at 4:57 AM, "Evert Lammerts" <[email protected]> wrote: > Hi Mike, > >> You really really don't want to do this. >> Long story short... It won't work. > > Can you elaborate? Are you talking about the bonded interfaces or about > having a separated network for interconnects and external network? What can > go wrong there? > >> >> Just a suggestion.. You don't want anyone on your cluster itself. They >> should interact wit edge nodes, which are 'Hadoop aware'. Then your >> cluster has a single network to worry about. > > That's our current setup. We have a single headnode that is used as a SPOE. > However, I'd like to change that on our future production system. We want to > implement Kerberos for authentication, and let users interact with the > cluster from their own machine. This would enable them to submit their jobs > from the local IDE. The only way to do this is by opening up Hadoop ports for > the world, is my understanding: if people interact with HDFS they need to be > able to interact with all nodes, right? What would be the argument against > this? > > Cheers, > Evert > >> >> >> Sent from a remote device. Please excuse any typos... >> >> Mike Segel >> >> On May 11, 2011, at 11:45 AM, Allen Wittenauer <[email protected]> wrote: >> >>> >>> >>> >>> >>>>> * a 2x1GE bonded network interface for interconnects >>>>> * a 2x1GE bonded network interface for external access >>> >>> Multiple NICs on a box can sometimes cause big performance >> problems with Hadoop. So watch your traffic carefully. >>> >>> >>> The information contained in this communication may be CONFIDENTIAL and is intended only for the use of the recipient(s) named above. If you are not the intended recipient, you are hereby notified that any dissemination, distribution, or copying of this communication, or any of its contents, is strictly prohibited. If you have received this communication in error, please notify the sender and delete/destroy the original message and any copy of it from your computer or paper files.
