Re: Question regarding access to different hadoop 2.0 cluster

2013-11-06 Thread Bobby Evans
Suresh, You are correct I did not explain myself very well. If one of the name nodes has hardware failure. In order to avoid updating the configs for every single service that talks to HDFS you have to make sure the replacement box appears to the network to be exactly the same as the original.

Re: Question regarding access to different hadoop 2.0 cluster

2013-11-06 Thread Todd Lipcon
We've discussed a few times adding a FailoverProxyProvider which would use DNS records for this. For example, you'd add a SRV record (or multiple A records) for the logical name, pointing to the physical hosts backing the cluster. I think it would help reduce client-side configuration pretty

Re: Question regarding access to different hadoop 2.0 cluster

2013-11-05 Thread Bobby Evans
But that does present a problem if you have to change the DNS address of one of the HA namenodes. It forces you to update the config on all other clusters that want to talk to it. If you only have a few clusters that is probably not a big deal, but it can be problematic if you have many

Re: Question regarding access to different hadoop 2.0 cluster

2013-11-05 Thread Suresh Srinivas
On Tue, Nov 5, 2013 at 6:57 AM, Bobby Evans ev...@yahoo-inc.com wrote: But that does present a problem if you have to change the DNS address of one of the HA namenodes. Not sure what you mean by this? Do you mean hostname of one of the namenode changes? If so, why is this is not a problem for