Hi Gregory, This is way outside the design parameters of HDFS. It may work, but you are very likely to run into issues, and I don't think anyone would recommend this as a solution. More reasonable would be a HDFS cluster spanning two datacenters within the same metro area (1-2ms latency), but even that isn't at all common.
For cross-site redundancy of HDFS, the usual solution is using distcp to make periodic backups between separate clusters. -Todd On Wed, Sep 16, 2009 at 12:09 AM, Touretsky, Gregory < [email protected]> wrote: > Hi, > > Does anyone have an experience running HDFS cluster stretched over > high-latency WAN connections? > Any specific concerns/options/recommendations? > I'm trying to setup the HDFS cluster with the nodes located in the US, > Israel and India - considering it as a potential solution for cross-site > data sharing... > > Regards, > Gregory Touretsky > Intel IT - Strategic Solutions and Architecture > Systems Analyst > gregory.touretsky AT intel.com > (+) 972-4-865-6377, Fax: 04-865-5999 > iNET: 465-6377, M/S: IDC10-2.3 > > > --------------------------------------------------------------------- > Intel Israel (74) Limited > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies. >
