On 16 July 2010 17:15, Daniel Pittman <[email protected]> wrote: > Jake Anderson <[email protected]> writes: > > On 15/07/10 16:14, Daniel Pittman wrote: > > [...] > > > We cant be the first people to come across this "branch office" scenario. > > Nope. Lots of people have, and wished there was a good solution, but it is > a > *really* hard problem. The difficulty curve in fixing it looks like a > backward L, basically: the most trivial bit is trivial, then the problem > more > or less instantly gets insanely hard. >
I think it's important to mention why it's hard: the speed of light is constant, so as distance increases, latency increases. The longer it takes to round trip, the more time you spend waiting for confirmation that your "atomic" operation (eg writing new data) has committed at both ends, and the more chance you have of conflicts arriving when both sides try to write (as your window of opportunity is now wider). Add to that the difficulties in sequencing events in a distributed system (i.e. how do both ends know the other end's clock is accurate enough (but then see also vector clocks)) and you see that suddenly there's a whole bunch of expensive problems to solve. So people work around it by sacrificing some of the things they need, like atomicity, or consistency, or currency; this is what they talk about in these newfangled "NoSQL" cloud storage systems. The less you need to write, or check (like, say, a foreign key to mainttain referential integrity) the faster you can write and replicate this data. If you don't cae that the immediate data is inconsistent at one end, but will be "soon", then you don't have to wait for the write to sync. But sometimes you can't make those sacrifices, such as in the case of trying to maintain POSIX filesystem semantics. The cloudy things like Google's GFS aren't POSIXy at all for just this reason. They like appends, but not overwrites, for example. Hope this helps. (Understanding, that is -- I know it doesn't help solve anything.) -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
