May 10, 2009

Why Sync Is So Difficult

By JEAN-GABRIEL MORARD
GigaOm.com

http://www.nytimes.com/external/gigaom/2009/05/10/10gigaom-why-sync-is-so-difficult-12208.html


As we increasingly struggle to manage our data spread, both on our 
devices and in the cloud, even Google is going offline, for remote 
access and online storage aren’t enough. There is a need for sync with 
local duplicates, one the likes of SugarSync, Dropbox, Apple’s MobileMe 
and Microsoft’s LiveMesh are aiming to fulfill.

I worked at SugarSync for almost three years (I no longer have any 
financial ties to the company), so I know the double-edged sword of sync 
firsthand. When you sell a sync product, you sell magic. (We free the 
data from their physical devices; just forget where you last edited the 
file, it’s gonna be everywhere.) But once it’s implemented, there’s no 
magic anymore, and the engineer is left to deal with asynchrony, slow 
bandwidth, third-party applications, and file systems that have 
different semantics.

Here’s why:

Push sync is deeply asynchronous

Conceptually, you have redundant copies of your data on various devices, 
and a service that keeps the copies in sync. That means that when a 
change is made on one device, you replay the change to the others. So 
when a file is edited on one device, it needs to be updated on the 
others. In the meantime, you can’t guarantee that the old version of the 
file remains untouched on the other devices. It could be edited, moved 
or deleted, which is when conflicts arise. It’s well known that 
concurrent programming is difficult; sync is just an extreme example.

Sync matches different data models

You can’t actually sync identical duplicates of your data because they 
live on different devices. So you have to translate the data to the 
local models. File names alone don’t sync properly from a Mac to Windows 
without careful unicode transformations, so imagine what becomes of 
extended attributes, resource forks, ACLs, etc. Even if you’re not 
cross-platform, most file systems can be set to either case-sensitive or 
case-insensitive. So you have to come up with an extensible strategy to 
deal with the different models, and there is infinite testing involved.

Sync messes with third-party applications

Applications tend to misbehave in various ways with their documents. 
When a sync product attempts to update a file with a newer version from 
another device, it can’t always know whether or not the file is open for 
reading or editing. In that case, the application may become unstable. 
Syncing app data is also dangerous.

Sync is hard to test

Sync maintains redundant copies of the user’s data through incremental 
updates. The devices are in sync as long as the redundant copies are 
consistent. Developers and testers will usually assume in their testing 
that the initial state is in sync; they will make a change on one side 
and see that the other side changes accordingly. So as soon as an error 
is introduced in the system (you’re out of sync to start with), you’re 
in a non-tested scenario. The system needs to recover from its own errors.

Several of these problems are on the client side, which is why building 
a sync client is hard — even on the best sync platforms out there, such 
as Sync Services or Live Framework.

Some of these difficulties are just inherent to sync, but the technology 
is maturing. Sync so far has meant something different to everybody. 
Even the industry’s main players still sell byproducts of sync rather 
than sync itself, which then compete with backup, online storage, 
photo-sharing or music-streaming services. Sync is a great enabler for 
all these connected services, one that’s becoming central to the 
personal cloud story of more and more companies.

---------------------
Jean-Gabriel Morard is a software engineer living in Paris.

-- 
================================
George Antunes, Political Science Dept
University of Houston; Houston, TX 77204 
Voice: 713-743-3923  Fax: 713-743-3927
Mail: antunes at uh dot edu

***********************************
* POST TO [email protected] *
***********************************

Medianews mailing list
[email protected]
http://lists.etskywarn.net/mailman/listinfo/medianews

Reply via email to