yeah, checked all those. They are post-factum, file synchronizers - can not manage directories (rename a dir => copy all the dir again). The only thing that manages directories is bazaar but it chokes after 8-9Gb of data (out of memory). Anyway seems i need to separate filesystem metadata (filenames, directories etc) from actual raw datafiles. e.g. fancy hierarchy of hardlinks, pointing to flat-dir files. And apply separate syncs to those two. But should think it out first.
On Thu, 28 Mar 2013 10:29:30 +0100 Nils Breunese <[email protected]> wrote: > svilen wrote: > > >> Nothing couchdb, but bup (https://github.com/bup/bup/) might be > >> the solution for you. > > will check it up, thanks. > > > >> What did you use before? rsync? > > rsync , lftp, manual cp -ru .. > > > >> What's the nature for the problem? > > well, the usual computer-geek one: 3 machines with copies of same > > huge file hierarchy (500G) somewhere. Anything changed on any one > > machine should sooner-or-later propagate to others. "worst" kind of > > changes are renaming of directories, splitting one dir into two, > > etc. filesystem-level meta-changes. Nothing that i know tracks those > > (except bazaar and maybe mercurial) although it's simple. no need to > > be fast or immediate, it's just me changing things manualy here or > > there. > > setting up special ocfs2 partitions is not an option. > > If it were two machines, you could use Unison, but since you have > three and apparently setting up a network filesystem is not an > option, you might want to take a look at csync2: > > http://linuxaria.com/howto/csync2-a-filesystem-syncronization-tool-for-linux?lang=en > http://oss.linbit.com/csync2/ > > You could also use csync2 with lsyncd if you need more instant > updates. > > https://www.axivo.com/community/threads/lightning-fast-synch-with-csync2-and-lsyncd.121/ > > Nils.
