> > > > What I'm trying to achieve: > > > > > > - NAS1: contains the s3ql filesystem (which is s3ql-mounted by a > linux > > server, through sshfs - but not 24/7) > > NAS1 is a computer, or a directory? What does it mean for the s3ql > filesystem to be s3ql-mounted by a linux server? Is NAS1 that Linux > server? Is sshfs the S3QL backend, or are you accessing S3QL via sshfs? > > NAS1 is indeed a "computer" (network attached storage, based on linux), which allows CIFS/NFS/SSHFS exposure on the LAN. It hosts the s3ql filesystem as a "local" backend towards my "real" linux server: he sshfs's the NAS1 directory which contains "myfsdata", after which I can mount.s3ql this local backend "myfsdata". Unfortunately, the linux on the NAS is "closed", s3ql cannot run on it, however legacy linux commands are available (eg. rsync).
> > - NAS2: contains a "scp -pr" of the NAS1 contents as a baseline. > > Of what exactly? The s3ql file system? Or the backend of the s3ql file > system? > Same remarks as above: NAS2 is a closed "linux" NAS system. The idea is that NAS2 is geographically separated from NAS1 (over the WAN), but for initial data load, we used local transfer of the "myfsdata" repositories (this off course without having mounted the filesystem). As NAS2 is now remotely stored, and I don't have a "remote" s3ql mounting capability (by having a remote linux available that is), so I analysed my synchronisation possibilities: 1) either I "sshfs" mount the remote NAS2 on my local linux server, mount.s3ql this backend and rsync between the local s3ql and the remote s3ql. 2) either I "rsync" natively on NAS level (thus rsyncen the two "myfsdata"s) without having them mounted anywhere. 3) ? > > I use the following command on NAS1 to rsync the delta's made by linux > > server, after which (off course) the s3ql filesystem got umounted > properly > > - the delta should be around 2GB. > > I don't understand. What delta's? How are they made? > Let's say the "myfsdata" on NAS2 is a weekly "snapshot". If I add data (in my example 2GB) to the s3ql file system locally mounted, I'd like to sync the differences (and only the differences) towards the "remote" s3ql filesystem - like in the 2 cases above: either *through *the filesystem, or *outside *the filesystem (eg. "myfsdata"), but *as network optimised as possible*. In case #1 by having the remote s3ql backend mounted through sshfs: rsyncing through the filesystems requires heavy network lifting (as for each rsync checksum, the data has to be transferred anyway), case #2 with native rsync doesn't seem to work either (for not having to transfer all the data over the network). > > 2. Is there a "smarter" way to synchronise *offline *s3ql file > systems > > (so *without* having to mount them)? > > Yes, but the procedure depends on the backend. With the local backend, > use rsync. With the remote storage backend, you need to use a tool > specific to that service. > > I think I understand the confusion: with NAS1 & NAS2, I got 2 local backends, however NAS2 is on a "remote" location, and I'd like to synchronise both once in a while (as a remote backup). The "remote" s3ql should be mountable over sshfs in case of file recovery, but should not be used to synchronise. The following image explains it graphically: http://nl.tinypic.com/r/29o4vp2/8 Let me rephrase the question differently: If I watch the original "myfsdata" directory, and looking at nodes like: myfsdata/s3ql_data_/188/s3ql_data_188998 Is there any reason why rsync *thinks* they have changed with having added only 2GB of data inside the file system. The only possibility I see is that umount.s3ql'ing the filesystem writes something in each of the nodes, but that is too far fetched I hope. Timestamps should not matter, as I'm forcing rsync to checksum. -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
